News & Ideas

Fast Forward: Utilities, the Smart Grid, and Predictive Data Science

For over 30 years, I had the privilege – and significant challenge – of being a change agent at utilities.

I had the privilege of leading new initiatives – the adoption of new technology, launching new products and services, and finding new opportunities – to drive profitable revenue, customer satisfaction and greater operational efficiency. From my vantage point today as the CEO of a predictive data science company, I find myself in much the same position, helping utilities understand and embrace change – and capitalize on it.

Not that utilities have much choice. Distributed energy resources are disrupting how they manage the grid, challenging them to gain visibility and control over highly variable generation assets they don’t own. And pressure to modernize the grid – while limiting price increases on consumers – is putting operations and maintenance (O&M) costs in the cross-hairs, and also forcing utilities out of their comfort zone to develop new business models and find new revenue streams.

Data sits at the intersection of each of these new demands and opportunities. Those utilities that can master data – pouring in by the terabyte from smart meters, and from sensors across the grid – will innovate and flourish. Those that can’t will struggle.

Fortunately, utilities have an ally in this critical endeavor in predictive data science, a technology-based approach to data that, when combined with the power of platform computing, delivers insights and a level of precision inconceivable even five years ago.

Smart Grid Realities: Opportunity for Innovative Change

The rise of distributed energy resources – from on-site renewables to storage to demand response – is forcing utilities to make an evolutionary leap toward becoming “distribution system operators.” In this incarnation, a utility’s primary role is to facilitate the distribution network – including the sources of power feeding into it – a terminus few imagined when deregulation first decoupled a utility’s generation and distribution assets.

Making this shift requires a more agile distribution network, one guided by the data. But gathering data by the terabyte doesn’t make grids any smarter – acting intelligently on that data does.

Predictive data science is an enabler of intelligent action. For example, it can be used to forecast demand at every individual service delivery point in a network and, in turn, aggregate those individual predictions into a precision demand forecast for an entire service territory. One large utility we work with had developed its demand forecasts using an annual report of historical usage. Now, armed with predictive data science, the company runs 5 million individual forecasts daily to predict demand-response events – ten for each of their 500,000 customers participating in the utility’s demand response programs – resulting in event forecasts that are 30-40% more accurate. More accurate event forecasts mean having to inconvenience fewer customers to deliver the needed DR resource – a win for both the smart grid and customer satisfaction.

Oh, and did I mention it takes them all of 30 minutes to do it? Plus, the resource performance is validated in real time following an event.

This step-change improvement in forecasting accuracy, turbocharged by the enormous processing power of platform computing, is what the smart grid is all about. It is also the kind of dramatic improvement needed to remove the uncertainty surrounding distributed energy resources and their impact on a utility’s ability to meet its fundamental mandate – to provide safe, reliable, affordable, clean power to consumers. 

Of course, most utilities would jump at the chance to gain these step-change improvements. So, why don’t they? What’s holding them back? I see three main reasons:

  1. First, it’s hard to see around the corner during a paradigm shift. It’s very difficult for utility executives to imagine what is possible today. Predictive Data Science coupled with Platform Computing is a phenomenon of right now and is without antecedent: the combination blows away anything thought possible even a few years ago. I know, because I have a front row, center seat on all of it. Which brings me to my second point.
  2. Many utilities aren’t reaping the rewards of data science, because they let the perfect get in the way of the good. I’ll let you in on a little secret: predictive data science is a “learn by doing” field. That’s why I am a huge advocate of discovery-driven planning. Utilities have to get their feet wet with predictive data science so they can begin to learn from experience, see the possibilities, then knowledgeably proceed to unleashing the power of predictive analytics at scale. It’s simple: in parallel to your planning and data governance work, get going with a predictive data science proof of concept, preferably addressing a business imperative that can make a difference – like precision forecasting, outage reduction, lower O&M costs, or expanding products and services. In a recent proof of concept for a large electric utility, we used machine-learning models to redefine their basic units of work, helping the company negotiate new maintenance contracts that are saving it millions of dollars annually.
  3. Finally, I’d suggest that utilities are held back by thinking in silos, relying on specific business functions to solve their own problems. The opportunity of data science requires utilities to step back and broaden their view. Put another way, data science gets really good when it crosses business functions. We did some recent “stop meter” work for a utility and were delighted to have representatives from three different business functions at the table with us: the meter group, billing, and customer service. All three could leverage the output of the individual forecasts produced by our models to reduce waste, streamline operational costs, and better engage with customers. Armed with results across business functions, the utility is now exploring a fuller engagement with predictive data science.

So, while there are still hurdles to overcome, I do see movement – and progress. When I took over the Regulated Energy Services business at Duke Energy back in the early 2000s, we longed for a cost-effective way to better understand customers so we could increase profitable revenue and customer satisfaction. But the technology was primitive, requiring a lot of primary research and side systems – we succeeded despite the technology.

Now, we can access so much data “behind the meter” and combine it with utility data to gain a deep understanding of each customer and prospect. So much so in fact, that we can help utilities literally map out and personalize a series of offers across a portfolio of products and services by each individual customer – something we call “The Power of One.”

This approach, rooted in a new mastery of data enabled by predictive data science, will help utilities drive top-line revenues profitably, even as it helps create new efficiencies and management capabilities within the distribution network itself, resulting in bottom-line savings. As an added benefit, as profits accrue, so, too, will customer satisfaction, reflecting the consumer’s enhanced experience with a smarter grid and deeper engagement with a smarter utility.

More profits, less waste, greater customer satisfaction: That’s a trifecta benefit this industry is 100% ready for, and it’s absolutely available today.


Fill out my online form.

Thank you.
We'll be in touch soon.