Astrid's Updates

Bridging the Gaps: Better Data for a DSO Future

Post by
Astrid Atkinson
Last Updated: 
July 21, 2022

As utilities seek to expand flexible, affordable, and zero-carbon power supply to every community across America, local resources like rooftop solar and battery storage (also known as distributed energy resources or “DERs”) will play an essential role. The ability to leverage DERs as flexibility resources within the context of independent system operators (ISOs) and larger-scale grid environments will be critical to achieving an efficient and decarbonized power system.  

The growing role of distributed resources is pushing grid operators to evolve their operating and business models towards that of a Distribution System Operator (DSO).

The DSO of the Future 

At Camus Energy, we’re actively supporting the emergence of the Distribution System Operator model with our customers. [If you are unfamiliar with the definition of a DSO, check out “What is a Distribution System Operator (DSO)?”].

By taking on the responsibilities of a DSO, distribution utilities can safely, reliably, and affordably operate low-carbon grids -- by tapping customer-owned resources for grid support. The distributed resources provide needed flexibility and enable investments to be made locally. They are an essential complement to utility-scale zero-carbon generation, both for operational benefits and for energy equity.

But the DSO model requires managing a more dynamic grid than ever before – and doing so in real-time. That minute-to-minute management of bidirectional power flows isn't something with which distribution utilities are familiar – requiring them to develop new capabilities.

Utility leaders rightly wonder: Is this approach feasible for utilities in every community?

The answer is yes! Utilities of all sizes and across all regions can manage a grid that seamlessly integrates local renewable generation and DERs at scale. 

This means that your community can and should embrace a mix of utility-scale and local energy resources to leverage the transition to a zero-carbon future for the benefit of your community. The first step is using existing data in new ways to provide a more complete, actionable, real-time view of the distribution grid.

The missing pieces of the grid

Real-world data is often messy, slow to gather, and incomplete. While today’s transmission environment has a high-degree of visibility and transparency, the distribution grid has a highly variable data landscape, with lots of missing pieces. 

Typical vs. Ideal Data Landscape (Visual)

The typical data landscape for distribution utilities is a far cry from the ideal landscape for managing tens of thousands of local resources. The graphic above shows how the gap exists at all levels of the distribution grid: From the feeder to the midline, all the way down to the meter and the device, gaps exist in the latency, frequency, and even measurement of key data. These gaps can compound to create a fragmented and delayed perspective that challenges a utility’s ability to maintain a reliable grid.  

To operate a more distributed grid, utilities need to bridge these gaps and pursue a robust data landscape at each level, including:

Typical vs. Ideal Data Landscape (Description)

The good news: This challenge has been tackled already in other industries, including big tech. By borrowing approaches proven in other mission-critical industries, utilities can gain comprehensive situational awareness and manage two-way grids in real-time.

Two ways to bridge the gaps 

1. Gather data from more sources 

Without clear visibility into the DERs, critical grid management tasks - including supply and demand balancing and reliable forecasting - can be extremely difficult. However, your DER data can be powerful, if you’re prepared to use it. By combining DER telemetry with advanced metering infrastructure (AMI) data, you can unlock a richer, more real-time view of your grid. 

To quote Camus’s CEO Astrid Atkinson, “every single device is exporting a hell of a lot of data!” At any given time, a DER is sending out information about its location, operations, future plans, and more. Instead of facing the overwhelming prospect of establishing utility-specific connectivity at high reliability to every single DER on your grid, it’s more tractable to source data via direct telemetry from the DER vendors or aggregators themselves, who are undoubtedly collecting the data of their deployed devices. There are also ways to develop DER programs that allow you to source this data directly from your customer. When we do this in the field, we first try to integrate through internet-facing APIs. Secondarily, if necessary, we will connect directly to the device via common communications protocols like DNP3, ModBus, Multispeak, ICCP, Sunspec, and MESA-DER to access the data. 

A common misconception (occasionally perpetuated by communications vendors) is that utilizing consumer wifi or vendor APIs is not reliable enough.  Utilities are told that they need to invest in thousands of high-reliability communications devices to reliably control DERs. This is not true – and it’s an excessively expensive approach. It is more effective to focus on the reliability of the system, as opposed to the reliability of its parts. This is a strategy borrowed from the development of the internet, the world’s largest distributed system. 

Once your team has access to the DER data, you can put it into context by integrating it alongside your AMI data. The increased granularity and latency of the DER data provides a richer picture. For example, customer-sited batteries often hide the real usage profile of solar or EV owners at a location. If you source the data from the customer or the battery itself, you can gain accurate real-time visibility into that location’s behavior. You can then use those sources of data to model the known behavior of customers with batteries or customers with EVs, and then back apply that to the AMI that's being connected from customers where there’s limited visibility. This is done in practice today and can be quickly deployed at other utilities.

2. Use machine learning to fill gaps 

Grid operators often lack high-fidelity data. This is true at meter endpoints, where metering data is reported with significant gaps or long delays. It’s especially common at mid-points of the grid, such as the distribution transformer, where few distribution utilities even collect time-varying data. In these cases, you can strategically bridge data gaps with the lower-fidelity data that you already have. 

While many existing approaches have been employed to fill gaps on the individual meter level, it’s historically been a challenge to scale them to the grid level. That’s changing.

Camus has been privileged to work with Pacific Northwest National Laboratory (PNNL) and Kit Carson Electric Cooperative to apply proven machine learning algorithms to fill gaps and fix inaccuracies in grid data. We have successfully developed an approach to apply a machine learning model of every meter to look backwards in time and fill data gaps, as well as slightly forwards in time to provide weather-informed forecasting at the meter level. When combined with SCADA data and even production meter data in some cases, this method is effective in filling shorter gaps that span hours all the way up to gaps of a day or month. 

When filling longer data gaps, the machine learning-driven strategy is not as accurate as if you were to collect 15-minute interval data from all of your meters. However, these data gaps are the reality for many distribution grids. Gaining a more comprehensive view of your grid, even if imperfect, has proven to provide a powerful boost to grid operators managing more changes in real-time.  

What does a “good” data foundation look like?

By augmenting existing data with additional sources and machine learning, you’re able to gain reasonably good data at and behind the meter. This visibility is helpful in and of itself, but it’s also valuable as an input for traditional, physics-based modeling for grid state, which otherwise would fail to run due to data gaps and errors.

To give a sense for what “good data” means in the context of coordinated DER control, we typically use the following guidelines as the necessary foundation: 

  1. Advanced Metering Infrastructure (AMI) at <60 minute intervals and <24 hour collection delay
  2. SCADA instrumentation at every substation and feeder
  3. Digitized GIS maps

We’ve found that while AMI does not need to be universally deployed, it is very helpful to have some deployment of AMI (>30-50%) out there and readable in under 24 hours.

From there, we recommend gaining DER telemetry and control capabilities via vendor APIs (as discussed above) and integrating these data sources into a single, shared data foundation.

Getting more value from your data

In every version of the future grid, safe and reliable operations will require more local insight and control, enabled by real-time data. While the real-world utility data landscape is far from ideal, there are ways to fill the gaps that don’t rely on massive communications investments. That includes accessing DER data directly from third parties and applying machine-learning to forecast and fill gaps.

With access to more comprehensive, real-time data, utilities will be able to take actions at all levels of the grid, managing operational conditions and leveraging local resources for grid support. Active real-world examples include preventing backfeed from distribution-connected solar arrays, integrating local resources into power procurement workflows, and calling upon DERs to reduce coincident peak demand.

The availability of real-time situational awareness opens the opportunity for local resources to play a bigger role in management of the broader grid, participating via programs and local markets. This enables the shift to a Distribution System Operator model and cleaner, more reliable, and more affordable electricity for all.

To learn more about how utilities can use data to transition towards a DSO Future, check out the ESIG Webinar on “DER Modeling & Distribution System Operations”.

For questions, you can reach Astrid at

Contact Us

Start your community energy transformation.