Demand Response by Design: How Analytics Transforms Programme Execution and Evaluation

Demand response has evolved from an emergency measure — a last resort to avoid rolling blackouts — to a core component of grid resource planning. FERC Order 2222, state-level clean energy mandates, and the economics of peak demand management have elevated DR from a niche programme to a strategic capability that utilities are expected to scale.

Yet the analytical infrastructure behind most DR programmes has not kept pace with their strategic importance. Customer recruitment relies on broad demographic targeting. Dispatch decisions follow conservative, rule-based protocols. Baseline estimation uses methods that are known to be inaccurate. Post-event evaluation takes weeks and often produces results that are contested. The result is programmes that consistently underperform their theoretical potential — delivering less curtailment per enrolled customer, at higher cost, with less reliability than the grid needs.

The opportunity is not to replace DR programme managers with algorithms. It is to give them analytical tools that match the complexity and importance of the programmes they run.

Customer Segmentation for Recruitment

The first analytical challenge in demand response is identifying which customers to recruit. Traditional approaches segment customers by rate class, consumption level, and sometimes by housing characteristics. This produces broad target populations with widely varying flexibility potential.

Machine learning applied to smart meter data enables fundamentally better segmentation. By analysing individual load shapes, identifying discretionary consumption patterns, and estimating the timing and magnitude of flexible load, utilities can rank customers by their likely curtailment potential before enrolling them. A customer with a large air conditioning load, consistent daily patterns, and evidence of discretionary consumption during peak hours is a better DR candidate than one with flat, baseload-dominated consumption — even if both have similar total monthly usage.

The analytical approach involves clustering customers by load shape similarity, then estimating each cluster's curtailment potential based on the difference between their typical peak consumption and their estimated baseload. This can be refined by incorporating thermostat data (where available), housing characteristics, and historical response patterns from existing programme participants.

Effective segmentation also identifies customers who are likely to respond reliably. DR programme value depends on predictable, aggregate response. A customer who curtails enthusiastically during mild events but overrides during hot days has different reliability characteristics from one who delivers consistent, moderate curtailment regardless of conditions. Predictive models trained on historical event participation data can estimate individual reliability, enabling programme managers to build portfolios with more predictable aggregate performance.

Dispatch Optimisation

Dispatch — deciding when to call DR events and which customer segments to activate — is where analytical sophistication has the greatest immediate impact on programme value.

Traditional dispatch follows conservative rules: call events when load forecasts exceed a threshold, activate all enrolled customers, and run events for the maximum allowed duration. This approach wastes flexibility by dispatching all resources simultaneously rather than sequentially, fatigues customers with unnecessarily long events, and misses opportunities to use DR for purposes beyond peak shaving — frequency response, renewable integration, or transmission congestion management.

Optimised dispatch requires three analytical inputs: an accurate short-term load forecast, a reliable estimate of available curtailment from each customer segment, and a cost-benefit model that evaluates the value of curtailment at different times against the cost of customer fatigue and programme attrition.

Machine learning load forecasts (discussed in detail in our companion article on load forecasting) provide the first input. Segment-level curtailment estimation — predicting how much load each customer group will actually reduce, accounting for weather conditions, time of day, event fatigue, and historical response rates — provides the second. The cost-benefit model is often rule-based but can be optimised using reinforcement learning techniques that learn dispatch strategies from historical event data.

The practical result is dispatch strategies that are more targeted and efficient. Instead of activating all enrolled customers for four hours, an optimised system might activate Segment A for two hours during the initial ramp, add Segment B during the peak hour, and release Segment A before the event ends — delivering the required curtailment with less total customer disruption and preserving flexibility for subsequent days.

The Baseline Estimation Problem

Baseline estimation — determining what a customer would have consumed in the absence of a DR event — is the most technically contentious aspect of DR analytics. It is also foundational: every measurement of DR impact, every settlement calculation, and every programme evaluation depends on the baseline.

The standard industry approaches — "X of Y" methods that average consumption from recent non-event days, sometimes with a day-of adjustment — are simple to implement and explain but have well-documented biases. They tend to overestimate baselines for customers who naturally reduce consumption on hot days (the very days when events are called) and underestimate baselines for customers whose consumption would have increased due to weather.

Machine learning baselines address these biases by training customer-specific or segment-specific models that predict consumption as a function of weather, time of day, day of week, and other relevant features. These models can produce more accurate counterfactual estimates because they learn the nuanced relationship between conditions and consumption for each customer.

The trade-off is complexity and transparency. Regulatory commissions and programme participants must trust the baseline methodology. A sophisticated ML model that produces better estimates but cannot be clearly explained may face adoption resistance. The practical approach is to use ML baselines for programme evaluation and planning while maintaining simpler methods for customer-facing settlement — or to develop ML baselines that are constrained to be interpretable (e.g., linear models with weather and calendar features).

Post-Event Evaluation

Post-event evaluation determines whether a DR event achieved its objectives — how much load was curtailed, whether the curtailment was delivered when needed, which segments performed as expected, and which underperformed.

Traditional evaluation is slow, often taking weeks as data is gathered, baselines are calculated, and reports are compiled. This latency limits the programme manager's ability to learn and adjust. If a particular segment consistently underperforms during afternoon events, that insight should inform the next dispatch decision — not appear in a quarterly report.

Near-real-time evaluation, enabled by streaming smart meter data and pre-computed baselines, can compress evaluation from weeks to hours. Programme managers can see preliminary performance estimates during the event itself, adjust dispatch in real time, and have validated results within a day. Over multiple events, this rapid feedback enables continuous programme optimisation.

Evaluation analytics should also assess programme health beyond individual events: participation trends, customer fatigue indicators, segment-level performance trajectories, and opt-out patterns. These metrics inform recruitment strategy, programme design modifications, and long-term resource planning.

Building the Analytical Foundation

Effective DR analytics requires an integrated analytical platform that connects customer data, meter data, weather data, grid conditions, and programme management workflows. Point solutions that address individual aspects — a segmentation tool here, a baseline calculator there — create integration overhead and make it difficult to build the feedback loops that drive programme improvement.

StratLytics' SLIQ platform provides this integrated analytical foundation for utility demand response programmes — from customer segmentation and curtailment estimation through dispatch optimisation and post-event evaluation. By connecting these analytical capabilities within a single intelligence layer, SLIQ enables the rapid, data-driven programme management that modern grid flexibility requirements demand.

Demand response is too important to the grid transition to be managed with spreadsheets and rules of thumb. The utilities that build serious analytical infrastructure around their DR programmes will deliver more reliable flexibility, at lower cost, with better customer outcomes — and that capability will become increasingly valuable as the grid's need for demand-side flexibility grows.