An AI-Ready Data Lakehouse as the Foundation of Every Utility's Second Grid

For over a century, utilities have poured massive investments into the physical infrastructure. Running right alongside it is the second grid, one that is made of data. And its job is to move understanding, knowing exactly what is happening, why it's happening, and what to do about it  

Think about a simple question: Which distribution transformers lost the most energy last month, and why?  

We have the meter readings in MDM, network connections in GIS and consumer records in billing. Yet bringing these pieces together to get a reliable answer can take multiple teams and weeks of work. This is where our second grid falls short. We've built systems that hold enormous amounts of data, but the connections needed to make sense of it remain fragmented.

And as AI agents become part of utility operations, they'll need this connected, trusted data to understand what's happening across the network and support operational decisions.  

I believe it's high time we started engineering this second grid with the same discipline we've applied to the physical one.

Smart Meter data is growing. But are we putting it to work?

By August 2026, India had installed 7.24 crore smart meters across the country under various schemes.

Now think about the volume of data we're dealing with. Just 10 million meters recording consumption every 15 minutes would generate nearly a billion readings daily. And that's before we account for voltage, current, power factor and event data.

We've become incredibly data-rich. But how much of that data are we actually turning into operational intelligence?

We've invested in sophisticated sensors to collect information about our networks. Yet the shared memory needed to connect those readings with network conditions and historical patterns remains underdeveloped. That's where we can begin to understand what individual readings mean for the wider network.

And with rooftop solar, battery storage and EVs adding complexity to distribution networks, that understanding has become even more important.

Building a trusted second grid

So, how do we bring all this information together and make it useful?  

We've tried different approaches over the years. Traditional data warehouses gave us reliable reporting, but they were often rigid and expensive when handling billions of meter readings. Data lakes offered more flexibility and lower storage costs, but without proper governance, they could become difficult to trust.  

This is where I believe a data lakehouse can make a difference.  

It combines the flexibility of a data lake with the reliability of a warehouse. Data can be stored in open formats, reducing vendor lock-in, while access controls and version history make it easier to govern, audit and reuse information across reporting, analytics and AI.  

But here's where things get interesting for utilities. Our data has a complexity that generic data platforms aren't designed to handle on their own.  

A utility data lakehouse must handle six distinct requirements simultaneously:  

• You've got time series data at an absolutely massive scale.

• You have a physical network topology that literally changes every time a switch flips in the field.

• There's spatial mapping for assets and things like flood zones.

• Highly precise timestamped events like tamper alarms.

• You're dealing with decades of messy, unsearchable PDF maintenance documents.

• And finally, you need to view real-time data side-by-side with historical data.

The real challenge is connecting all six. That's where industry standards like the IEC Common Information Model (CIM) come in. They provide a common structure for linking meters, transformers and feeders, so we don't have to rebuild those relationships from scratch for every new project.

What makes the second grid AI-ready?

So, what's the ultimate goal of bringing all this data together?  

Traditional data platforms were largely built to support reports and dashboards. But an AI-ready Lakehouse has a bigger role to play. It becomes a shared memory for AI agents that need to analyse information, reason through problems and support decisions across the grid.  

To make that possible, we need five essential capabilities:  

• A knowledge graph is a digital representation of the network, connecting meters, transformers, feeders, substations and consumers. Imagine a transformer showing signs of failure. With an up-to-date network graph, an AI agent can identify which consumers sit downstream and even which ones have backup batteries.

• A semantic layer establishes consistent definitions for operational metrics, such as what counts as an outage minute or an energy loss, so engineers, reporting systems and AI agents use the same underlying logic.

• Document intelligence allows AI to search records (such as maintenance records, inspection reports and engineering notes) by meaning, helping engineers find relevant information from past incidents.

• A feature and model store provides trusted, reusable inputs for AI models, such as a transformer's loading history or a feeder's loss trends. This ensures different models can use the same reliable data signals.

• A write-back loop allows AI agents to record their findings and recommendations, along with whether operators confirmed, rejected or corrected them. This feedback creates a valuable record for improving future recommendations.

Now think about what this means for the Agentic DISCOM/Utility. If AI agents are becoming part of our digital workforce, they need the same trusted information our engineers rely on. The second grid becomes their shared memory, allowing people and AI agents to work from the same understanding of the network.

What could an AI-ready second grid change in our operations?

With a unified, AI-ready second grid in place, the way we operate our networks could change significantly.  

Imagine the grid helping correct its own map. By comparing voltage patterns across smart meters, utilities could identify incorrect network connections and flag errors in GIS records. In fact, an Electric Power Research Institute (EPRI) study demonstrated over 95% accuracy in identifying meter phases on the circuits tested. Suddenly, every smart meter becomes a potential last-mile grid sensor.  

And that's just the beginning.  

• Energy accounting could happen every morning, helping identify emerging losses within days instead of waiting for periodic reports.

• Engineers could ask questions about their networks in plain English and receive answers based on connected operational data.

• Hosting capacity assessments for new solar and EV connections could use more current network information, helping reduce lengthy connection studies.

Lakehouses are already being used in utility operations. A U.S. electricity utility has used a lakehouse to combine smart meter, GIS, outage and weather data for storm modelling. A UK-based energy supplier uses lakehouse technology to process large volumes of smart meter data. Meanwhile, a European water utility has used a lakehouse-based platform to automatically prioritise hundreds of thousands of meter alerts.

Bringing this kind of vision into everyday utility operations comes with its own challenges. A data lakehouse is a capability we must carefully build and maintain over time. It takes more than installing another technology platform.

Roadmap for building the second grid

Let's start with one of the hardest parts: data quality. If we map consumers to the wrong transformers, even the most advanced AI cannot produce reliable energy-loss assessments. Second non-negotiable aspect, you cannot breach the wall between IT and critical operational technology, like your SCADA systems. You absolutely must have strict one-way data flows, so nothing can ever send rogue commands back to the physical grid.  

We also need to watch out for rising cloud computing costs, vendor lock-in and the temptation to build technology without a clear operational problem to solve.  

So, where should we begin? I believe the best approach is to build the second grid one valuable use case at a time.  

• Step 1: Start with daily energy accounting. Why? Because it forces you to do the really hard work of joining meters to transformers right out of the gate. And it literally pays for itself through recovered energy losses.

• Step 2: Land your raw data in open formats immediately, because you can't go back and collect history later.

• Step 3: Build your grid model, like CIM, early in the process.

• Step 4: Embed privacy and governance directly into the platform. Do not leave it as an afterthought.

• Step 5: Connect your very first AI agent with a write-back loop.

• Step 6: Measure your success in actual money saved and recovered, not just in terabytes stored. Will your data grid decide your survival?

Every use case we implement should strengthen the data foundation for the next. That's how we can build the second grid progressively while delivering measurable value along the way.  

Is our second grid ready for what's coming?

Just look at the demands coming down the pipeline right now. We're talking massive solar integration, home battery storage everywhere, a literal tsunami of electric vehicles plugging in, and autonomous AI agents managing load.  

Is your fragmented, and siloed digital infrastructure really robust enough to handle that future? For most utilities right now, let's be honest, the answer is no.  

We would never allow our physical electricity networks to grow with unmapped lines, undocumented connections and assets we cannot trace. Why should we accept the same for our data infrastructure?  

I believe we need to engineer the second grid with the same discipline we've applied to the first, building a trusted foundation that our engineers and agents one can rely on