Cyber Physical Systems (CPS) and the Internet of Things are examples of phrases used to describe the integration of a modern information backbone into the functioning of physical systems and infrastructures. What has received much less attention is the integration of techniques for the analysis of massive amounts of data (“Big Data”) in near real time to better operate CPS. A leading adopter of such technologies is the energy sector: the smart grid, oil and gas, and other industrial infrastructures associated with the modern energy ecosystem.

An estimated $2 trillion is being invested this decade in upgrading the power infrastructure globally to make devices in the power grid remotely machine addressable. In addition to smart electric and gas meters, many of the devices that constitute the grid are adding sensors including transformers, substations, and other power generation and control componentry.

Collectively, smart connected devices across today’s energy system generate massive amounts of information. Highly sophisticated statistical algorithms are necessary to integrate the data, create data-driven statistical models with predictive power, and extract the value and economic benefit from this otherwise incomprehensible stream of information.

The application of energy analytics is not limited to natural gas and electric utilities. Large amounts of data are being collected on oil and gas production, transportation, refining and consumption.

Data analytics and machine learning are critical to smart grid optimization and the modern distributed energy system. By holistically correlating and analyzing all data associated with the dynamics and interactions of the end-to-end power infrastructure – including real-time and predicted demand, consumption, technical and non-technical loss, weather, and generation capacity across the entire power value chain – dramatic advances in safety, reliability, affordability and energy efficiency can be realized. Other economic and social benefits include:

A recent McKinsey & Company study has shown that the economic and social benefit of smart grid analytics can exceed $300 per meter, per year. In the oil and gas sector, a unified algorithmic framework for decision making at both the supply and demand level will produce improvements in safety and new paradigms of operational efficiency.