Putting the ‘Smarts’ Into the Smart Grid: A Grand Challenge For Artificial Intelligence

The phenomenal growth in material wealth experienced in developed countries throughout the 20^th century has largely been driven by the availability of cheap energy derived from fossil fuels (originally coal, then oil, and most recently natural gas). However, the continued availability of this cheap energy cannot be taken for granted given the growing concern that increasing demand for these fuels (and particularly, demand for oil) will outstrip our ability to produce them (so-called ‘peak oil’).⁹ Many mature oil and gas fields around the world have already peaked and their annual production is now steadily declining.

Back to Top

Predictions of when world oil production will peak vary between 0–20 years into the future, but even the most conservative estimates provide little scope for complacency given the significant price increases that peak oil is likely to precipitate.¹ Furthermore, many of the oil and gas reserves that do remain are in environmentally or politically sensitive regions of the world where threats to supply create increased price volatility (as evidenced by the 2010 Deepwater Horizon disaster and ongoing unrest in the Middle East). Finally, the growing consensus on the long-term impact of carbon emissions from burning fossil fuels suggests that even if peak oil is avoided, and energy security assured, a future based on fossil fuel use will expose regions of the world to damaging climate change that will make the lives of many of the world’s poorest people even harder.¹⁵

Against this background, many governments around the world have begun taking action to transition to a low carbon economy. For example, the U.K. has legislated to reduce CO₂ emissions by 80% by 2050 (compared to 1990 levels).⁸ Achieving this aim requires that the direct use of fossil fuels that we are familiar with today is almost entirely eliminated. Thus, the use of electric vehicles (EVs) and high-speed electric trains will have to become widespread in order to reduce our reliance on oil for transportation.^a Likewise, our homes and offices will have to be heated by efficient ground and air source heat pumps powered by electricity rather than existing natural gas and oil fired boilers.²² As a result (and given the general growth of the world economy), electricity demand across the world is predicted to increase by 76%, or 4,800 gigawatts (GW), by 2030 (compared to 2007 levels).²⁰ Crucially, much of the electricity needed to meet this demand will have to be generated from renewable wind, solar, and tidal sources rather than the coal and natural gas power plants that we use today.

It is this increased demand for electricity, and the requirements for its generation, that present perhaps the greatest challenge. In most countries, the electricity grid has changed very little since it was first installed, and all existing grids are predicated on the central idea that electricity is produced by a relatively small number of large fossil fuel burning power stations and is delivered to a much larger number of customers, often some distance from these generators, on-demand. The grid itself relies on ageing infrastructure (for example, 40-year-old transmission lines and transformers, and 20-year-old power stations), is plagued by poor information flow (for example, most domestic electricity meters are read at intervals of several months), and has significant inefficiencies arising from losses within the transmission (on a national level) and distribution (on a local level) networks.¹²

The vision of an electricity grid that makes extensive use of renewable generation challenges this current situation. Renewable generation is both intermittent and distributed, with the output of such generators being determined by local environmental conditions (such as wind speeds and cloud cover in the case of wind turbines and photovoltaic (PV) solar panels, respectively) that can vary significantly over minutes and hours. Thus, it will no longer be possible for supply to continuously follow the vagaries of consumer demand, but rather, the demand-side will have to be managed to ensure that demand for electricity is matched against the available supply. EVs will play a part in this, since not only do they represent a significant extra load that must be satisfied, but more positively, they also provide a distributed form of energy storage,^b which may allow the grid to smooth out this variable supply.

Furthermore, meeting the increased demand for renewable generation may require hundreds of thousands, or even millions of such generators, distributed across both the transmission and distribution networks. These generators may need to act together, effectively working as virtual power plants (VPPs), or may be located on every building across the grid, resulting in a distributed network of prosumers^c who both produce and consume electricity depending on their local requirements. Thus, unlike existing grids where electricity generally flows one-way from generators to consumers, this will result in flows of electricity that vary in magnitude and direction continuously. To guarantee the security of the network (such as, the maintenance of stable voltages and frequencies, and the reliability of supply) and to avoid the cascading failures that plague today’s grid,^d new control procedures must be devised. Indeed, the number and variability of generators will require that the grid is able to act autonomously, under human supervision but not necessarily under human control, to diagnose potential problems and self-heal.

Thus, there is a growing consensus that existing grids cannot simply be extended to address these challenges, but rather, a fundamental reengineering of the grid is required; one that envisages the creation of a ‘smart grid’, described by the U.S. Department of Energy¹² as: A fully automated power delivery network that monitors and controls every customer and node, ensuring a two-way flow of electricity and information between the power plant and the appliance, and all points in between. Its distributed intelligence, coupled with broadband communications and automated control systems, enables real-time market transactions and seamless interfaces among people, buildings, industrial plants, generation facilities, and the electric network.

What is perhaps most striking about this vision is that not only does it present many challenges in terms of power systems engineering, telecommunications, and cybersecurity, but at its core are concepts, such as distributed intelligence, automation, and information exchange, that have long been the focus of research within the computer science and the artificial intelligence (AI) communities. In this article, we argue that the smart grid provides significant new challenges for research in AI since smart grid technologies will require algorithms and mechanisms that can solve problems involving a large number of highly heterogeneous actors (for example, consumers with different demand profiles or generators with different volatilities), each with their own aims and objectives, having to operate within significant levels of uncertainty (such as, where the network conditions and the outcome of actions taken by individual entities on the grid will be more unpredictable or uncontrollable) and dynamism (where demand and supply at different points in the network will be in a significant state of flux). Hence, we illustrate how such issues arise within the key components of the smart grid—demand-side management, EVs, VPPs, the emergence of prosumers, and self-healing networks—and by showing which components and which interactions need to be smart, we provide a research agenda for this community for making the smart grid a reality.

Back to Top

Demand-Side Management

A key requirement for a safe and efficient electricity grid is that supply and demand are always in perfect balance. Now, in the day-to-day running of today’s electricity grid, this is achieved by varying the supply side in real time to match demand (increasing and decreasing the output of generators such that voltage and frequency are maintained across the grid). Hence, the idea that electricity should be available at all times at the flick of a switch has permeated most, if not all, of our daily activities in the modern world.

However, as far back as the 1980s, Schweppe and colleagues highlighted numerous reasons why demand for electricity should be made more adaptive to supply conditions.³⁴ They noted that doing so would allow peaks in demand to be “flattened,” thus allowing generation assets to be reduced; particularly, expensive (and carbon-intensive) peaking plants that might only be used for several hours or less each day. This flattening would result in longer term and cheaper production contracts, producing a more efficient grid with lower prices for consumers. Furthermore, it would also provide significant benefits for grid operators. For example, if generation capacity was temporarily restricted due to some unforeseen event (either due to faults or if renewable energy sources are unavailable), then controlling demand would ensure that those generators available were not overloaded. In addition, after a power failure has occurred, the ability to synchronize demand with supply as connections are recovered and generators are brought up to speed would significantly accelerate recovery from such failures (a point we discuss later).

The need for demand-side management is even more apparent within a grid that makes extensive use of intermittent renewable generation. In this case, there is a high likelihood that there will be periods when there is insufficient generation capacity to meet demand. It is thus imperative that demand can be reduced at these times. Conversely, there may also be times when renewable energy is plentiful, and demand should increase to make the best use of this energy.

---

Unlike existing grids where electricity generally flows one-way from generators to consumers, [the smart grid] will result in flows of electricity that vary in magnitude and direction continuously.

---

To date, approaches to reduce demand have been limited to either directly controlling the devices used by the consumers (for example, automatically switching off high-load devices such as air conditioners at peak times), or to providing customers with tariffs that deter peak time use of electricity. The advent of the smart grid with two-way information flows, and smart meters making real-time measurements of consumption, would allow demand-side management to be deployed at scale across the entire grid, providing every home and every commercial and industrial consumer with the ability to automatically reduce load in response to signals from the grid.

However, doing so may be ineffective, or at worst, detrimental, since such initiatives tend to reduce the natural diversity of consumers’ peak demands and shift all of these peaks to specific periods.³⁶ For example, static time-of-use (TOU) pricing where the price of electricity at night is cheaper than during the day, has been observed to create significant additional peaks in demand as soon as the off-peak period is reached.^30,36 Similarly, critical peak pricing (CPP), which is often applied on the West Coast of the U.S to control air conditioners at peak times, can often create additional peaks as devices turn back on as soon as the critical period is over. Given this, a number of researchers have suggested that more sophisticated tariffs, such as real-time pricing (RTP) or spot pricing (where the price per kWh of electricity consumed is different for each half-hour and is provided to the consumer a day, or a few hours, ahead of time), in conjunction with more sophisticated ‘agents’ that can autonomously respond to these price signals, would avoid this.³⁴ However, even RTP can create unexpected peaks in demand, when all individuals respond to a signal in the same way, and inadvertently synchronize with others.³⁰

Thus, it appears that demand-side management technologies that simply rely on reacting to control or price signals will not be enough. Rather, what is necessary are more sophisticated approaches that are truly adaptive to the state of the grid, that are able to learn the correct response given any particular situation, and that can look ahead and predict both supply and demand trends in the near future, in order to prepare for future reductions in available supply, or to make the most effective use of supply when it is available.

The design of such intelligent systems is challenged by the complexity of the domains in which they are deployed. For example, within a home, demand reduction may involve shifting the time of use of a number of electrical appliances, each with their own individual constraints (for example, lighting cannot be shifted, a washing machine can be shifted by a day or two, while a dishwasher may be shiftable by a few hours²⁴). Similarly, both heating (given that this will likely be electrified through the use of efficient heat pumps) and cooling loads can be shifted as long as the comfort and temperature preferences of the householders are met. To be effective in this, it may also be necessary for such systems to learn the thermal properties of the home in which they are deployed, as well as the local weather conditions, and the way in which these local conditions impact on the heat loss, or gain, of the home. Crucially, these approaches will have to take into account the fact that each individual householder will have his or her own preferences, and these preferences must either be explicitly elicited, or learned. Since these preferences are likely to exhibit change over time, and depend on the current activities of the householder and local weather conditions, in computational terms this translates into an online learning and scheduling problem under uncertainty.

Similarly, commercial and industrial consumers will be constrained by existing contracts and commercial considerations (for example, a factory may have to deliver products within certain deadlines, while a data center has to be available to its customers 24 hours a day), and must balance demand reduction against these additional factors. Large industrial consumers of electricity with significant heating, cooling, or pumping loads may have considerable flexibility regarding when they actually consume electricity as long as some overarching constraints are satisfied.”^e However, to do so in a responsive way requires that the usage optimization algorithm that is deployed is able to model and predict both the prices within the grid, and also the industrial processes themselves (similar to the home heating scenario where a thermal model of the home must be learned). Furthermore, in both settings, it will be essential that the householders and business owners are able to understand the consequences of the automated actions that are taken, and are happy to delegate control to an intelligent device or software agent. In this respect, it will be important to define the adjustable autonomy of such systems; to what extent should the agent automatically decide to shift devices to run at certain times, and when should it ask for confirmation from the user.³³

---

It will be important to design simulation systems that can accurately represent both the grid and the reaction of consumers, in order to predict the emergent properties of the system under a range of different conditions and worst-case scenarios.

---

The development of these autonomous technologies raises the prospect that such systems will be widely deployed in possibly millions of homes, each individually reacting to prices and to the preferences of householders. Defining the convergence properties (that is, how the aggregate demand profile will respond to price signals) of such a complex system will be central to the definition of what constitutes safe and efficient behaviors for the grid. In particular, it will be necessary to ensure that neither significant inefficiencies, nor excessive volatility ensue from these autonomous systems converging to poor equilibria (or not converging at all). Hence, it will be important to design simulation systems that can accurately represent both the grid and the reaction of consumers, in order to predict the emergent properties of the system under a range of different conditions (for example, weather patterns or social activities) and worst-case scenarios (some generators fail or lines trip).

Against this background, recent work has begun to research the use of autonomous agents, representing individual consumers, that interact through markets,^10,40 and individually learn to optimize their use of electrical loads or storage devices in a number of simplified settings.^28,30 Simulations of such systems point to the effectiveness of adaptive behaviors (that learn to react to prices) on the grid. In addition, human-computer interaction technologies have also been proposed to improve the reaction of users to the information from smart meters.^16,37 While promising, we believe this work represents only the beginnings of the research needed in this area.

Thus, in summary, we believe the key AI challenges in demand-side management are:

• Designing automation technologies for heterogeneous devices that learn to adapt their energy consumption against real-time price signals when faced with uncertainty in predictions of future demand and supply, the individual users’ preferences, and the constraints of the overarching system (domestic, commercial, or industrial) within which it is deployed;
• Developing the means by which the automated decisions of these systems can be effectively communicated to, and controlled by, their human owners, while allowing a varying range of autonomous behaviors; and
• Developing simulation and prediction tools to allow the systemwide consequences of deploying pricing mechanisms and energy management agents to be assessed by grid operators and suppliers.

Back to Top

Electric Vehicles

With the advent of commercially viable EVs, such as the Nissan Leaf and the Chevy Volt, the coming years are likely to see the large-scale EV adoption that will shift the energy requirements of transport from fossil fuels to renewable electricity from the smart grid.^12,26 EVs are one of the key mechanisms to deliver significant reductions in carbon emissions as the transport sector is one of the largest contributors in most developed countries (approximately 20% in the U.K. and 30% in the U.S.), and the majority of these emissions are the result of private motor vehicles. As millions of EVs are deployed onto the roads, novel mechanisms, building upon the communication infrastructure and distributed intelligence in the smart grid, will be needed to ensure the batteries of these vehicles are fully charged when their owners need to use them, without overloading the network. In addition, these same batteries will form part of the decentralized demand-side management system used to reduce variations in demand and supply by charging when low-carbon renewable energy is plentiful, and discharging back into the grid when it is in short supply; so called vehicle-to-grid or V2G.

EVs place a considerable additional load on the grid due to the high charging rates that are necessary to ensure both a reasonable vehicle range of around 100 miles, and the ability to rapidly charge the battery. While a typical house may use between 20- to 50kWh of energy per day, an EV battery may be charged with 32kWh of energy in just a few hours.¹⁸ Thus, the total energy required by these vehicles may be comparable to the total electricity consumption within the domestic sector, but all of this demand is likely to be concentrated over particular periods of the day, and over particular geographical areas; both of which are subject to shifts. For example, if all the EVs in a local neighborhood are charged at the same time (as is likely to happen as householders return home at the end of the day), the local distribution network, and in particular, the street-level transformer (which is typically undersized and allowed to cool over night), may become a significant bottleneck to supply. When the owners of these vehicles drive to work and plug in, the demand will shift in both time and geographic distribution. Similar issues occur when a large number of EVs simultaneously attend large-scale social events at sporting arenas or shopping malls.²⁶

Given these continuously changing demands imposed on the local distribution network by the movement and charging of vehicles within it, and the variable supply of renewable energy, it will be necessary to devise sophisticated approaches to schedule the charging of EVs. This scheduling should make the most effective use of what renewable energy is available, while also ensuring the vehicles’ batteries are fully charged when required by their owners. Furthermore, this must be done in the context of uncertainty regarding both the future availability of renewable energy, and future vehicle use. Building upon this, it will be important to design decentralized control mechanisms that can guide the charging of EVs to various points in the network, given its dynamic conditions and constraints. In particular, these mechanisms will have to take into account that consumers must be incentivized (for example, in terms of charging prices or speeds at specific points) to adapt their behavior as they may only care about their individual travel needs. The challenge is to ensure such incentives are properly designed to induce charging profiles that stabilize the grid (that is, ensure flows are secure and transformers are not overloaded) while satisfying the needs and preferences of the highly heterogeneous population of EVs each with their individual battery capacity, charging speeds, and usage pattern.

More positively, EVs will also be a key resource in the demand-side management systems discussed previously. In such systems, the ability to defer demand to times when renewable energy is more plentiful is essential, and currently, this is only possible with the subset of electrical loads that are not required to have immediate effect (for example, washing machines or dishwashers). However, the ability to store energy within large batteries allows any electrical load to be shifted, and we are likely to first see energy from EV batteries support the shifting of loads within their owners’ home (vehicle-to-home or V2H), and then to providing energy back to the grid itself (V2G).^25,26,f While the impact on the user’s lifestyle of scheduling loads in the home may be minimized through the use of the EV battery, the scheduling of the battery charging and discharging cycles will need to ensure there is sufficient capacity to satisfy the loads in the home, and the travel needs of the vehicle’s owner, while minimizing the cost of electricity used. Moreover, this schedule will need to be optimized for, and adapt to, the changing needs of the vehicle owner, the (real-time) price paid for feeding back to the grid, as well as the battery capacity and efficiency. Hence, such optimizations will also require learning algorithms to predict the pattern of use of the vehicle, and also the demand of the home.

Addressing these challenges requires intelligent systems that can fully automate the charging and discharging of these vehicles, while taking account of the current and future availability of the renewable generation, and being aware of the local constraints of the distribution network. Recent work has begun to address these challenges with online mechanism design being used to elicit users’ travel requirements (that is, the amount of charge required and the time at which the EV is needed) and schedule the charging of their vehicles,¹⁷ and suggestions to apply peak and dynamic pricing to shift demand across a city.²⁵ These mechanisms are likely to work and be of social value (that is, not impede the daily activities of the vehicle owners) only if they minimize waiting (charging) times for consumers and never leave consumers stranded. As such, these systems will have to draw on diverse sources of information, such as distribution network load information (for example, load on the lines, number of EVs connected at various positions and prices at different charge/discharge points), traffic information from road cameras, and geolocation services such as Google Latitude (http://latitude.google.com) or Facebook Places (http://www.facebook.com/places) that contain rich information that can be mined to predict future movements of consumers to specific locations and, hence, likely bottlenecks on specific lines and transformers in the system. Systems that can optimize the charging cycle of an EV by making sense of such a wide range of heterogeneous information sources are likely to play a key role in ensuring EVs are seamlessly integrated into the smart grid.

Thus, against this background, we identify the key AI challenges in the deployment of EVs in the smart grid as follows:

• Predicting individual users’ EV charging needs based on data about their daily activities and travel needs;
• Predicting aggregate EV charging demands at different points in the network given the continuous movement of EVs, the available charge in their batteries, and the social activities their users engage in;
• Designing decentralized control mechanisms that coordinate the movement of EVs (each with different battery capacities and charging speeds) to different charge points by providing incentives to consumers to do so. The aim is to maintain secure flows on the grid and ensure transformers do not trip due to excess demand; and
• Designing algorithms to optimize the charging cycles of EVs to satisfy the predicted needs of the user (to shift loads or to travel) while maximizing the profits generated from participating in V2G sessions.

Back to Top

Virtual Power Plants

As larger numbers of actors (for example, EVs, homes, or renewable energy providers) in the smart grid communicate and coordinate with each other to control demand at different points in the network (for example, using demand-side management to ensure demand is able to follow the supply of renewable energy, and EVs discharging to the grid to cope with excess demand), it will be important to harness synergies that exist between them to improve the efficiency of the grid (EVs discharging to satisfy demand at times when demand-side management techniques cannot shift enough usage to later times). To this end, the concept of a VPP² has been proposed to capture the notion of a number of actors, coming together to sell electricity, as an aggregate.^g However, several challenges arise in the formation and management of VPPs that coordinate a number of heterogeneous actors (EVs or renewable energy providers) to maximize the amount of energy delivered in the system while minimizing the costs and uncertainties in doing so. In particular, these individual actors must be able to come to an agreement in technical (that is, how they coordinate their consumption or production patterns) and economical (how they share the profits generated by the VPP) terms in order to maximize the value of the set of energy services (providing electricity, storing electricity, or shifting demand) they provide as a VPP.

The process of forming VPPs at a technical level means the individual actors must synchronize the largely heterogeneous services they provide within the VPP in an agile fashion to meet the requirements of the contracts they make with their customers. In particular, individual actors need to estimate the impact of their individual production (or demand reduction) on the aggregate performance of the VPP, and communicate and optimize the joint actions taken to meet the VPPs’ objectives (that is, satisfy demand). These technical arrangements may need to be specified on a daily, and even on an hourly basis to maximize the profits of the individual actors. This is because if some actors can only produce energy at specific times of the day (for example, PVs generate energy during the day and tidal energy may be available at night), they will want to choose those partners they can complement better at those times (for example, a PV farm and a tidal generator may generate energy out of phase with each other and hence be highly complementary, while wind energy providers whose turbines are located in the same region will generate energy at the same time and hence be less complementary). In turn, if new actors become better partners due to changes in the environment (more wind blows at night resulting in higher predicted wind energy production than tidal or more EVs converge to a specific region due to a social event, resulting in more storage being available), then some of them might decide to leave their current VPP and form a new one (for example, PV owners may be better off storing their excess energy during the day in the EVs to be able to supply at night rather than collaborate with a tidal energy provider). Given the scale and dynamism of this optimization problem, it will be important to design decentralized coordination algorithms and strategies that allow individual VPP participants to come to the most efficient arrangements within a reasonable time. Moreover, they will need to ensure such arrangements do not overload the local distribution networks in which they are connected. Given this, and the restrictions imposed by the network operator due to possible network congestion, the VPP may further have to re-optimize individual members’ operations. Typically, such optimizations would have to be done while being confronted with uncertainty about the individual members’ generation and consumption capacity.

The negotiation of technical arrangements must take into account that each potential member of a VPP is typically motivated to maximize its own profit, even though, as a group they compete against other actors (individuals, VPPs, or large power stations) in the system to maximize the group’s profits. Therefore, it is in each actor’s interest to take actions that will cost it the least while maximizing its share of the profits obtained by the VPP operations as a whole. This leaves some room for any individual resource to manipulate what it reveals as its predicted capability (such as, production, demand-response, or storage ability) as opposed to what it actually delivers on the day. For example, given their uncertainty about their production, some resources may prefer to understate their predicted production profile in case they get penalized by the group for underproducing. Alternatively, some resources may prefer to overstate their predicted production in the case that penalties for underproducing are not significant, and doing so increases their share of the profits. Such strategic considerations highlight the need to capture the provenance of decisions made by the VPP, such that it is possible to track and verify the individual actions, reports, and resulting rewards of each VPP member. The amount of provenance information this will generate will require efficient frameworks and mechanisms to represent, store, audit, and share it. Building upon provenance information it may then be possible to model the trustworthiness of individual VPP members through trust and reputation mechanisms similar to those used in online marketplaces, such as eBay or Amazon.²⁹ These mechanisms would, in turn, need to be designed to ensure they are robust to wrong or manipulative reports so that security measures can then be taken to ensure those actors with low trust do not cause significant disruption to the network in case they do not fulfill their part of the VPPs’ operations.

---

It will be important to design decentralized coordination algorithms and strategies that allow individual VPP participants to come to the most efficient arrangements within a reasonable time.

---

Assuming trust and reputation mechanisms can render VPPs reliable, it is important to ensure the negotiations that individual energy providers engage in converge in such a way that the most efficient VPPs (those generating the maximum social welfare) are most effectively formed (that is, in minimum time and with minimum communication costs) in the system.¹¹ Here, convergence is achieved when all the members of the VPP are satisfied with their share of the profits generated. The strategic and computational aspects of such negotiation processes are typically studied within multi-agent systems using tools such as cooperative game theory⁴ to partition the profits of groups among their members and combinational optimization algorithms to partition actors into the most efficient groupings for the system respectively.²⁷ However, the VPP formation process presents a number of unique challenges for AI research. In particular, given that all actors are connected in a network where flows are limited on each line, the actions (energy production or consumption) taken by each actor or VPP restricts the actions (to different degrees) of all VPPs in the system. The formation of each VPP can have significant externalities (for example, the flows created by one VPP can congest some lines, which, in turn, may prevent other VPPs from using energy sources or providing energy to consumers at the nodes connected to those lines). Moreover, the fact that each VPP compounds the uncertainty in production of each member (for example, due to uncertainty in the weather forecast or demand-side managed consumption) renders the VPP formation process highly stochastic.

All these issues will require the definition of computationally efficient search algorithms to allocate the payoffs to individual members of VPPs (as defined by game-theoretic solution concepts), while taking into account uncertainty in defining the relative contributions of each member to the aggregate performance (that is, mainly the profits generated) of the VPP. Moreover, given that different coalitions may be formed over time, an energy provider will choose its membership of coalitions in such a way as to maximize its revenues in the long run. This makes the search for efficient payoff allocations exponentially harder since it extends the search space to include future possible coalitions (and their expected returns) as well as present ones. Initial work in applying multi-agent systems approaches to the VPP formation process include Chalkiadakis et al.⁵ that provides solutions to the formation of VPPs of wind turbines with uncertain production and Dimeas and Hatziargyriou¹³ that presents an agent-based framework for VPP formation. These approaches, however, are still at a preliminary stage.

To advance the state of the art in this domain, the following key AI challenges still need to be addressed:

• Designing agent-based models of different VPP actors and processes in order to capture the complexity of the technical arrangements needed to form and manage VPPs;
• Distributed combinatorial optimization of the technical arrangements of demand-side management, V2G sessions, and micro-generation, to maximize rewards;
• Designing online mechanisms to form statistically correct trust measures for energy providers and automatically capture, track, and reason about the provenance of information revealed by energy providers to form VPPs; and
• Designing search algorithms and negotiation mechanisms for individual actors to agree on which VPP to form at different points in time and how to share the profits, using computationally efficient game-theoretic solution concepts, of a VPP given uncertainty in their performance, trust in their revealed capabilities, and changing weather and demand patterns.

Back to Top

Energy Prosumers

Our discussion so far has highlighted the significant heterogeneity of the large numbers of renewable energy resources in the smart grid and the complexity of the interactions between them and consumers. When taken altogether, this will necessitate significant changes in the way energy is bought and sold. This is set against the current operation of the grid where, in many countries (for example, the U.S., U.K., and in many parts of the EU), the electricity market is deregulated, such that large generators (located far from the point of use) trade directly with retailers who then sell the electricity to consumers through fixed contracts and tariffs.^19,35 In these countries, electricity is traded in forward and futures markets on a long-term ahead basis (weeks, months, seasons, and even years) and on day-ahead spot markets through a range of different contracts (for example, baseload, off-peak, or half-hourly contracts). Any real-time excess or shortfall in supply and demand with respect to contracted volume is settled in the balancing market (also termed the settlement process) where the price to buy and sell electricity is typically set by the market maker rather than being based on the direct matching between bids and offers in the day-ahead market.

---

The widespread adoption of renewable generation at the level of individual homes and businesses will lead to the creation of markets composed of many millions of prosumers who both produce and consume energy.

---

In contrast, market operations in the smart grid will have to adjust to a much larger number of heterogeneous entities, distributed throughout the network (closer to the point of use of electricity), trading much smaller amounts of energy. Indeed, the widespread adoption of renewable generation at the level of individual homes and businesses will lead to the creation of markets composed of many millions of prosumers who both produce and consume energy.¹⁴ Given this, while some prosumers may try to find an agreement with other prosumers to form VPPs (and resort to cooperative game-theoretic solutions as discussed earlier), many will directly trade in the electricity market where the game-theoretic considerations are purely noncooperative. Hence, compared to typical consumers who are mainly concerned about optimizing their electricity usage and who are typically agnostic to the real-time conditions on the electricity market, prosumers will need to optimize both their production and consumption of energy in order to make trading decisions in real time, through Internet-based interfaces to spot or forward markets, so they maximize the profits they can make by buying (to consume or store) and selling energy (either energy that they generate, or have stored earlier). By making their own localized trading decisions, prosumers may reduce the inefficiencies (added costs for end users and lower margins for generators) resulting from retailers hedging their energy purchases to minimize their exposure to risk (in the balancing market) and selling fixed long-term contracts to their consumers at high costs.

To do so, however, means prosumers must be endowed with effective trading strategies that can cope with uncertainty in the market. To minimize this uncertainty, they will need to be informed by predictions of their own demand (that may vary according to their needs and social activities) and generation capacity (for example, using weather forecasts or their EV usage needs), as well as the future price of electricity on the market. Given these trading decisions may need to be taken in real time, these predictions must also be generated in real time, and furthermore, to ensure users understand the lifestyle or operational implications of, and agree to, autonomously chosen trading decisions, human-computer interaction mechanisms will have to be designed to ensure large numbers of users trust and participate in these markets.

Essentially, as more prosumers populate the market, electricity will become a commodity with similar properties to those traded on stock markets. Given this, prosumers will be able to speculate in markets, buying and selling not simply to consume or supply electricity, but also to profit. However, while speculation may help make the market more efficient, it may also adversely impact the operation of the grid, if the traded flows do not actually satisfy the physical constraints of the distribution network. Potential solutions point to the application of regulatory measures to reduce speculation and more importantly, to congestion pricing mechanisms³⁹ within the distribution network, similar to the location-based pricing used within the transmission network in many parts of the U.S.³⁵ In such mechanisms, prices vary geographically throughout the network to ensure the flows of electricity within it do not exceed the limits of any of the transmission lines. To ensure these mechanisms do guarantee an efficient system it will be important to study the equilibrium conditions (for example, market efficiency, loads on transmission lines) resulting from the application of these congestion prices against significantly heterogeneous populations of prosumers.

In summary, the AI challenges involved in endowing prosumers with the intelligence to trade in electricity markets while ensuring safe network flows include:

• Developing computationally efficient learning algorithms that can accurately predict both the prosumers’ consumption and generation profiles (instead of only the usage profile for a consumer) as well as the price of electricity in real time in order to inform profitable trading decisions;
• Developing autonomous trading agents that can use such predictions to maximize their profit in the electricity market, and efficient algorithms to marry congestion management with market operation in distribution networks while guaranteeing good equilibrium conditions in the system; and
• Developing human-agents interaction mechanisms to allow prosumers to guide their trading decisions that take into account the prosumers’ daily constraints and preferences to consume or produce energy.

Back to Top

Self-Healing Networks

We have discussed a number of ways in which the electricity flows are likely to become both more unpredictable and bidirectional in the smart grid. This will result in a greater need for decentralized control strategies given the sheer number of active entities embedded in the system. While this renders fault-correction mechanisms in the network even more complex, the intelligence on which these active entities rely to make their consumption or generation decisions could also be used to naturally distribute (and hence make more robust) the decision making needed to apply self-healing strategies on the network when faults occur. Generally speaking, faults may arise either because lines become overloaded or because of old infrastructure becoming more prone to failure. To prevent such faults and remedy them, network operators already rely on a number of intelligent systems at the transmission network level. Traditionally, this is achieved with the help of automatic voltage regulators and using supervisory control and data acquisition systems⁶ with phasor measure units^h for situational awareness. Using such systems, active network management²¹ techniques can help to automatically reconfigure the network and send control signals to individual generators to increase generation or to precontracted loads to reduce their consumption.⁷ By endowing individual components on the network with the intelligence to apply these techniques, they can automatically correct faults as and when they occur and therefore let the network self-heal.

Extending these techniques to the management of the distribution network where large numbers of prosumers will operate will require a much larger number of phasor measure units to be deployed, both because the distribution network contains many more nodes, but also because the heterogeneity of the prosumers within it means network conditions are likely to vary more rapidly, necessitating accurate and timely monitoring and control. Fully instrumenting such networks is likely to be too expensive, and thus, there is a clear need for the development of state estimation systems that do not need to have every node in the network monitored. More importantly, we will need systems that can, using information gleaned from across the grid, learn correlations between state parameters at different nodes to provide accurate and robust estimates of the system state. The vast amount of data generated from multiple actors and sensors, and the microsecond-level measurements being made, will present formidable computational challenges in trying to estimate or predict the future state of the system.

If accurate information about the network can be obtained, active network management techniques, supported by distributed intelligence in the network, can help recover from faults faster than previously possible. For example, if voltages tend to drift in some parts of the network, automatic actions on transformers may be taken to reestablish the correct voltage levels, or assistance may be requested from EVs that are currently plugged into the network.³⁸ Furthermore, if faults are detected in one part of the network, that part of the system could be disconnected, leaving other independent parts running separated (that is, effectively ‘islanded’) provided they can sustain the balance between supply and demand (for example, using demand-side management). This could eventually avoid rolling blackouts or even help recover from those blackouts that do happen.

---

The smart grid must be able to make efficient use of intermittent renewable energy sources and supply the additional electricity required by EVs; doing so will require extensive use of demand-side management and VPPs to balance supply and demand.

---

To build such self-healing mechanisms, however, will require that all these actors can communicate their action space (for example, limits on voltage regulation, generation capacity, demand reduction ability) and agree on joint actions to implement islanding strategies. Given the uncertainty that permeates the actions of some of these entities (weather patterns that affect generation or social activities that affect the movement of EVs), it will be important to predict the impact of such uncertainty on the joint actions chosen to avoid electing those that may result in cascading failures in the worst case. Moreover, given the individual preferences of all actors involved (to consume electricity for specific activities or to sell electricity to maximize profits) these joint actions may need to be negotiated rapidly among them to ensure they end up in an agreement all parties commit to.²³

Initial approaches aiming to achieve this level of coordination express the problem as centralized (constrained) optimization problems that can be solved using (non) linear programming tools.⁷ Clearly, centralizing active network management involving potentially thousands of different types of actors, each with their own energy generation and production requirements is unlikely to scale very well in both the communication and computation costs it incurs. Hence, more scalable decentralized planning approaches that rely on short range communication between individual actors (for example, distribution network nodes, consumers, and EVs) will be needed.^32,38

Hence, we summarize the AI challenges of self-healing mechanisms as follows:

• Designing computationally efficient state estimation algorithms that can predict voltage and phase information at different nodes in the (partially observable) distribution network, in real time, given the prosumers’ current and predicted energy demand and supply;
• Enabling distributed coordination of automatic voltage regulators and energy providers and consumers for voltage control and balancing demand and supply during recovery from faults; and
• Automating distributed active network management strategies given the uncertainty (either because they cannot be accurately measured or there is incomplete information about certain nodes) about demand and supply at different points in the network.

Back to Top

Conclusion

There is a significant drive within the developed world to reduce our reliance on fossil fuels and move to a low-carbon economy in order to guarantee energy security and mitigate the impact of energy use on the environment. This transition requires a fundamental rethinking and reengineering of the electricity grid. The ensuing smart grid must be able to make efficient use of intermittent renewable energy sources and supply the additional electricity required by EVs; doing so will require extensive use of demand-side management and VPPs to balance supply and demand. It will also see large numbers of prosumers, buying and selling electricity in real time while automated network control algorithms maintain the safe operation of the grid and allow it to self-heal when something goes wrong.

The automation, information exchange, and distributed intelligence needed to deliver such technologies create many new challenges for the AI communities investigating machine learning, search, distributed control, and optimization. In this article, we have enumerated what we believe are the main challenges that, if met, will allow the full potential of the smart grid to be realized. Our claims build upon an extensive survey of the state of the art that goes beyond the papers cited and includes a large number of references (spanning technical papers, books, and policy documents relating to the deployment of specific smart grid technologies and evaluations of these) provided in the online appendix. In particular, we have highlighted the key issues in learning and predicting demand or supply at various points in the network given the variety of demand control mechanisms (for example, demand-side management and EV charging) and energy sources, each with different degrees of uncertainty in their production capability (VPPs or renewable energy sources). Moreover, we showed that the automated decentralized coordination between such entities (to balance demand and supply while ensuring flows on the network are always secure) must factor in both the individual properties of all actors (EVs with different batteries, different types of renewable energy sources, users with their own understandings of trading decisions and their agents’ decisions) involved and the incentives given to them to behave in certain ways (consumers shifting demand due to real-time pricing, or VPPs sharing profits equitably). Building upon this, we also examined some initial attempts at solving them within the various sub-areas of the smart grid.

Cutting across these various challenges are the issues of human-computer interaction, heterogeneity, dynamism, and uncertainty that are an intrinsic part of decision making and acting in the smart grid. By dealing effectively with these factors, we believe it will be possible for future generations to rely on their energy systems to deliver electricity efficiently, safely, and reliably.

Finally, we note that many of the issues present within the smart grid also arise within other domains such as water distribution, transportation, and telecommunication networks where large numbers of heterogeneous entities act and interact in a similar fashion to those within the grid. Hence, there is potential to transfer technologies across these domains and also address broader issues that affect the sustainability of such systems in a unified manner, such as cybersecurity and the ethics of delegating human decision making to intelligent systems.

Back to Top

Acknowledgments

The authors are supported by the iDEaS (www.ideasproject.info) and ORCHID (www.orchid.ac.uk) projects.

Back to Top

Back to Top

Back to Top