Smart Grid Data Analytics

With Smart Meter deployments reaching 30% market penetration level, a huge new opportunity for data analytics is emerging. Instead of collecting data once every month, utilities are going to be collecting data 96 times a day or roughly 3000 times per month. This is a 300,000% increase in the amount of customer data that utilities have had before. Converting this data into actionable intelligence will create a multi-billion dollar opportunity. By applying smart analytics and machine learning techniques to the smart meter data, utilities can start to understand their customers' behavior in real-time and take advantage of this understanding to serve them better while improving the efficiency of the grid.

According to a reccent report from Pike Research, the software and services that will enable smart grid data analytics will represent one of the largest growth opportunities in the utility sector over the next few years, increasing from a relatively small market of $356 million in 2010 to nearly $4.2 billion in annual revenue by 2015 with a CAGR of 65%.



Industry analyst Marianne Hedin says the 'data tsunami' that will wash over utilities in the coming years is a formidable IT challenge, but it is also a huge opportunity to move beyond simple meter-to-cash functions and into more robust business intelligence capabilities, true situational awareness with real-time optimization of their operations, and even predictive analytics".

Like most emerging technology sectors, the market is highly fragmented. Large IT companies (such as IBM, Oracle, SAP, Google), traditional power systems provider (such as ABB, Siemens, Areva, GE), specialized consulting firms (SAIC, Accenture, Jacobs, CapGemini, Infosys) and pure-play smart grid providers (Itron, eMeter, Alcara, Ecologic, Opower, Telvent) are all working towards this market but as Hedin puts it "at this stage of the race, the competitive market is wide open".

Smart Meters - Crossed the Chasm?

Despite consumer concerns and recent reports around cooling of the smart-grid market, the Smart Meter deployment is well underway and actually picking steam. According to the recent report from Pike Research, the total number of planned smart meter installations has grown to 57.9 Million. Over 90 utilities are either installing smart meters or have approved plans for installations. By end of 2010, about 21 Million smart meters will be installed in United States. With an estimated 150 Million electric meters in US, very soon we will have more than 30% market penetration of smart meters. Looks like Smart Meters have truly crossed the chasm and are now into the "early majority" phase of the technology adoption life cycle.

Here is a link to the ARRA funded Smart Meter deployments projects. Interesting thing to note is that the ARRA funded projects while significant are a small minority of total smart meter deployments happening in the country. Out of the approximately 20 Million smart meters that will be installed by the end of 2010, only about 2 Million are from the ARRA funding. Rest are approved by state public utility commissions  and are funded by the  public.

M&A in Smart Grid

Interesting chart from a report on Smart Grid vendor ecosystem showing how the Smart Grid area deal-making is accelerating over the years -

Constellation buys CPower: More consolidation in DR ahead?

CPower was acquired by Constellation Energy today. Although the terms of the deal are not disclosed, chances are this is a good exit for investors who came in the last round of funding in April 2009. CPower was the smallest of the three main DR providers. The other two much larger players are EnerNoc and Comverge - both are public. Is this acquisition a sign for more consolidation in the DR industry ahead. The two other main players, EnerNoc and Comverge are small-caps with market capitalization of a few hundred million dollars making it an attractive target for the larger IT or Utility Provider company to move in this growing space. I wouldn't be surprised if one of the IT majors get into the action by purchasing one of them. Cisco, Oracle, IBM, Google & Microsoft are all looking to make in-roads in the Smart Grid sector. None of the existing IT folks have particularly strong channels in the utility world. So, in addition to giving them a good entry in a fast growing DR market, the acquisition will also provide a major channel access. Cisco is already making several acquisitions/partnerships in this area (Itron partnership, Archrock acquisition) and count on them to become a major player. Comverge and/or Itron will be a great addition to Cisco's hardware business. EnerNoc is particularly well suited for Oracle or IBM given their DR portfolio as well as their recent entry into the Carbon footprint management software. 

Energy Storage on the Grid

The idea of using energy storage to increase the efficiency of power systems is almost as old as the industry itself. There was a burst of activity in the area of electricity storage in 1880's, especially in Britain that took a significant lead over the United States in this area. The first application of large-scale energy storage in the US occurred in 1929 when pumped hydroelectric power plan was placed in service. Pumped hydro which involves pumping water from lower elevation to a higher elevation and using this to generate electricity at the time of peak demand is still the most widely used storage mechanism for electricity and are used to manage grid frequency and provide reserve generation capacity. Other common forms of bulk storage are -

• Compressed  Air Energy Storage (CAES)
• Batteries
• Sodium Sulphide (NaS)
• Nickel-Cadmium 
• Li-ion (Cobalt Oxide or Phosphate) 
• Vanadium Redox
• Zinc Bromine
• Lead-Acid
• Flywheels
• Superconducting Magnetic Energy 
• Electrochemical Capacitors

The following table shows the breakdown of major bulk storage technologies in the grid presently. 

Storage in the Grid

Storage Type	US (MWs)	Rest of the World (MWs)
Pumped Hydro	22,000	88,000
CAES	110	367
NAS	8	250
NiCad	26	0
Other	10	10

With the advent of Electric Vehicles (EVs), a significant amount of distributed storage is expected to come online in the next few years. Optimal control and management of these EVs offers a significant opportunity to increase the of efficiency of the electric grid.

Storage Model

The key physical attributes that differentiate the various energy storage technologies and therefore determine which applications they are most suited for are -

• Storage Capacity [MWh] - This is the total energy that can be stored in the system. 
• Power Rating [MW] - The maximum power output (or input) capacity. This is the rate of charging (or discharging).
• Efficiency - The ratio of energy discharged by the system to energy input to the system to charge it. Conversion Efficiency refers to the losses experienced when charging the system. Storage Efficiency refers to the 'leakage' losses during the time energy is stored. 
• Reaction Time - The time needed to "turn on" the system and begin charging or discharging or switch between the charging/discharging modes.
• Cost - Cost has many components including fixed cost of installing the battery and the cost to charge or discharge the battery. Energy density is an important factor in determining the cost as it has an influence on the real-estate costs. Note that the cost of discharging the battery may not always be negligible. For example, in a CAES system, natural gas might be needed to run the turbine for converting compressed air into electricity. 

Economic Benefit of Electricity Storage

There are many economic benefits that storage can provide. These include -

1. Load leveling between off-peak and on-peak times
2. Peak Generation
3. Arbitrage
4. Spinning Reserve
5. System Regulation
6. Deferred cost of Transmission and Distribution upgrades
7. Environmental Impacts

Useful Smart Grid Related Resources

• IEEE Smart Grid
• Smart Grid Clearing House
• IEEE Power and Energy Society (PES)
• FERC
• ISO-RTO Council
• AESO (Alberta, Canada)
• CAISO (California)
• ERCOT (Texas)
• IESO (Canada)
• ISONE (New England)
• MISO (Midwest)
• NBSO (New Brunswick, Canada)
• NYISO (New York)
• PJM (Pennsylvania, Jersey, Maryland)
• SPP (Southwest Power Pool)
• WECC
• EPRI

Does the world need nuclear energy? | Video on TED.com

Here is an interesting debate on Nuclear vs Wind/Solar. I agree with Mark that going Nuclear is not worth the risk of proliferation. However, if a breakthrough like TeraPower as outlined by Bill Gates in another TED talk becomes possible, I will change my view. I do think that we need to invest in all possibilities as we never know where the silver bullet might come from.

Debate: Does the world need nuclear energy? | Video on TED.com

Bill Gates on energy: Innovating to zero! | Video on TED.com

Bill Gates talks about how if he had one wish, it won't be to find a vaccine for malaria or even AIDS but to get to a cheaper, cleaner energy. This is the single biggest thing that can improve the quality of life for poor.

Bill Gates on energy: Innovating to zero! | Video on TED.com

Components of Electricity Rates

The electricity rates paid by consumers at the retail level can be broken down into several components - 

• Power cost - This is the cost of producing electricity and a strong function of the fuel cost. This can be easily more than 50% of the consumers' electricity bill. 
• Transmission and Distribution costs - these costs are generally regulated and reflect the cost of building, maintaining, and operating the bulk transmission network and the local distribution networks. 

Here are some figures taken from the NE ISO Electricity Cost Whitepaper published in 2006 - 

Components of the average forecast New England retail consumer cost of $156/megawatt-hour in 2006

Components of the wholesale electricity cost for a typical hour with gas units on the margin. 95% of the cost (total = $76.25 / MWh in 2005) is due to fuel and capacity payments.

Data Center Energy Usage

Data Center

We are in the midst of a major IT revolution where more and more applications are moving to the "Cloud". With the rise in cloud-computing and Software as a Service (SaaS), there is an accompanying rise in large data centers that host these applications and data in the "cloud". Google, Microsoft, Amazon, IBM and virtually every major IT company is providing some type of cloud infrastructure of their own. With this rapid rise in cloud computing, the electricity needed to power these data centers is also growing at a tremendous rate and doubling every 5-years or so.

In 2006, the energy used in data centers was around 61 Gigawatt-hours representing around 1.5% of *all* US Electricity consumption. Just to put this in perspective, this is equivalent to about 5.8 Million average households and the dollar cost of this electricity is around $4.5 Billion. By 2011, the data center consumption is expected to cross 100 Gigawatt-hours (about 12 Gigawatt peak power) at a cost of about $7.5 Billion. 

It is estimated that the cost of electricity of running a server outpaces the cost of the server in about 4-years. Clearly, if we compare this to any other appliance (like our Car), the ratio is just way out of line (of course, we don't run our car 24/7/365 like these servers). But still, it is clear that more energy efficient servers are a huge lever as far as the overall data center operating costs go. 

Another equally important factor determining the data center energy cost is the efficiency factor. This is the ratio of total energy used by the data center to the energy used in the servers themselves (i.e. useful energy towards computing). The best of breed data centers have managed to push this ratio down to around 1.4X by innovative rack and building designs and heat dissipation techniques. More typical number is around 1.8X for the industry at this point. 

Clearly there is some serious opportunity in improving the overall data center efficiency all around - in design of hardware, building and also software to manage the scheduling of computing tasks in a more energy efficient manner. A lot of the large data center operators are becoming power producers themselves and are now locating the facilities near the source of the power to reduce power costs. They are also trading directly in the wholesale power markets instead of buying electricity from a utility company. It would be interesting to see what role electric storage can play in the operation of these data centers by storing electricity when the cost of electricity is low and then using it at times of high demand. 

Electricity Consumption at a Large IT company

Recently I had the privilege of meeting the Global Energy Director of one of the world's largest networking equipment companies and learn about the company's electricity usage and energy efficiency programs. There has been a lot of buzz about the building and data center electricity management and I wanted to get an idea of some of the numbers involved. 

• This company has direct access in California which means that they are allowed to buy electricity on the wholesale day-ahead markets. Here are some figures that I was able to gather - 
• There spend on the day-ahead market is about $25 Million / year
• By actually buying electricity in the wholesale markets, the company is paying around $0.10 cent / KW-hour as opposed to $0.14cents that PG&E would have charged them. 
• Overall spend on electricity is around 125 Million / year
• Use around 1.2 Gigawatt-hours of energy per year
• Are usually able to predict their demand within +/-5% of actual for trading in the day-ahead market. But even a 5% error is around 1.25 Million / year. 
• The energy factor is about 1.8. This is the ratio of electricity used by the servers to the total electricity used in the data-center. A number of 1.0 means all the electricity is used by the servers. Anything over that is the overhead of cooling the building and heat dissipation. Industry's best numbers are around 1.4. Obviously, each decimal point reduction has big value in terms of cost.

Note, this is just one company. So the actual market for energy efficiency is huge! No wonder companies like Enernoc, Comverge, Honeywell etc. are all going after the building efficiency market.

The War of EV Charging Stations

The War

A war is  brewing between two Silicon Valley companies about the future of electric vehicle charging infrastructure. The reigning champ is none other than Palo Alto based Better Place with over $700 Million in funding. The company's most recent round of $350 Million places the company at a reported market cap of $1.25 Billion. Estimates call for another $1 billion in funding to realize the companies ambitious vision of getting the world off of fossil fuel based cars. Better Place is promoting a hybrid model of charging and battery swapping stations. The advantage is a fast turn-around time similar to the time needed to refuel the gas tank at a conventional gas station. There are other operational advantages also in this approach, such as the ability to pre-charge the batteries at off-peak charging times when the price of electricity is lower and then re-use it anytime. However, the company faces the daunting task of enabling the entire eco-system and creating some sort of standardization in the battery that is used with different makes/models. The company will also need to overcome the emotional issues associated with car-owners not owning one of the most critical and expensive pieces of equipment in their cars.

Better Place Battery Replacement Service

On the other spectrum of this debate is a relative new comer which has been installing charging stations which are more like conventional gas stations. These charging stations are installed at parking spots, homes and streets and consumers can access them by becoming a subscriber which gives them access to any of the charging stations in the entire fleet. The software also allows customers to discover the nearest available charging point given the current location. While the business model is very clean and the company is quite capital efficient having raised $14 Million in series B and has more than $37 Million in federal funding grants to establish a number of charging stations in North America. The charging times with the current charging stations is quite large (around 45 minutes with Level 3 chargers) but the low-cost, low capital, distributed nature of Coulumb's charging solution provides a very interesting alternative to Better Places's highly capital intensive model.

Coulomb Charging Station

Please leave your thoughts and comments on who you think will dominate the EV charging infrastructure of the future.

Electric Vehicles (EVs)

In the recent weeks there has been a number of events that have led me to believe that 2011 could be the breakout years for EV adoption in the US. First is the BP oil spill which just reinforces the fact that drilling for oil is simply not sustainable in the long term, not to mention the enormous security risk of transferring trillions of dollars to outside economies,  some of whom are not friendly towards us (Note: Canada is our largest oil trading partner closely followed by the middle-east).

Fisker-Karma

Tesla Roadster

Tesla Model-S

Some of the recent headline events -

• Almost all major manufacturers have now announced their plans for introducing EVs. 
• Toyota announced its plan to introduce its 2nd generation of RAV4 EV in partnership with Tesla. Toyota also announced a $50-Million investment in Tesla recently. 
• With Prius, Toyota already has the largest market share in the Hybrid Electric Vehicles which can be converted into a plug-in electric vehicle using A123's Hymotion L5 PCM. 
• Honda announced its plan to introduce EVs in 2012 in US
• GM will have Chevy Volt in 2011
• Nissan has Nissan Leaf and a rumored sports version of the Leaf
• Ford has Focus Electric
• Hot IPOs and funding events
• Tesla is trading at over $2Billion in market cap. Their 4-door Model-S sedan is expected to come on the market in 2011
• A123, a provider of batteries for EVs is trading at $1-Billion in market cap
• Fisker is expanding its manufacturing base with a new round of $500-Million+ funding from DOE to manufacture its Karma sedans. 
• Emergence of charging infrastructure from independent third party providers is reaching critical mass now - 
• BetterPlace  - providing a network of battery replacement centers
• Coulomb - providing a network of charging stations

EVs represent one of the biggest revolutions in the automobile industry in over a century. Introduction of EVs will give rise to a number of interesting problems in terms of managing the electric grid which is going through its own once in a century revolution with introduction of new Smart Grid technologies. The next 10-years promise to be an exciting time to be at the intersection of two of the largest industries in modern society undergoing simultaneous and overlapping transformation!

Introduction to (Electricity) Demand Response

Demand Response is a way to make electricity demand respond to conditions on the supply side to balance the flow of power. Generally, this means that customers would reduce their load in response to price signals during critical times when the demand is expected to surge.

Typically, the ratio of average load to the peak load (i.e. the capacity factor) is around 40%. This means that on an average only 40% of all the available capacity is used to supply the electricity. Usually, the peaking power plants are  more expensive than the base load production plants. Demand response can be thought of as an alternative to peak power plants that are more efficient and 100% clean. Instead of bringing expensive peak plants online during the periods of high demand, a utility can initiate a DR request and ask customers to reduce the load in such an event.

In competitive power markets operated by ISOs/RTOs, there are a number of market instruments that allow demand response provider and industrial as well as residential customers to supply demand response without having to own transmission resources. There are four broad categories of demand  response -

• Capacity - Capacity resources commit (usually for a term of year or longer) to reduce load when directed by ISO/RTO. This is the largest segment of DR and is now nearly 10% of the peak demand. 
• Ancillary Services - In ancillary services market, DR typically provides non spinning reserves. 
• Energy Prices - Resources that commit to reduce consumption based on price on a short-term basis
• Energy Voluntary - Resources are compensated for reducing consumption during emergency conditions but are not obligated to do so. 

According to a study published by ISO New England, a 500 MW increase in DR cuts cost by $32 million / year. A 5% reduction in peak electricity demand during the top 1% of the hours of the year would have a net present value of about $60 Billion in benefits.

An Overview of the Wholesale Electricity Markets in the US

At present about two-thirds of the population in US and more than half the population in Canada is served by the transmission system and wholesale markets operated by Independent System Operators (ISO) and Regional Transmission Operators (RTO). There are 10 ISOs/RTOs in the US and Canada as shown in the figure below. 

The key responsibilities for ISO/RTOs are - 

• Bulk power system operations - This includes maintaining reliable operations of the transmission network including outage coordination, generation scheduling, voltage management, ancillary services provision, and load forecasting. 
• Wholesale Electricity Markets - A market where energy providers submit their supply offers and purchasers submit their demand bid. ISO establishes the market clearing prices so that the lowest cost supplies are used until the demand is met. A number of studies have confirmed that such a centrally controlled competitive market market has reduced the cost of electricity for the consumers. Typically, an ISO operates several different markets to cover the power system operation over different time-scales. These include - 
• Day Ahead - Usually, the buyers and sellers submit bids and offers based on the day-ahead forecast of supply and demand and the ISO clears the market for each hour of the day so that supply and demand are matched at the minimum cost subject to transmission and other constraints. 
• Hour Ahead - Similar to the day-ahead market but now an hour ahead instead of the day-ahead.
• Real-time - Also called Energy Balancing, cleared every 5 or 10 minutes usually in "increments or decrements" from the hour ahead commitments previously established. Since the day-ahead and hour-ahead markets are cleared for "flat" power profile during the hour, the real time markets are needed to balance the changing demand and production continuously. 
• Ancillary  Services
• Frequency Regulation - This is a market to provide service that balances second-by-second variation in output to match instantaneous demand and maintain system frequency and voltage levels within a very tight tolerance. Typically, this is around 0.5-1% of the peak load and generators that provide this service have to hold that capacity out of energy production to accommodate the fluctuations. They incur an opportunity cost for lost production (if the market prices are higher than the cost of production) as well as additional wear-and-tear on the equipment. 
• Spinning Reserves - This is the extra capacity on line (i.e. spinning) that can respond quickly in case of a major system disturbance such as the loss of a generator. Typically, this is around 5-10% of the peak load and sufficient to cover the operation of the largest single generator going off-line. 

The ISO plays are key enabling role in promoting sustainable energy technologies for the grid. These include - 

• Demand Response - In 2009, about 30 Gigawatts of demand response in North America ISO/RTO markets representing about 7% of the peak electricity demand in these regions. This is an equivalent of 60+ power plants of 500MWs each. 
• Renewables - ISO/RTOs provide a one-stop shop for an power producer such as a wind farm for interconnection to the grid, access to energy spot markets, financial mechanisms like day-ahead market and financial transmission rights. By providing a flexible and competitive marketplace operating over a broad region, ISOs enable a renewable resource owner to sell its power outside of a local region. ISOs also coordinate the transmission planning process to evaluate and improve the transmission system and more efficient integration of renewable resources into the grid. 

A Brief History of Electricity (1900 - 2000)

First half of the 20th century was the golden period of Electric Power Industry. During this period the focus was on increasing capacity and improving efficiency. A key insight that came early in the 20th century was of "economies of scale (i.e. big is better!)".

• In 1903 Samuel Insull installed a 5MW generator in Chicago and managed to increase his load factor (average load / maximum load) to increase profits.
• In 1907 Insull further realizes that profitability from managing economies of scale and load factor grows with corporate size, and so buys all of his competitors.
• In 1907, States begin to recognize that electric companies are constitute natural monopolies with large economies of scale requiring huge capital investment, and that it was not socially efficient to have multiple competitor

Based on the realization of bigger is better, the trend rapidly moved to larger and larger centralized power plants.

It was also clear that higher transmission voltage meant lower transmission loss. 

A Brief History of Electricity (up to 1900)

(Source: James D. McCalley, Iowa State University)

AC vs DC - A Historical Perspective

Ever wondered why AC current is the dominant way of transmitting and distributing power to the household? The original system developed by Edison were based on DC transmission. While Edison found the "killler app" for electricity, namely the light bulb, the AC system that is deployed world wide today was developed by Nikola Tesla for George Westinghouse. The key "sustainable" differentiation for AC over DC was the ability to convert voltages from one level to another by  the use of "Transformers". Transformers allowed AC current to be transmitted at much higher voltage (thereby, allowing power to be transmitted for longer distances by using less wiring --> cheaper) and then be down-converted to a suitable low voltage (e.g. 110V) for use at the loads.

Here is a fascinating history of the AC vs DC war. For a brief history of electricity, please check out this post. 

What is the Smart Grid?

Smart Grid has become a buzz word today with estimates of over a trillion dollar of investment needed to modernize our aging electricity infrastructure over the next two decades. However, if you ask 10 people to define what a "Smart Grid" is, chances are you will get 10 different versions of what is expected of a "Smart Grid". I find this sample conversation between different participants in the smart-energy system helpful in understanding what is generally referred to as a Smart Grid. 


(Ref: The Smart Energy Network: Electric Power for the 21st Centure - by P. Mazza):

As morning gets underway, a generator tells the network, "I have X megawatts of production capacity available today at Y price". The transmission system responds, "I can carry those megawatts today at Z dollars until midafternoon when extremely hot temperatures are forecast. I expect to cut loads then to protect my lines." 


Meanwhile down at the distribution level smart appliances and equipment are projecting the day's power use to smart meters, which then report expected demands to their substations. The aggregate information indicates heavy afternoon load from air conditioners. That goes back to the transmission system, which then foresee the afternoon pinch. So the transmission agent posts an offering price for demand reductions. Distribution agents communicate the offering to the smart meters, which then confer with their appliances and equipment and report back. The distribution agents determine they can shut-down enough water heaters during high stress hours to meet transmission system's need. So they post demand reduction offers and the transmission agent accepts. Transmission lines reduce loads and comfortably ride out the afternoon. By the time people arrive home and take their after work showers, most water heaters will be back on. And everyone involved in the deal will receive a small credit on power bills. 


Through a real time conversation, the energy network has self-optimized a low-cost solution without brownouts, blackouts or costly overbuilding of transmission lines!

A critical aspect of making the grid smart is also making it more "sustainable". This means less reliance on fossil fuel based generation to reduce carbon emission, incorporation of large amount of renewable energy like wind and solar, and increasing the energy efficiency throughout the electricity supply chain by improving technology and providing financial incentives to market participants to generate, transmit and use electricity in more intelligent and sustainable ways.

Here is a cool animation on what the Smart Grid means!

And finally, here are some useful links for finding more about the smart grid.