Google opened up on its data center operations today at an industry event in Phoenix. It shared how its thinking and practices have changed as it seeks to lower the costs and environment impact of its servers and IT infrastructure.
Data Centers have come under attack as terminally wasteful and “dirty” enterprises that offer little in the way of jobs. Joe Weinman, senior VP at Telx, disputes that, and says in fact they indirectly employee countless thousands across many industries.
This is Jonathan Koomey’s fourth essay in a series of four this week that highlights, and excerpts from, his upcoming book, “Cold Cash, Cool Climate,” which discusses how entrepreneurs and investors can profit from tackling climate change, one of this century’s greatest challenges.
Yesterday Matt Davidson from JouleX, a data center energy management company, argued that power usage effectiveness (PUE) should be abandoned as a data center energy efficiency metric, a point of view I share. In its place Davidson suggests performance per watt (PPW). PPW is great in that it is straightforward and tells you how much actual compute a data center is doing for its energy use. No metric is perfect and one issue with PPW is that because there are so many different applications and server setups, it’s hard to establish baseline performance values against which to benchmark different data centers’ computing performance (to calculate PPW, you need standard measures of computing performance). That said, PPW at least allows us to shift the focus toward the ultimate outcome and purpose of a data center—the performance of the servers.
Last Thursday ARM put up an almost 10 percent single-day gain, which turned on the controversial possibility that a large-scale disruption of the server market was looking more possible. In the drive to produce servers based on lower-power chips that reduce operating costs and conserve energy, the $13 billion Cambridge, England–based microprocessor design company is seen as the great hope. Companies like Google and Amazon face the fact that their data centers are massive power hogs that impact both the environment and their bottom lines, and they’re searching for ways to reduce their energy use. So as Wall Street’s excitement over ARM’s announcements settles, what will the implications be for data centers and low-power servers, as well as big players like HP and Microsoft?
There were two key pieces of news last Thursday:
- The Wall Street Journal reported that HP, which has a 30 percent market share in servers, would collaborate with ARM-funded startup Calxeda to produce servers based on ARM chips.
- ARM will produce 64-bit chips. The ARMv8 architecture includes a 64-bit instruction set, opening up new possibilities for webscale data centers that want the option of running the latest software and application tasks. Microsoft, in particular, had complained that the absence of 64-bit processing was one of the reasons it had refused to support ARM architecture in its server software lineup.
One of the indicators of disruption is that the key players in a market are starting to hedge their bets against the market’s looking different in the future. HP is concerned that not all of its future customers will want Intel/AMD-based servers. And Microsoft, which has already opted to support ARM in its next desktop OS, must consider the risks of staying so closely aligned with Intel, a move that devastated its position in the mobile and tablet market.
What was striking about ARM’s press release was the quote from Microsoft’s General Manager, KD Hallman, describing ARM as “an important partner.” Microsoft must hedge against the possibility that the same energy concerns that drove processor design in mobile are becoming apparent in data centers. Many in the semiconductor industry have told me they believe future Windows Server support for ARM is inevitable, and adding 64-bit processing support dampens one Microsoft objection to supporting ARM.
The future of software design must include 64-bit architecture, largely because analyzing big databases and running data-heavy applications like digital video is easier on 64 bits versus 32 bits. Major cloud-computing companies like Amazon’s have to offer their clients 64-bit processing, and Amazon’s distinguished James Hamilton, who focuses on Amazon Web Services’ data center efficiency and scaling, has been open about the fact that he sees ARM designs as “appropriate for server side computing.”
Calxeda’s ARM-based server prototypes offer a 10-to-1 space advantage over traditional servers as well as an excess of a power envelope smaller by 10 times versus standard x86 servers. HP clearly believes there’s a market for ARM-powered servers, and volume production is slated for the second half of 2012. No doubt major cloud-computing players like Google and Amazon will run trials of these servers (if they haven’t already).
Intel owns just under 95 percent of the PC server market, a staggering degree of dominance that led one software executive to joke with me that Intel has had to ensure AMD’s survival just to avoid antitrust allegations. But unlike AMD, which has always struggled to differentiate itself from Intel, ARM has big power-saving advantages to entice companies to consider it as a cost-savings option.
The world is changing, and energy conservation is the new black. Intel’s dominant position is not in jeopardy, but Intel does have a competitor in ARM with the precise value proposition that data center customers want. Given the messages coming from major players like Amazon, ARM will most certainly get its shot to impress, and the help it gets from Microsoft and HP will only aid ARM’s cause.
Question of the week
Follow physics professor Tom Murphy on an exercise in galactic energy that points out the absurdity that results from the assumption that we can continue growing our consumption of energy forever.
Last week Google disclosed the details of its energy consumption, and its data center engineers argued that the leading figure cited to assess how energy-efficient a data center is, power usage effectiveness (PUE), must be continuously measured and averaged over a twelve-month period. This was a veiled shot at some companies that measure their data centers on a cold day in January when their cooling costs are zero and then publish a great PUE number. Google is right. We need more transparency surrounding PUE.
But it’s time to go beyond PUE and examine how we view the whole project of what efficient data center computing means. Leading companies like Facebook, Amazon and Google are all approaching 1.1 with their PUE, so it’s a metric with diminishing returns. With that in mind, here are three shifts in focus with regard to data efficiency that will matter in the future.
1. Admit the limits of the Power Usage Effectiveness metric. While PUE has been helpful in making it clear that a data center will be judged on its energy efficiency, it tells us nothing about the efficiency of the hardware and software. Here’s a hypothetical that Power Assure’s CTO, Clemens Pfeiffer, and I recently discussed:
You’ve got a hundred old servers in your data center that you decide you can do without, so you turn them off. The problem is, your PUE just went up. The same amount of power is being used by the facility to cool and light the building even though there’s less power being used by IT equipment. This illustrates a fundamental point: It’s time to address how efficient hardware and software are themselves as they relate to performing actual compute tasks. If a server’s on but it’s not doing anything, that’s wasteful.
2. Think about software. The entire conversation about data center efficiency over the past few years has revolved around facilities management and hardware. But for the first time, we’re seeing the beginnings of a basic question: What software platform is optimal for reducing energy use?
Stanford professor and current Google fellow Christos Kozyrakis has looked at how energy-efficient the widely used software platform Hadoop is. But one of the problems with Hadoop is that it requires nodes to remain powered on even if they’re not being used. “Hadoop is doing a lot of things that are wasteful, and those things have to be optimized,” says Kozyrakis.
When a semiconductor, like an Intel Atom or Xeon chip, is designed, engineers are constantly considering the energy characteristics of the final product. The same thinking now needs to be applied to software platforms.
3. Integrate hardware and software efficiency metrics. The buzzword in data centers is “heterogeneous computing environment.” Engineers are no longer just dealing with uniform servers built around Intel Xeon chips. They work with all sorts of configurations, ranging from high performance setups to low power servers, from Intel Atom–based Seamicro to Linux-based Tilera and maybe even one day ARM-based Calxeda chips.
Here is an opportunity to figure out which programs are appropriate for which server configurations and to optimize efficiency. Kozyrakis cited an example where an MIT professor asked students to write an application in a simple language like C and then in a high-level program, Java. The execution time for the application differed on the order of thousands. This translates into very different energy characteristics for that program.
In the end, PUE is a metric that’s about reducing waste and making sure the energy going into a data center is being used by the server. But the next frontier of data center efficiency is optimizing software for the multitude of emerging hardware platforms. This is more difficult, because it requires a shift in focus among major cloud players, like Google and Rackspace, as well as a new period of cooperation between programmers and hardware designers. It will take time, but there are clear benefits in terms of power consumption and total cost of ownership for the companies operating the data centers driving cloud computing.
Question of the week
Google released its number this morning: 2.2 billion kilowatt hours. That’s how much energy it used last year. That’s one hundredth of one percent of global energy use. Of the 1.46 million metric tons of CO2 that Google produced last year, most of it came from data centers, and Google has said that number would be double without efficiency measures. Perhaps the most interesting info coming from the site that Google built to communicate its energy use was how aggressively it argues that PUE (Power Usage Effectiveness) needs to be measured as often as possible and over a 12 month period. We’ve covered the issues with PUE as a metric for how efficient a data center is, but one of the core problems is that companies take PUE measurements during cold months when cooling costs are low. Measuring it monthly or even daily over the course of a year will give the most honest data on how efficiently a data center is functioning.
Android cell phones are notoriously bad at battery drain. But recently I discovered the single app that has smartly and simply solved many of these battery life problems for me: the PowerMax Android app made by Volt-Up.
Last week’s launch of Silver Spring Networks’ long-awaited IPO is a big deal for the smart-grid industry. The company’s early public performance could be considered a gauge of the sector’s health as it emerges from a stimulus-backed growth spurt into an uncertain future. And in the longer run, Silver Spring’s success or failure will be closely tied to its plans to build a smart-grid application-delivery platform from its smart-meter networking base — a challenge many utilities face in integrating smart-meter deployments into their smart-grid offerings.
The company, whose networking technology is inside 8 million deployed connected devices and another 9 million under contract, has benefited from the $4 billion U.S. federal smart-grid stimulus. Its S-1 reports $422.2 million in deferred revenues, compared with 2010 revenues of $70.22 million, with much future revenue tied up in stimulus-funded projects. But that stimulus is coming to an end, and last week’s federal action plan on demand response (PDF) said the industry shouldn’t expect any more federal funds beyond existing stimuli. Likewise, the Obama administration’s smart-grid initiative, launched last month, contained little new funding beyond $250 million for rural grid projects.
For Silver Spring, that means new growth must come either from outside the U.S. — something that has begun to happen in Australia and that could happen in markets such as South America — or by adding new businesses to its existing smart-metering deployments. How to tackle that challenge is an important question, not just for Silver Spring but also for competing startups such as Trilliant, SmartSynch, Tantalus and Tropos Networks, not to mention giants like Itron and Landis+Gyr.
Home energy management could be one route. Silver Spring has home-energy and demand-response platforms, and pilots show that it works to drive down energy use. But home energy management remains a very uncertain market, as the withdrawal of Google and Microsoft from the field indicates. A June survey by Black & Veatch (PDF) found lack of customer engagement the biggest barrier to utilities’ justifying investments in customer-facing smart-grid deployments. Silver Spring might have to wait awhile for its utility partners to start spending on home energy management. Even then, utilities may choose another HEM provider to run over Silver Spring’s networks.
Silver Spring could also tackle the utility side of the smart grid. Corporate smart-grid M&A activity has been booming, and most of it has been aimed at utility-centric software and hardware systems. Silver Spring has distribution grid systems, and it is testing them with utility AEP in Ohio. But it will be competing against some huge multinationals like ABB, GE, Siemens and Alstom for that business.
In the long run, Silver Spring wants to build a host of applications — plug-in car management, demand-response controls and the like — on the foundation of its smart-grid networking platform. Whether utilities will choose Silver Spring’s in-house systems or pick other companies in those fields to run over Silver Spring’s networks may decide whether the company’s growth potential is limited to making smart-meter networking cards or whether it will expand to become a services provider for the grid — a move it will want to make to take part in the broader transformation of power grids to come.
In an interesting way, U.S. utilities at large face a similar challenge. They invested at least $2 billion last year into more than 12.8 million smart-meter deployments and are expected to invest a little bit more this year and next. This investment is with the promise of using the smart meters not just as digital cash registers but also as grid-management devices and gateways to new forms of customer interaction. Maybe Silver Spring can be the smart-grid champion to get the ball rolling. If utilities can’t deliver the full range of services and savings they’ve promised from their smart meters, however, regulators and customers might start to believe the entire smart grid is a waste of money, spelling disaster for everyone involved, Silver Spring included.