eBay shows the world how to measure MPG for data centers

eBay is busy building some of the world’s most-efficient data centers, and its efforts aren’t just show. The company has figured out a way to tie its computing infrastructure to specific business concerns and plans to continuously tweak its operations to meet top-level mandates. On Tuesday, eBay released a whitepaper describing how it accomplished this and laying out a framework for companies that want to do the same.

Dean Nelson, eBay’s vice president of Global Foundation Services, says the effort, called the Digital Service Efficiency report, “is the miles per gallon measure for technical infrastructure for eBay.” Essentially, the company has boiled its business down to a single currency — transactions (specifically URL requests) associated with users’ buying and selling on the site — and created a slew of metrics that measure how efficiently it delivers those transactions in terms of revenue, performance, cost and carbon footprint.

The project has been about 18 months in the making, Nelson told me during a recent phone call, and eBay was finally able to set a baseline measurement of its performance in 2012. Now that it knows what’s in place and how its infrastructure performs over the course of a year, the goal in 2013 is to cut its computing-related carbon usage and costs by 10 percent and increase performance in terms of transactions per kilowatt-hour by 10 percent.

In order to meet these goals, he said, every member of the technical team — from facilities managers to software engineers — has be striving toward them and also be cognizant of how turning their “knobs” will affect the other metrics eBay is measuring. “Think of it like a Rubik’s cube,” Nelson explained. “You can solve one side but screw up the rest of them.”

eBay plans to release quarterly updates on its progress along with its earnings reports, but employees will have access to down-to-the-second visibility into what’s going on. “It makes it personal for them,” Nelson said. “They can see what their efforts mean.”

Digital Service Efficiency

52,075 servers doing a lot of work

Nelson offered some pretty compelling examples of how the Digital Service Efficiency project works in practice. If the goal is to decrease cost per transactions, data center engineers might try to minimize power usage at the facility level while server engineers might look to lower-power gear or better utilization on existing gear. They essentially reduce the denominator in that equation “and the net result is we should make more money from those transactions,” he said.

In one real-world instance, a software engineer tweaked some code that affected how much memory an application requires and the company was able to eliminate 400 servers. That cut energy usage by 1 megawatt and a $2 million savings in capital expense when the time would have come to refresh those servers.

eBay also has created a “list of fame” and a “list of shame” that highlight the 1,000 best- and worst-utilized servers within the company. “We have a hit list,” Nelson said, and it’s going to examine the bottom 20 percent to figure out why they’re as wasteful as they are.

However, he added, it’s important to remember on the server front that improving cost, performance and carbon usage doesn’t always mean buying lower-power gear. If eBay can improve the power density of its racks using technology such as liquid cooling — something its Project Mercury data center in Phoenix is pre-equipped for — it can handle more transactions on less gear. It already has some racks running at a sustained rate of 35 kilowatts and thinks it can push that up to 50 kilowatts, Nelson said.

Clean transactions with solar panels and Bloom boxes

On the carbon front, eBay has nothing but an open field in front of it thanks to some big clean-energy projects set to go live in 2013 in its new Salt Lake City, Utah, data center called Project Topaz. For starters, it’s using Bloom Energy boxes as the primary power source, which mean a slightly higher cost per transaction, but also a 13 percent reduction in carbon emissions and increased reliability (downtime costs eBay a lot of money).

Also, the company has finally cleared some regulatory hurdles to tie an on-site solar array back to the grid. Because of changes to a Utah law that eBay lobbied for, it’s about to start sourcing off-site clean energy for its data centers, as well.

“That is a corporate priority,” Nelson said. “We want to create the cleanest commerce engine on the freakin’ planet.”

Trying to change an industry

Of course, the Digital Service Efficiency methodology isn’t the only attempt by a major data center operator to show the world how efficient it is. Google publishes annual Power Utilization Efficiency (PUE) ratings for its data centers, and Facebook occasionally does as well. On Monday, Salesforce.com released a statement underscoring its commitment to sourcing renewable energy.

dse chart

However, Nelson pointed out, what eBay is doing — and encouraging others to do — is more transparent in that it gives a lot more depth about operations, including the company’s server count. Even if companies don’t publish their results, tying operational efficiency to other business objectives should have a positive effect on the bottom line and the environment, regardless. Every company will have its own base currency, Nelson explained, and they’ll have to find their own metrics to measure and figure out what are the knobs that each part of the company can turn to meet goals.

“We all have the same challenges, the same things to solve for, but we have numerous ways to solve it,” Nelson said. …”[Their implementations] may change completely, but the point is the conversation is starting.”