Using a cluster management system to improve data center efficiency

Data center efficiency remains a critical problem as utilization rates of individual servers can often run as low as 10 percent, remaining idle and on standby in the event of a sudden surge in use. This problem is one many data center infrastructure management (DCIM) companies are trying to solve.

And it’s not just the private sector that is keen to take a stab at the problem. Stanford engineer and associate professor Christos Kozyrakis, along with doctoral student Christina Delimitrou have developed a cluster management system called Quasar that the two believe improve data center efficiency.

Stanford News reports on the idea behind Quasar:

It’s easy to understand how a reservation system lends itself to excess idle capacity. Developers are likely to err on the side of caution. Because a typical data center runs many applications, the total of all those overestimates results in a lot of excess capacity.

Kozyrakis has been working with Delimitrou, a graduate student in his Stanford lab, to change this dynamic by moving away from the reservation system.

Instead of asking developers to estimate how much capacity they are likely to need, the Stanford system would start by asking what sort of performance their applications require. For instance, if an application involves queries from users, how quickly must the application respond and to how many users?

Under this approach the cluster manager would have to make sure there was enough server capacity in the data center to meet all these requirements.

Attempting to maximize utilization by using predictive analytics to minimize the number of servers required for each processing task is always complicated. Data center operators fear downtime and having a data center be overwhelmed with processing requests. But if Quasar can make it into some real world data centers and prove itself, we could see another incremental step toward a more efficient data center.