Lean, Green Machines

On November 19, three matching German supercomputers easily won the latest Green500 prize, which rewards not just raw computing power but energy efficiency. The machines, at the Research Center of Jülich and the universities of Regensburg and Wuppertal, were designed for high-energy-physics calculations as part of the "QPACE" project.

The designers didn’t set out to win the Green500 award, says physics professor Tilo Wettig of the University of Regensburg, but they did think carefully about energy consumption. "Nowadays, supercomputers use so much energy that it’s probably more money than the purchase price," he says, so "the power efficiency is extremely important."

Green500 co-founder Wu-Chun Feng of Virginia Tech in Blacksburg, VA says that high temperatures also degrade overall system reliability and availability. The rankings, though, consider only the energy usage. The Top Green500 list, produced since November 2007, reorders the well-knownTop500 list  of the fastest machines in the world, based on the LINPACK matrix-algebra benchmark. The speed is given in millions of floating-point operations per second, or MFLOPS.

To encourage participation, Wu and his colleague let competitors re-use this same speed benchmark, but divide it by the total power used by the system. The resulting MFLOPS/watt metric, essentially the number of calculations performed for a particular amount of energy, tends to favor smaller systems, he notes.

Wu cautions that LINPACK emphasizes "pedal to the metal" performance, so Green500 is also exploring more general ways to assess efficiency. A new "Little " Green500 list — also topped this time by the QPACE systems — admits machines that would have been eligible for the Top500 within the previous 18 months. Two other lists that use more flexible performance metrics are also under development.

An efficient processing chip is key to an efficient system, Wettig says. "If you don’t start with the right processor, I guess you have no chance." He says the IBM PowerXCell 8i in the QPACE systems is the most energy-efficient available. This chip is from the same family used in Sony’s Playstation 3, Wettig notes, but "you have to work closely with IBM to get access to it."

But the chip is only the beginning. The QPACE machines clocked in at 723 MFLOPS/W, far ahead of the nearest competitor’s 458 MFLOPS/W, even though they used the same processor. To get an efficient system, Wettig says, "we also designed a novel liquid cooling system, which is very energy efficient." The system saves another 12 percent or so by adjusting the voltage of each of the 2,000-plus processors at start-up to the minimum necessary.

Although supercomputers pose unique challenges, the efficiency lessons are trickling into less imposing systems at data centers. "Techniques for building the hardware to be more energy efficient are there," Wu says. But he thinks there is much more to be saved. "From the software standpoint, it’s barely out of the starting blocks."