Data Centers and HPC: the Energy Challenge

If you were asked to list industries whose carbon footprint contributes dramatically to global warming and represents a threat to our planet’s delicate climate balance, you would probably not think of IT as one of the top “energy hogs”.

You would be wrong. IT is rapidly climbing in this not-so-virtuous ranking, with its double digit growth rate in energy consumption dwarfing the transportation industry’s own of 1% per year. According to the industry consortium GreenTouch, the IT industry accounts today for roughly 2% of the world’s total energy consumption, comparable to the airline industry.

While the fantastic computing power that technology is making available to businesses and to strategic fields such as science, applied research and industrial R&D is enabling progress, ultimately contributing to the better good of human kind in ways rarely seen before, the desire for even faster progress is boosting the need for ever increasing computing power and for access to huge amounts of data, all but accelerating this segment’s demand for energy.

Data centers are responsible for a large chunk of the energy consumed by IT. According to the U.S. Environmental Protection Agency (EPA) the amount of energy consumed by data centers doubled between 2000 and 2006 alone. This trend slowed down in 2007 amid economic crisis and better data centers efficiency, to start accelerating again in the last 2 years. A recent report from the Natural Resources Defense Council (NRDC) claims waste and inefficiency in U.S. data centers – that consumed a massive 91 bn kWh of electricity in 2013 – will increase to 140 bn kWh by 2020, the equivalent of 50 large (500 megawatt) power plants.

This has become largely unsustainable.

In an ideal situation, IT equipment should use 100% of the energy consumed by a data center. Unfortunately, reality is different. The percentage of energy used for IT equipment varies between 60% and 30% of the total energy consumed by the whole data center. The parameter that best measures data center energy efficiency is PUE (power usage effectiveness). The closer PUE is to 1 the better: a PUE of 2.5 means that for every Watt consumed by the data center 1 is used  for the IT equipment and 1.5 Watt goes for cooling or other not essential activities. A PUE of 1 means that 100% of the energy used by the data center goes into IT equipment.

A 2014 Uptime Institute annual data center survey reveals that data center power usage efficiency (PUE) metrics have plateaued at around 1.7 after several years of steady improvement.

The main reasons of this low efficiency are energy waste in the electric conversion needed to power equipment (transformers, rectifiers, UPS) and energy used to cool IT equipment through chillers and CRAC units.

There are many technologies that are being used to improve energy efficiency in data centers: virtualization, hot and cold aisle containment, increase of thermal envelope, air flow optimization, DCIM (Data Center Infrastructure Management). These are all low hanging fruits that allowed for an increase in efficiency, but they don’t allow reaching sustainable levels.

HPC (High Performance Computing) has long seen energy cost and availability as the biggest challenges for future developments. It is not a surprise that the HPC segment is currently adopting the most advanced solutions for energy efficiency, aiming to reduce consumption of both IT equipment and datacenter infrastructure, as well as reusing the thermal energy servers produce.

Designing more energy efficient systems means taking an approach where efficiency comes first. This implies making HW and SW design choices that maximize performance within a target power budget, leveraging heterogeneous architectures, accelerators, solid state disks, no-fan liquid cooled systems and in general choosing always the components that can guarantee more efficiency.

Wherever possible, the goal must be to achieve “free cooling”, meaning the IT equipment should be cooled without using additional energy, for instance by eliminating chillers, thus pushing down the datacentre PUE to levels around 1.05, very close to the ideal value of 1. Free cooling is only feasible when the coolant has a temperature higher than the external air temperature. If the outside temperature is very low, for instance at high latitudes or elevations, the coolant may be air, in all other cases it has to be liquid, typically water, warm enough to be cooled with outdoor air also in hot seasons. The only way to use warm water to cool IT equipment is to bring it as close as possible to where the heat is generated, at direct contact with the components (“direct liquid cooling”).

The development and optimization of the technologies leading to better energy efficiency in IT, and in HPC, requires non trivial R&D investments by the manufacturers of large scale computers used in datacenters and HPC centers. While better energy efficiency contributes significantly to lowering the Total Cost of Ownership of IT equipment, the R&D costs manufacturers sustain may lead to higher prices for equipment built according to energy efficiency criteria.

As for many other areas around carbon footprint reduction, “doing the right thing” may end up being economically less attractive than doing the “wrong” one.

While in other industrial and domestic segments (transportation, heating, renewable energy generation), policy, recommendations, stricter regulations and incentives start yielding tangible results, the IT industry and HPC have been only marginally touched by such initiatives.

As long as setting up an energy inefficient datacentre is an economically viable option for IT equipment owners, it is unlikely that substantial progress will be made towards reversing a dangerous trend.

While the issue is a planetary one, now is a good time for Europe to take it in its own hands and show the planet the way towards a more responsible and energy conscious future for the IT industry and High Performance Computing.

Data Centers and HPC: the Energy Challenge

The European HPC Strategy – Regional Vision and Strategy

I have been invited to deliver a presentation on the European HPC strategy at the Big Data and Extreme-scale computing in Barcelona on January 29, 2015. I am very grateful to the organisers for the opportunity, in particular because of the global nature of the event and the high level international representation. This post summarises the main elements of my presentation.

Why Europe needs a HPC strategy?

The short answer to this question is: HPC helps addressing big scientific, industrial and societal challenges and fosters innovation and business opportunities and Europe must be able to take full advantage of it. Some trends:

  • More computing cores in the chip at less energy requirements are profoundly changing the semiconductor industry and all end-user markets
  • Cloud computing, Mobile devices and Apps, Social Technologies, Big Data Analytics today represent about 20% of global IT spending, growing 6 times faster than traditional IT (80% of the IT market by 2020)
  • HPC is pushing the computing frontier (exascale) needed for new applications (e.g. Human Brain Project, Square Kilometre Array)
  • Sustainable HPC : 100 times current computing power consuming 100 times less energy
  • Europe consumes 30% of world HPC… but supplies 5%
  • Europe decreasing its score in top 500 world supercomputers (~20% and only 2 in top 10)

It can be safely argued that HPC is a strategic resource for the EU’s future from both the scientific and industrial point of view. Computational Science is already the “third pillar” of science. Scientific endeavours increasingly rely on data, simulation and models. The most powerful supercomputers are needed to address scientific and societal grand challenges in demand of huge computing and data resources. Industry relies more and more in HPC for innovation in products and services. Several of the most profitable and vibrant industrial sectors in Europe are big HPC users (manufacturing, oil & gas, pharmaceutical industry, etc.). These are some examples:

  • Smart Cities. Cyber-physical embedded systems and real time simulation and control for drinking/waste water, electricity consumption and distribution (e.g. Dalkia/Veolia, EDF). Propagation of electromagnetic waves for reducing power in antennas (Geomod)
  • Health. HPC for modelling instead of animal testing (e.g. L’Oreal, Rhenovia). Understanding the human (Virtual Phisiological Human, Human Brain Project Flagship)
  • Transport. Virtual prototyping, reducing time-to-market (Airbus, Renault, Porsche, etc
  • Big data analytics for finance (Paypal -online fraud detection-, Geco -real-time calculation of insurance quotes-, Amazon –predictive purchase-), health (personalised medical diagnosis) or Global System Science

The European HPC strategy and its implementation through Horizon 2020

The European HPC strategy consists in three tighly interelated elements and is supported by Horizon 2020. The three pillars are:

  • Computer Science: towards exascale HPC. This pillar is supported by a special Horizon 2020 Future and Emerging Techologies initiative focussing on the next generations of exascale computing as a key horizontal enabler for advanced modelling, simulation and big-data applications [HPC in Future and Emerging Technologies (FET)]
  • Providing access to the best supercomputing facilities and services for both industry and academia. This is implemented through PRACE, which is partly funding by Horizon 2020 eInfrastructure programme.
  • Achieving excellence in HPC applications. Implemented through the Centres of Excellence for scientific/industrial HPC applications in domains that are most important for Europe. This pillar is funded through Horizon 2020 eInfrastructure programme.

These three pillars are complemented with training, education and skills development in HPC. This is how they three pillars are related to each other:


And this is how the overall landscape looks like, with some budgetary figures attached:


The Public Private Partnership for HPC

In order to bring together all European actors in the HPC arena, the European Commission signed in January 2014 a Public Private Partnership (PPP) with ETP4HPC, the European Technology Platform for HPC. All PPP partners commit to support the development and implementation of Research & Innovation activities required to implement the European HPC strategy. Other partnerships have been set-up in the following related areas: Factories of the Future, Energy-efficient Buildings, European Green Vehicles, Sustainable Industrial Process, Future Internet (Advanced 5G network), Robotics, Photonics and Big Data Value Chain.

The objectives of the PPP for HPC are the following:

  • To build a European world-class HPC technology value chain that is globally competitive – synergy between technology development, applications and computing infrastructure
  • To achieve a critical mass of convergent resources in order to increase the competitiveness of European HPC vendors and solutions
  • To leverage the transformative power of HPC to boost European competitiveness in science and business
  • To expand the HPC base, especially SMEs (both as users and suppliers of competitive HPC technology solutions)
  • To develop a EU leadership and world-wide excellence in key application domains for industry, science and society

At this moment in time the PPP focuses on two of the three pillars of the European HPC strategy:

PPP scope

Next steps in Horizon 2020

I will conclude with a brief look at what’s on the agenda for 2015.

We are currently drafting the Work Programme 2016-2017. This is very much work in progress. Our initial ideas cover   (1) the support of the implementation of the Pan-European HPC infrastructure through PRACE, (2) the procurement of innovative solutions in HPC through the Public Prucurement of Innovation (PPI) instrument, (3) the development of core technologies towards exascale in the context of the PPP in HPC and (4) the procurement of the HPC platform of the Human Brain Project.

In 2015 we will be busy setting up the first Centres of Excellence for which the call has been recently closed.

By mid 2015 we will have the final results of the Study to follow the progress on the implementation of the European HPC strategy. This study covers the HPC market and HPC R&I landscape in the EU, the impact and the return on HPC investments in innovation and economic progress and growth in the EU and the status of the implementation of the HPC Communication Action Plan.

Finally, by the end of 2015 we will report to Parliament and Council on the implementation of the European HPC strategy. We are convinced that both Parliament and Council will be very satisfied with the achievements of the European HPC community.

The European HPC Strategy – Regional Vision and Strategy