GenAI: Giga$$$, TeraWatt-Hours, and GigaTons of CO2

For more than a decade, we have speculated about the impact of artificial intelligence (AI)/machine learning (ML) on the environmental sustainability of computing (see ACM²). It has become clear that Al’s carbon emissions (scope 2), lifecycle carbon (scope 3), and other negative environmental impacts are growing explosively. Generative AI capabilities and applications exemplified and popularized in ChatGPT, DALL-E 2, Stable Diffusion, and Copilot, are the drivers. The evidence:

Giga$$$s of increased spending on AI computing equipment and infrastructure is driving a dramatic increase in infrastructure: AI computing silicon and datacenters.

Nvidia. From May 2022 to April 2023 (12 months), Nvidia’s datacenter group sold $15.5B of GPUs. During a May 24 conference call earlier this year, the firm doubled guidance for the next quarter, increasing quarterly datacenter GPU sales forecasts from $4B to $8B (to an annual rate of $32B).³ Dramatic growth in GPU demand for generative AI was the cited reason.

Amazon, Microsoft, and Google. The growth of GPUs for AI comes on top of 12% per year cloud growth and is reflected in large capital investments in datacenters (CPUs, networks, cooling, power, buildings, and so on). The three largest hyperscalers (Amazon, Microsoft, and Google) have reported large increases in datacenter CapEx investment from $78B in 2022 to $120B in 2023, a 54% increase.⁸

Evidence of excess demand. Endemic reports of “GPU shortages” indicate the cloud cannot satisfy the compute demands of well-funded AI developers in both new ventures and existing companies.

Terawatt-Hours (TWh) of increased datacenter power consumption will result from the growing investment in AI hardware. The most accurate retrospective estimates of datacenter power consumption are based on worldwide computing equipment sales: processors, networking, and storage. These backward-looking studies outlined the rapid growth of hyperscalers as the key driver of datacenter power consumption (30% per year).⁵ We apply similar methodology to these massive GPU investments.

Estimating the annual power consumption of $32B of GPUs involves GPU prices, operational parameters, and associated server and datacenter power. Using A100 prices ($10k) and 300W average incremental load (GPU at 50% of TDP, server balance of 50%, and PUE of 1.0: 3.2M GPUs → 960MW → 8.4 TWh/year.

To put this into perspective, 8.4 TWh/year was 54% of Google’s total datacenter power in 2020. Nvidia’s sales-rate increase alone corresponds to 27% of Google’s total 2020 fleet. A more complete comparison is difficult as several hyperscalers have stopped disclosing total power consumption due to growing public outcry.⁷ 8.4 TWh corresponds to 0.25% of the USA’s annual electric power consumption.

Gigatons of CO₂ emissions will result from the growth in datacenter power consumption and perhaps threaten the reliability of the power grid. Datacenter carbon emissions should be computed based on the average grid emissions at the time power is consumed or worse, since some studies suggest that data-center loads harm grid decarbonization.⁴ Using 2021 U.S. average grid emissions, 8.4 TWh is 3.25 gigatons of CO₂, the equivalent of five billion U.S. crosscountry flights. In high-penetration areas, cloud datacenters already exceed 20% of grid power; their further growth is being limited by grid reliability risks.

Back to Top

Summary

This analysis is based on economic data, and the conclusion of major growth was validated by industry leaders at Google’s recent “Future of Data Center Workshop” (June 1, 2023), where I served as a panelist. The growth analyzed is just one year. With the boom in generative AI, the situation is now far different than recent reports.⁶

The growing technical power and economic drivers make it improbable the AI juggernaut can be stopped. This means the sustainability of computing/AI must be a concern for all computing researchers.¹