AI data centers: what they are, and what actually limits them

What makes an AI data center different

A data center is a building that turns electricity into computation and heat. An AI data center does the same thing at a density the industry was not designed for. The International Energy Agency's April 2026 report Key Questions on Energy and AI puts it plainly: the power density of AI servers rose about 11 times between 2020 and 2025, and is set to rise another fourfold by 2027. By then a single rack, roughly the size of a large refrigerator, could have the peak demand of 65 households.

To make that concrete: NVIDIA's GB200 NVL72 puts 72 Blackwell GPUs and 36 Grace CPUs in one liquid-cooled rack. Estimates of its draw cluster around 120 kW, and HPE's own specification lists 132 kW nominal. Compare that with the typical enterprise rack: Uptime Institute's 2026 survey puts the average at about 7.8 kW and the most common figure at 11 kW, as reported by Data Center Knowledge. One AI rack carries the load of ten to fifteen ordinary ones.

Three design consequences follow from that density:

• Cooling moves to liquid. You cannot push enough air through a rack to carry 120 kW of heat out of it. The current generation of rack-scale accelerators is designed for direct-to-chip liquid cooling. See liquid cooling for AI data centers.
• The network becomes part of the computer. Training a frontier model means tens or hundreds of thousands of accelerators exchanging gradients constantly, so they are built as one campus with a dedicated fabric. Microsoft now links its Fairwater sites in Wisconsin and Atlanta with a dedicated AI WAN so two sites can work as one.
• The load swings. Training jobs produce large, fast power swings that a utility notices. The IEA expects 20 to 25 GW of battery storage inside data centers globally by 2030, partly to smooth that.

Training sites vs inference sites

The headlines are about training campuses: a gigawatt in one place because the job needs one tightly coupled cluster. Inference, which is where most enterprise spend lands, has different needs. It wants to be close to users, it tolerates being split across many smaller sites, and it cares more about cost per token than about peak cluster size. That distinction matters when you choose a provider. A neocloud that wins a training contract for a 100,000-GPU cluster is not automatically the right place for a latency-sensitive inference endpoint, and vice versa.

The four constraints

1. Power

The IEA estimates data centers used about 485 TWh of electricity in 2025 and projects about 950 TWh in 2030, around 3% of global demand. AI-focused data centers grow faster than that total, roughly tripling over the same period. The United States accounts for about half of global installed capacity, and Texas grid operator ERCOT alone has received interconnection requests for 160 GW of data center load, which the IEA notes is twice the existing global capacity of hyperscale and colocation data centers. Many of those requests are speculative. The numbers, and where they come from, are on AI data center power and water use.

2. Equipment and grid connections

Chips are no longer the only scarce input. Transformers, switchgear, gas turbines and grid interconnection queues are now the long poles. That is why developers are putting generation on site: the IEA counts 15 to 27 GW of on-site gas that may power data centers by 2030, mostly in the United States, and notes that reliable on-site gas needs 30% to 70% more generation than the load because of redundancy. It also notes that with gas turbines in short supply, on-site generation is not obviously faster. Nuclear is the longer bet; the earliest small reactors are covered in nuclear microreactors for AI data centers.

3. Cooling and water

Liquid cooling solves the heat problem and, done as a closed loop, can cut on-site water use close to zero. Microsoft says its Atlanta Fairwater site uses a closed-loop system whose initial fill equals what about 20 homes use in a year. The bigger water footprint usually sits upstream, in the power plants. Water is now a siting and permitting issue in its own right, not a footnote.

4. Capital

Hyperscalers and neoclouds announced about $715 billion of capital expenditure for 2026, a 75% increase, according to the IEA, which is more than annual investment in the entire US energy sector. Much of it is now debt-financed. For a buyer this is not abstract: a provider whose capacity depends on its next financing round is a different counterparty from one building out of operating cash flow.

How to read the gigawatt headlines

An announced capacity is a plan for a campus, usually over several years. What you can rent is the energised IT capacity that exists today. The gap is large. Meta's Hyperion campus in Louisiana is announced at up to 5 GW, but Meta expects about 2 GW by 2030. Epoch AI's satellite-based frontier data centers dataset estimates that the largest single sites draw roughly 0.9 to 1 GW of IT power as of October 2026. The ranking, with the gap between announced and running power for each site, is on the largest AI data centers.

When a provider pitches capacity, I would ask four questions:

1. How many megawatts are energised today, and on which sites?
2. What is the delivery date for my tranche, and what happens if it slips?
3. Which GPU generation, and what interconnect within and between racks?
4. Who is the power supplier, and is the grid connection already signed?

Build, lease or rent

Almost no company outside the hyperscalers and the frontier labs should be building an AI data center. The real choice for most engineering organisations is between a hyperscaler, a neocloud, or capacity you run yourself for a narrow set of workloads. The last option is smaller than it sounds, and it is the subject of the separate local AI cluster: a few workstations or a single server for development, privacy-sensitive data or predictable inference. For the economics of buying compute under scarcity, compute bottleneck planning is the companion piece.

In this cluster

• The largest AI data centers — Stargate, Colossus, Hyperion, Fairwater and Rainier: location, operator, announced versus running power
• Neoclouds compared — CoreWeave, Nebius, Lambda, Crusoe, Together AI, RunPod and Vast.ai, with listed GPU prices
• AI data center power and water use — the IEA, LBNL and EPRI numbers, and what they do and do not say
• Liquid cooling for AI data centers — direct-to-chip vs immersion vs rear-door, and why 120 kW racks force the choice
• Nuclear microreactors for AI data centers — Valar, Radiant, Aalo, Oklo and who is actually ahead

Frequently asked questions

Sources: International Energy Agency, Key Questions on Energy and AI (April 2026); Epoch AI Frontier Data Centers dataset (updated 19 October 2026); NVIDIA; HPE; Uptime Institute via Data Center Knowledge; Microsoft. Announced capacities are as stated by operators; running capacities are third-party estimates.