The hospital running patient records and the bank clearing transactions both depend on a data center nobody writes about. The industry conversation has fixated on gigawatt AI campuses and nine-figure GPU clusters, and the 3 to 25 MW facility that runs the rest of the economy has dropped out of it.
This is about what is running right now, not about the past. Goldman Sachs puts 86% of global data center power on non-AI workloads today. The facilities serving that 86% are not being replaced. They are being ignored while the engineers, capital, and transformer deliveries they need go to hyperscale AI campuses serving a different set of customers.
Then, in the last months of 2025, the CPU numbers moved. Intel reported an unexpected uptick in data center CPU demand and raised its 2026 capex guidance. AMD said publicly that it expects the server CPU total addressable market to grow in the "strong double digits" in 2026. Frontier AI labs started competing with cloud providers for commodity x86 servers because they were running out of CPUs to feed their reinforcement learning pipelines. The chip that supposedly became irrelevant the day ChatGPT launched is now in short supply at the most advanced AI sites in the world. [1]
CPUs never left. The infrastructure press just stopped covering them.
86%
of global data center power runs non-AI workloads: cloud 54%, traditional 32%, AI 14%
Goldman Sachs Research, 2025~75%
of data center workloads were non-AI in 2025. JLL puts AI at "about a quarter" of the total
JLL 2026 Global Data Center Outlook38 GW
of non-AI data center capacity installed globally in 2026, the base that runs today's enterprise compute
Programs.com, Jan 202648 MW
CPU and storage building at Microsoft Fairwater, supporting the 295 MW GPU cluster next door
SemiAnalysis, Feb 20261:6
CPU-to-GPU power ratio at Fairwater, expected to tilt further toward CPU with future GPU generations
SemiAnalysis, Feb 2026
The two-speed industry
The global data center market was worth about $354.75 billion in 2024 and is projected to reach $1.08 trillion by 2034, an 11.5% CAGR. [5] The AI subset that gets the headlines is a fraction of that base. It is growing faster, but it sits on top of a large installed base of CPU-dense, general-purpose infrastructure that isn't going anywhere.
This is a two-speed industry. On one track are the hyperscalers. Amazon has committed over $100 billion in capital expenditure this year, Microsoft is spending $80 billion, and Meta's capex could reach $65 billion. [6] Those numbers are real. They also distort the picture of where most of the world's compute lives.
On the other track are the thousands of 3 to 25 MW facilities housing ERP systems, databases, electronic health records, payment processing, email, and file storage. These sites don't get a booth at Supercomputing and their operators don't ring the opening bell at NASDAQ. They process most of the world's business logic anyway, and they run on CPUs.
| Company | 2025 CapEx guidance | Primary use |
|---|---|---|
| Amazon (AWS) | > $100B | AI infrastructure, hyperscale DC expansion |
| Microsoft | $80B | AI training and inference infrastructure |
| Meta | $60B to $65B | AI research and production serving |
| Google / Alphabet | $75B | TPU/GPU clusters, cloud AI |
Case study: Microsoft Fairwater Real-world example
Sources: SemiAnalysis (Feb 2026) · SemiAnalysis, Microsoft AI Strategy Deconstructed (Nov 2025) · Microsoft Source
Microsoft's "Fairwater" AI data centers for OpenAI are among the most advanced facilities ever built: 315-acre campuses housing hundreds of thousands of NVIDIA GB200 and GB300 GPUs, linked by a dedicated AI wide area network between Wisconsin and Atlanta, and built to train the next frontier models. Phase 1 came online in early 2026.
The detail that doesn't make headlines: each Fairwater campus is two buildings. One is the GPU building, about 295 MW of NVIDIA Blackwell racks at 140 kW per rack and 1,360 kW per row, built from 72-GPU NVLink-connected NVL72 racks with 800 Gbps GPU-to-GPU backend networking. The other is a standard 48 MW CPU and storage facility.
That 48 MW building exists only to support the GPU cluster next door. Tens of thousands of CPUs handle storage, sharding, and indexing for training runs, image and video decode for multimodal models, and the reinforcement learning environments that compile, verify, and interpret code outputs in parallel to generate training rewards. Without them the GPU cluster sits idle.
The CPU-to-GPU power ratio at Fairwater is about 1:6. SemiAnalysis expects the CPU share to grow with future GPU generations like Rubin, because GPU performance per watt is improving faster than CPU performance per watt, so each GPU needs proportionally more CPU behind it. The most advanced AI factory in the world still needs a CPU data center.
| Building | Power | Rack density | Primary role |
|---|---|---|---|
| GPU building | ~295 MW | 140 kW/rack (NVL72) | AI training, NVIDIA Blackwell GB200/GB300 |
| CPU & storage building | ~48 MW | Standard enterprise density | Data management, RL environments, multimodal decode |
| CPU:GPU power ratio | 1:6 | - | Expected to tilt further toward CPU with Rubin-generation GPUs |
| GPU networking | - | 800 Gbps backend | NVLink across 72-GPU NVL72 racks |
| Campus footprint | 315 acres | - | Wisconsin + Atlanta, connected via AI WAN |
Sources: SemiAnalysis (Feb 2026) · Data Center Dynamics (Nov 2025)
CPUs are back and never really left
In the five years before ChatGPT launched in November 2022, Intel shipped over 100 million Xeon Scalable CPUs to cloud and enterprise data centers. That installed base didn't evaporate when AI arrived. It kept running SAP, Oracle databases, hospital EHR systems, trading infrastructure, and government workloads, none of which care what is happening in the frontier AI race. [1]
What changed in 2026 is that CPU demand is now coming from two directions at once. The enterprise installed base is on its normal refresh cycle, with AMD's Turin generation consolidating sockets at up to 10:1 and retiring fleets of aging Cascade Lake servers. On top of that is new AI-driven demand that even the largest GPU operators cannot avoid.
SemiAnalysis's February 2026 analysis names two AI drivers. The first is reinforcement learning training loops, where CPUs compile, verify, and interpret code in parallel to generate rewards for the model, a bottleneck that grows with each GPU generation. The second is agentic and RAG inference, where AI agents issue API calls to databases and web services at a rate no human user could, which pushes CPU-served network traffic up a step. AWS and Azure are both building out their own Graviton and Cobalt ARM CPU lines and buying more x86 commodity servers to serve this demand. [1]
| Driver | Category | Detail |
|---|---|---|
| Enterprise refresh cycle | Traditional | AMD Turin offers up to 10:1 socket consolidation vs. aging Cascade Lake fleets |
| ERP / database / EHR | Traditional | SAP, Oracle, Epic: CPU-bound workloads with no GPU substitution path |
| RL training environments | AI-driven (new) | CPUs compile, verify, and interpret code outputs to generate GPU training rewards |
| Agentic / RAG inference | AI-driven (new) | AI agents issuing massive volumes of API calls to CPU-served databases and services |
| Multimodal data decode | AI-driven (new) | Image and video preprocessing for training, handled on CPU before GPU ingestion |
| Hyperscaler ARM buildouts | AI-driven (new) | AWS Graviton, Azure Cobalt: large-scale CPU procurement to serve inference traffic |
Source: SemiAnalysis, CPUs Are Back: The Datacenter CPU Landscape in 2026 (Feb 2026)
The power density divergence
The clearest way to see the gap between hyperscale AI and the 3 to 25 MW middle market is rack power density. The two are built to different physics.
| Facility type | Typical rack density | Cooling method | Primary workloads |
|---|---|---|---|
| Traditional enterprise (3 to 25 MW) | 5 to 15 kW | Air: CRAH units, hot/cold aisle containment | ERP, databases, EHR, email, file storage |
| Retail colocation | 10 to 30 kW | Air + optional in-row cooling | Mixed enterprise, some inference |
| Modern AI inference cluster | 30 to 60 kW | Rear-door HX or direct-to-chip liquid | Production inference (H100, L40S) |
| Hyperscale AI training (Fairwater) | 140 kW (NVL72 rack) | Liquid cooling, purpose-built infrastructure | Frontier model training, GB200/GB300 |
Sources: Data Center Dynamics (Nov 2025) · SemiAnalysis (Feb 2026)
A facility built in 2010 for 6 to 8 kW per rack cannot be retrofitted into a GPU training cluster. The electrical service, cooling plant, and floor loading are wrong for it. It doesn't need to be one. The HR software, the hospital patient database, and the bank's core ledger don't need 140 kW racks. They need uptime, compliance, low latency, and predictable cost. The trouble starts when operators feel pressure to "AI-ify" a building that was never built for training, or when capital goes where the hype is instead of where the workload is.
Two completely different buildings
A CPU data center and a GPU data center differ in kind, not degree. They are designed to different thermal loads, built to different floor loadings, fed by different electrical service, and cooled by different methods. The assumption that one converts into the other, or that the same operating experience transfers cleanly, is one of the more expensive misconceptions in the market right now.
One number makes this concrete. A standard 1U CPU server draws 300 to 500 watts. A 4U GPU server with eight H100s draws 6,500 to 7,500 watts, roughly 15 to 20 times the power from a box that takes four times the rack space. [19] Scale that up to a rack and the difference becomes structural. A typical CPU rack drawing 6 to 8 kW sits comfortably under a standard 30-amp circuit. A fully populated GPU training rack drawing 140 kW needs 208V or 400V three-phase feeds, 600-plus amps, redundant PDUs on separate feeds, and upstream electrical capacity that most existing enterprise facilities do not have and cannot easily add.
| Metric | CPU server (enterprise, 1U) | GPU server (AI, 4U, 8x H100) | Multiplier |
|---|---|---|---|
| Server power draw | 300 to 500 W | 6,500 to 7,500 W | ~15 to 20× |
| Per-chip TDP (CPU vs. GPU) | 150 to 350 W (Xeon / EPYC) | 700 to 1,000 W (H100 / Blackwell) | ~3 to 6× |
| Typical rack power draw | 5 to 15 kW | 60 to 140 kW (training); 30 to 60 kW (inference) | ~10 to 20× (training) |
| Cooling method | Air: CRAH/CRAC, hot/cold aisle containment | Liquid: rear-door HX, direct-to-chip, or immersion | Categorically different |
| Power circuit requirement | Single 30A circuit (208V) | 100 to 600A, 3-phase, multiple redundant feeds | ~10 to 20× ampacity |
| Floor load (structural) | ~100 to 150 lbs/sq ft | ~200 to 400 lbs/sq ft (liquid manifolds + servers) | ~2 to 3× |
| Cooling system share of facility power | ~30 to 35% of total facility power | ~38 to 40% of total facility power | Higher in absolute terms |
Sources: Netrality, High-Density Colocation for AI and GPU Workloads (Dec 2025) · U.S. Congressional Research Service, Data Centers and Energy Consumption (2025) · Hanwha Data Centers, AI Data Center Power Requirements (Dec 2025)
The cooling problem is a physics problem
Heat decides which class of building can hold which class of compute. NVIDIA's Blackwell GPUs put out up to 1,000 watts per chip, more than three times the heat of GPUs from seven years ago. [20] Rack densities in AI training facilities have gone from 15 kW, already near the top of what well-designed air cooling can handle, to 120 to 140 kW. Dell'Oro Group reports that liquid cooling revenue doubled in a single year and projects racks reaching 600 kW in the near term, with 1 MW configurations already under consideration. [21]
Air cannot carry heat away fast enough at these densities. At 30 kW per rack, CRAH-based cooling struggles and hotspots appear. Above 40 kW, air alone is not enough, and above 60 kW it does not work without liquid. The plant required at 140 kW (chilled water loops, coolant distribution units, direct-to-chip cold plates, leak detection, and purpose-built mechanical rooms) has almost nothing in common with the raised-floor CRAC room of a typical 2005 to 2015 enterprise data center.
| Cooling method | Max rack density supported | How it works | Retrofit to existing facility? |
|---|---|---|---|
| CRAH / CRAC air cooling | Up to ~20 kW | Chilled air circulated under raised floor and through hot/cold aisles; standard in pre-2015 enterprise builds | Existing standard; no changes needed for CPU workloads |
| In-row cooling units | ~15 to 25 kW | Cooling units placed between racks in the row; delivers cold air directly adjacent to heat sources | Moderate retrofit; requires row reconfiguration and chilled water supply |
| Rear-door heat exchangers (RDHx) | ~40 to 72 kW | Chilled-water radiator bolts to the rear door of an existing rack; cools exhaust air before it enters the room | Best retrofit option for legacy facilities entering inference; bolts on if chilled water is available |
| Direct-to-chip liquid cooling | ~80 to 120 kW | Cold plates mounted on CPUs and GPUs carry coolant directly to the chip; heat transferred to facility water loop via CDU | Significant retrofit; needs water distribution, CDUs, and leak detection, but feasible in phases |
| Immersion cooling | 100 to 250 kW+ | Entire servers submerged in dielectric fluid tanks; highest thermal efficiency, eliminates fans entirely | Major investment in floor loading, fluid handling, and vendor compatibility; not a retrofit for most legacy sites |
Sources: KAD, Data Center Rack Density in 2025 (Dec 2025) · Schneider Electric, Upgrade Legacy Data Centers with RDHx (Nov 2025)
What most people don't know about the numbers
The AI infrastructure conversation is almost all about compute: GPUs, chips, training runs. The physical plant gets less attention, and some of the numbers surprise even experienced operators.
| Data point | Figure | Why it matters | Source |
|---|---|---|---|
| Liquid cooling revenue growth | Doubled YoY in 2025 | The transition from air to liquid is happening faster than most operators have planned for; liquid cooling is no longer experimental | Dell'Oro Group, Q1 2025 |
| One GPU's daily energy use | ~30 kWh/day, about what a 4-person home uses | NVIDIA ships hundreds of thousands of GPUs a quarter; each one is a household's worth of power running around the clock at full load | Blocks & Files, Jul 2025 |
| Average rack density today | ~15 kW/rack industry average; AI workloads require 60 to 120 kW | The average facility is far below where AI workloads need to run; most of the installed base cannot hold GPU training | Dell'Oro Group, 2025 |
| Cooling's share of facility power | 38 to 40% of total data center electricity | Roughly 40 cents of every dollar of facility power goes to moving heat, which makes cooling efficiency a first-order cost | U.S. Congressional Research Service, 2025 |
| Half of U.S. data centers are over 10 years old | ~2,500 of ~5,000 U.S. facilities | Most of the installed base predates current density standards; retrofitting these sites, not building new, is where most of the near-term work will happen | Uptime Institute via Infinitum, Oct 2025 |
| Motor/cooling upgrade energy savings | ~20% reduction in total energy consumption | Replacing aging CRAH fan motors with IE5 motors, without touching the compute, cuts energy use and improves PUE | Infinitum, Oct 2025 |
| UPS efficiency: old vs. new | Legacy UPS: 80 to 90% efficient. Modern UPS: 95%+ | A 10-point UPS efficiency gap means a 3 to 25 MW facility is burning 300 kW to 2.5 MW on losses alone, an operating cost a modern UPS mostly removes | Data Center Dynamics, 2024 |
| Computational power per sq ft growth | 5× increase between 2020 and 2025 (projected) | The same footprint is expected to carry five times the compute load it was designed for, which stresses the structural and electrical assumptions of legacy builds | Gartner via Data Center Knowledge, 2024 |
What to do with legacy infrastructure: a practical guide
The most common mistake operators make with the AI narrative is treating it as a binary: either the facility is AI-ready or it is obsolete. Neither is true. "Can this building run GPU training?" is the wrong question. For most existing facilities the answer is no, and that is fine. The useful question is "what is this building good at, and how do I make it better at that?"
The rule from people who have done this work: start from the building, not the workload. Know your power envelope, floor loading, cooling plant, and utility connection before you commit to any rack environment. Operators who decide they want to host inference and then find out the substation has no spare capacity have wasted a lot of planning time. [22]
| Path | Best for | What you do | What you don't do | Capex intensity |
|---|---|---|---|---|
| 1. Efficiency modernization | Facilities serving stable CPU workloads (ERP, EHR, financial systems) with no near-term density pressure | Replace legacy UPS (target 95%+ efficiency), upgrade CRAH fan motors to IE5+, seal raised floor openings, improve hot/cold aisle containment, add DCIM metering | Touch the cooling plant or power distribution; too costly for the workload profile | Low; operational savings often pay for the upgrade within 2 to 4 years |
| 2. Density upgrade for inference | Retail colocation operators or enterprise facilities with available chilled water capacity wanting to capture AI inference demand | Add rear-door heat exchangers (RDHx) to a dedicated zone; upgrade power distribution to that zone to 30 to 60 kW/rack; add CDU if chilled water is available | Attempt whole-facility conversion; a zone-based approach keeps existing tenants in place and limits disruption | Moderate; bolt-on RDHx does not need a full plant overhaul |
| 3. Hybrid AI hub model | Operators with sufficient land and utility capacity to add new construction adjacent to existing facility | Build a liquid-cooled pod or module next to the existing air-cooled facility; run CPU in the old building and inference or light training in the new pod | Try to convert the main hall to 140 kW density; it will not work structurally or economically | High for new build, but existing facility remains productive throughout |
| 4. Strategic repositioning | Facilities where power, floor loading, or location make GPU upgrades uneconomical but CPU demand remains strong | Double down on the CPU workloads the facility is good at (enterprise colocation, healthcare, government, edge inference) and compete on uptime, compliance, and relationships rather than density | Chase hyperscale or AI training business that the building cannot physically support | Lowest; the moat is how well you run the site, not new plant |
Sources: Data Center Knowledge, Bridging the Gap (Apr 2025) · Schneider Electric (Nov 2025) · Data Center Knowledge, Retrofitting and ROI (2024)
The retrofit sequence that works
If you have decided to pursue a density or inference upgrade, the order matters as much as the equipment. The sequence below is what practitioners have found works, and it avoids the common failure of committing to a rack environment before the supporting plant is checked.
| Step | Action | What you're determining |
|---|---|---|
| 1 | Utility power audit: available capacity at the substation, interconnect queue position, upgrade lead times | Whether power growth is even possible, and on what timeline. Transformer backlogs of 24 to 36 months make this a gating constraint. |
| 2 | Structural engineering assessment: floor loading capacity, ceiling height, column spacing | Whether the floor can take 200 to 400 lbs/sq ft liquid-cooled rack loads; CDUs and overhead manifolds need 12 to 15 ft of ceiling clearance |
| 3 | Cooling plant assessment: chilled water availability, cooling tower capacity, outdoor heat rejection infrastructure | Whether RDHx is viable (requires chilled water supply); if not, whether a standalone CDU with dry coolers is feasible in the available yard space |
| 4 | Power distribution infrastructure: available ampacity, switchgear age, PDU architecture, UPS bypass capacity | The maximum rack density the existing electrical plant supports, and whether a zone upgrade is possible without replacing the main switchgear |
| 5 | Zone selection: identify a specific contiguous area (typically one row or one pod) for the density upgrade | Limits scope, protects existing tenants, and creates a production test environment before committing to full-facility changes |
| 6 | Deploy, measure, and iterate: instrument the upgraded zone with thermal, power, and coolant telemetry before scaling | Whether the assumptions from steps 1 to 5 hold under real load; catches design gaps before they get expensive at scale |
Sources: Data Center Knowledge (2024) · DLR Group, Data Center Adaptive Reuse (Apr 2025) · Schneider Electric (Nov 2025)
Is there still a place for legacy air-cooled infrastructure?
Yes. The 70% of enterprise workloads that have not moved to public cloud are not going to move to 140 kW GPU clusters either. The regulatory, latency, and operational reasons that kept them in regional Tier III colocation in 2020 are the same in 2026. Air-cooled infrastructure that is maintained and upgraded where it counts is a going concern.
These facilities don't compete on density. They compete on reliability, compliance record, and the relationships that come from decades of serving the same enterprise and healthcare customers. The 12 MW facility running hospital EHR systems in Memphis does not need GB200 racks. It needs 99.982% uptime, HIPAA-compliant physical access controls, staff who pick up the phone, and power costs that don't swing with the commodity markets.
What legacy operators should avoid is reactive capex: spending on GPU-oriented upgrades because the trade press says AI is coming, without a signed tenant or a clear read on whether the building can carry the load. The facilities that will struggle are not the ones that stayed CPU-focused. They are the ones that half-upgraded toward GPU density, spent the money, and then found the power or cooling limits that make full deployment impossible.
Who lives in the middle?
Three kinds of operator make up the 3 to 25 MW segment:
| Operator type | Market size / share | Typical workload | Source |
|---|---|---|---|
| Enterprise-owned private DC | ~70% of enterprise workloads not in public cloud | Oracle, SAP, EHR, financial systems, government | Gartner [8] |
| Retail colocation | 53 to 70% of the $69B to $84B colo market (2024) | 10 to 500 kW deployments; regional insurance, healthcare, finance | Grand View Research [9] |
| Tier III colocation | 56 to 58% of colocation market share (2024) | 99.982% uptime SLA; enterprise apps, compliance workloads | Mordor Intelligence [10] |
What the energy data shows
Lawrence Berkeley National Laboratory's 2024 U.S. Data Center Energy Usage Report gives the clearest view of where compute lives. In 2023, U.S. data centers used about 176 TWh of electricity, 4.4% of national electricity consumption. [11]
Conventional CPU servers used around 60 TWh of that in 2023, double the 30 TWh of 2014, which tracks the steady growth of enterprise and cloud workloads. GPU-accelerated servers went from under 2 TWh in 2017 to over 40 TWh in 2023. That is a steep curve, but it is still below the conventional server base in absolute terms as of the latest reported data. [6]
| Server type | 2014 | 2017 | 2023 | Growth (2014 to 2023) |
|---|---|---|---|---|
| Conventional CPU servers | ~30 TWh | ~38 TWh | ~60 TWh | +100% |
| GPU-accelerated servers | < 1 TWh | < 2 TWh | > 40 TWh | > 4,000% |
| Total U.S. DC consumption | ~70 TWh | ~90 TWh | ~176 TWh | +151% |
Sources: Lawrence Berkeley National Laboratory, 2024 U.S. Data Center Energy Usage Report · Brightlio, 2025
The IEA estimates AI-focused data center electricity demand is growing about 30% a year, against 9% for conventional server workloads. Both are growing. The non-AI base, still mostly CPU, is about 38 GW of existing global capacity, and it isn't being decommissioned. It's being refreshed, maintained, and run by people nobody writes about. [4]
The capital misallocation risk
Capital is flowing toward gigawatt AI campuses, and the semiconductor supply chain has reorganized around GPU and accelerator production. Yole Group reports that GPUs alone were $100 billion of the $209 billion data center semiconductor market in 2024, with NVIDIA taking 93% of server GPU revenue. [13]
| Category | Revenue (2024) | Share of total | Notes |
|---|---|---|---|
| GPU / AI accelerators | ~$100B | ~48% | NVIDIA holds 93% of server GPU revenue |
| CPU (x86 + ARM) | ~$14B | ~7% | Intel + AMD; demand uptick flagged Q4 2025 |
| Memory (HBM + DRAM) | ~$60B | ~29% | HBM3e dominates AI server configurations |
| Networking / other | ~$35B | ~17% | InfiniBand, Ethernet switching, NICs |
Meanwhile the supply chain for the middle market has tightened. High-power transformers and chillers are quoting 24 to 40 week deliveries, with transformer backlogs stretching to 36 months. [14] Hyperscalers have the purchasing power and balance sheets to work around that. The operator of a 12 MW facility in Columbus, Ohio does not. About 300,000 data center positions are projected to go unfilled in 2025, and the most specialized engineers are being pulled toward hyperscale AI projects paying 30% more, away from the middle market that needs them just as much.
Why the middle market holds up
Attention deficit aside, the 3 to 25 MW segment has structural features that make it durable, and for its target workloads it is often the better option than hyperscale.
| Factor | What it means in practice | Data point | Source |
|---|---|---|---|
| Regulatory immovability | HIPAA, SOX, PCI-DSS, and federal data residency requirements anchor workloads to controlled environments | 50% of critical enterprise apps outside public cloud through 2027 | Gartner |
| Hybrid IT pendulum | Cloud repatriation driven by egress costs, compliance overhead, and latency | 75% of organizations considering moving AI workloads from public cloud back to colocation | CoreSite State of DC 2024 |
| Distributed AI inference | Inference is delivered regionally, not from centralized training clusters | 30 to 50 kW/rack inference achievable in modernized middle-market facilities | Grand View Research |
| Non-AI absolute growth | 30% share of capacity growth on a $354B+ base is larger in dollar terms than the total AI DC market was 3 years ago | $1.5T non-AI capex projected through 2030 | McKinsey, 2025 |
What the middle market needs
Durable or not, the 3 to 25 MW segment has modernization work to do over the next 3 to 5 years.
| Imperative | Timeline | Detail |
|---|---|---|
| Power density upgrades | Near-term (1 to 3 yr) | AMD Turin's 10:1 socket consolidation pushes rack density past 6 to 8 kW ceilings even on a standard CPU refresh. The electrical upgrades need to land before the current generation ages out. [1] |
| Cooling modernization | Near-term (1 to 3 yr) | 2005-era CRAH units are inefficient vs. rear-door heat exchangers and in-row cooling retrofits. Full liquid cooling is a longer-horizon option for sites that begin hosting inference at moderate GPU densities. |
| AI inference readiness | Mid-term (2 to 4 yr) | 30 to 50 kW inference racks are achievable in modernized existing facilities. Healthcare, finance, and manufacturing inference does not require a hyperscale campus. |
| Renewable energy & sustainability | Mid-term (2 to 4 yr) | 72% of enterprises ranked sustainability as a major site-selection factor in 2024, up from 48% two years earlier. Operators without a renewable procurement plan lose ground in RFPs. [14] |
The bottom line
The AI buildout is real and so are the gigawatt campuses. But even Microsoft's Fairwater, the most advanced AI facility ever built, needs a dedicated 48 MW CPU building to keep its GPU cluster fed. The missing middle was never missing from the ground, only from the conversation. Roughly 38 GW of conventional data center capacity is running globally right now, serving the 70% of enterprise workloads that haven't moved to hyperscale cloud and won't. That infrastructure needs to be upgraded, staffed, powered, and run. The operators who focus on it don't need to out-build the hyperscalers. They need to serve the market the hyperscalers are not building for, and that market is most of the economy.
Sources
- SemiAnalysis, CPUs Are Back: The Datacenter CPU Landscape in 2026 (Gerald Wong & Dylan Patel, Feb 2026)
- Goldman Sachs Research, AI to Drive 165% Increase in Data Center Power Demand by 2030 (2025). Workload mix: cloud 54%, traditional 32%, AI 14%
- JLL, 2026 Global Data Center Outlook (Jan 2026). AI represented ~25% of workloads in 2025
- Programs.com, Data Center Statistics 2026 (Jan 2026). Non-AI workloads: 38 GW globally
- Polaris Market Research, Global Data Center Market, 2025 to 2034
- Brightlio, 255 Data Center Statistics (2025)
- Data Center Dynamics, Microsoft Launches Atlanta Fairwater Data Center (Nov 2025)
- Gartner, 50% of Critical Enterprise Applications Outside Public Cloud Through 2027
- Grand View Research, Data Center Colocation Market Report
- Mordor Intelligence, Data Center Colocation Market Size & Trends
- Lawrence Berkeley National Laboratory, 2024 U.S. Data Center Energy Usage Report
- IEA, Energy and AI: Energy Demand from AI (April 2025)
- Yole Group, Data Center Semiconductor Trends 2025
- MarketsandMarkets, Data Center Colocation Market
- CoreSite, State of the Data Center 2024
- McKinsey, The Cost of Compute: A $7 Trillion Race to Scale Data Centers (2025)
- Microsoft Source, From Wisconsin to Atlanta: Microsoft's First AI Superfactory
- SemiAnalysis, Microsoft's AI Strategy Deconstructed: From Energy to Tokens (Nov 2025)
- Netrality, High-Density Colocation for AI and GPU Workloads (Dec 2025). CPU server 300 to 500 W vs. GPU server 6,500 to 7,500 W; circuit and cooling requirements
- MLQ.ai, AI Data Center Cooling: Vertiv, Modine, Schneider (2025). Blackwell GPU 1,000 W/chip; rack densities 120 to 132 kW; Vertiv $9.5B backlog
- Blocks & Files, Power Consumption and Data Centers (Jul 2025). Dell'Oro Group: liquid cooling revenue doubled, racks nearing 600 kW; one GPU = daily energy of a 4-person home
- Data Center Knowledge, Retrofitting, Refurbishment, and the ROI for Legacy Data Centers (2024). Start from the building, not the workload; compute per sq ft projected to 5× between 2020 and 2025
- Data Center Knowledge, Bridging the Gap Between Legacy Infrastructure and AI-Optimized Data Centers (Apr 2025)
- Infinitum, Retrofit Revolution: Why Reviving Old Data Centers Is Critical (Oct 2025). About half of U.S. DCs over 10 years old; 20% energy savings from motor upgrades
- Schneider Electric, Upgrade Legacy Data Centers for AI Workloads with RDHx (Nov 2025)
- Data Center Dynamics, Five Reasons to Upgrade Legacy Data Center Power Infrastructure (2024). Legacy UPS 80 to 90% vs. modern 95%+
- DLR Group, Data Center Adaptive Reuse: 5 Strategies for Existing Buildings (Apr 2025)
Frequently asked questions
What is the 'missing middle' in data centers?
The missing middle is the 3 to 25 MW data centers that run enterprise workloads such as hospital records, banking systems, and ERP. Non-AI workloads draw 86% of global data center power, yet these facilities get almost no attention next to gigawatt AI campuses.
Do legacy data centers need to be converted to support AI workloads?
No. Most legacy facilities should stick to what they're built for: CPU workloads like ERP, databases, and healthcare systems that need uptime and compliance rather than GPU training.
Can existing data centers be retrofitted for AI inference?
Yes, selectively. A facility with spare chilled water capacity can add rear-door heat exchangers and upgrade power distribution in one zone to support 30 to 60 kW inference racks without converting the whole building.
Why are CPUs still in high demand if AI uses GPUs?
CPUs run reinforcement learning environments, the API calls behind agentic inference, multimodal data preprocessing, and the whole installed base of enterprise workloads. Even Microsoft's Fairwater AI campus needs a dedicated 48 MW CPU building to support its 295 MW GPU cluster.
What's the power difference between CPU and GPU servers?
A standard 1U CPU server draws 300 to 500 watts. A 4U GPU server with eight H100s draws 6,500 to 7,500 watts, roughly 15 to 20 times more, and needs different electrical and cooling infrastructure.
What cooling methods work for different rack densities?
Air cooling handles up to about 20 kW per rack, rear-door heat exchangers 40 to 72 kW, direct-to-chip liquid cooling 80 to 120 kW, and immersion cooling 250 kW and above for the densest GPU training racks.
Is it cheaper to retrofit or build new data center capacity?
It depends on the facility's power envelope, floor loading, and workload. If the site has the fundamentals, retrofitting for efficiency or a moderate density upgrade is usually faster and cheaper than new construction.
What should operators prioritize when upgrading legacy facilities?
Start with a utility power audit, then check floor loading, cooling plant capacity, and power distribution before you choose a rack environment or commit capital to a density upgrade.
Why aren't enterprise workloads moving to hyperscale cloud?
Regulation (HIPAA, SOX, PCI-DSS), data residency rules, latency, and cloud repatriation keep 70% of enterprise workloads in private or colocation facilities through 2027.
What percentage of data center energy goes to AI workloads?
In 2023, GPU-accelerated servers used over 40 TWh of U.S. data center electricity and conventional CPU servers used about 60 TWh. AI is growing faster but still has the smaller share.
Last updated: March 2026