AI workloads are reshaping infrastructure - here’s what data centers need to know

Artificial Intelligence (AI) is being adopted across industries at remarkable speed. From finance to healthcare, AI is driving new services and unlocking new business models – fundamentally changing the way we all live, learn and work. But with progress comes challenges, and in the case of AI, fast adoption means that the infrastructure supporting it is under increasing pressure.

Data centers, once optimized for traditional enterprise workloads, are being pushed to accommodate entirely new operating patterns. The rise in high-performance computing means more power, more heat, and more volatility. Established systems are struggling to keep up.

AI workloads don’t just demand scale. They require IT infrastructure that can react to dynamic, unpredictable demand. And as organizations expand their use of AI, the supporting environment must evolve too.

Rack density is climbing quickly

One of the clearest shifts data center operators are experiencing is in rack density. Standard deployments have typically operated at around 10 kilowatts to 15 kilowatts per rack. But AI hardware - especially clusters of Graphics Processing Units (GPUs) - consumes much more power and generates far more heat.

In many AI deployments, racks now draw 40 kilowatts or more. Some experimental training environments exceed 100 kilowatts. This isn’t just about energy consumption. It’s a challenge for every part of the power chain, from uninterruptible power supply (UPS) systems to power distribution units (PDUs), to the facility’s own switchgear.

Older data centers may not be able to support these loads without major upgrades. For those expanding into AI, the layout, redundancy, and zoning of rack space needs careful planning to avoid creating thermal or electrical bottlenecks.

Cooling is reaching its limits

Conventional air cooling was never designed for today’s thermal loads. Even with hot aisle containment and optimized airflow, facilities are finding it hard to remove heat fast enough in high-density zones.

This is why liquid cooling is gaining ground. Direct-to-chip cooling systems, already common in high-performance cloud computing environments, are being adapted for data centers supporting AI and where densities exceed 50kW/rack. Immersion cooling is also being explored where space is tight or where energy efficiency is a priority where densities exceed 150kW/rack.

Installing liquid cooling involves significant changes - from plumbing and pumping systems to maintenance protocols and leak prevention. It’s a major shift, but one that is becoming necessary as traditional cooling approaches run out of headroom.

Load volatility is forcing a rethink

AI workloads behave differently from legacy compute. Training cycles can move from idle to peak and back in a matter of seconds. And inference jobs often run continuously, putting steady pressure on electrical and cooling infrastructure.

That variability puts systems under stress; power systems need to be fast and responsive and cooling systems must avoid overshooting or lagging behind, sensors and controls need to act in real time, not based on average load assumptions.

This is driving investment in smarter infrastructure. Software-based power management, predictive analytics, and environmental telemetry are no longer add-ons. They are becoming essential for resilience and efficiency.

Commissioning is getting more involved

Designing infrastructure for AI is one thing. Proving that it works under pressure is something else.

Commissioning teams are having to simulate conditions that didn’t exist just a few years ago. That includes sudden spikes in compute load, failure scenarios under high thermal pressure, and mixed environments where air and liquid cooling run side by side.

This means that simulation tools are being used earlier in the design process, with digital twins helping to test airflow and thermal modelling before equipment is installed. On-site commissioning now includes more functional testing, and more collaboration between electrical, mechanical and IT teams.

Power constraints are slowing progress

In some parts of the UK and Europe, getting access to the grid has become a significant barrier. Long connection times and limited capacity are delaying new builds and expansion projects.

This real and growing challenge is leading some operators to turn to on-site energy generation, energy storage systems, and modular buildouts that can grow in stages. Others are prioritizing regions with better access to power - even if they aren’t the original target location.

Cooling strategies are also directly affected. Liquid cooling systems require consistent energy supply to maintain stable operation. Any power disruption can quickly become a cooling issue, especially when workloads can’t be paused. And, in high-density environments, even brief interruptions to power can have thermal consequences within seconds - leaving no room for infrastructure to catch up after the fact.

Heat reuse is being taken seriously

AI workloads generate a lot of heat- and more than ever, operators are exploring ways to efficiently reuse waste heat.

In the past, heat recovery was often seen as too complex or not cost-effective. But with higher temperatures and more concentrated thermal output from liquid cooling systems, the picture is changing.

Some new facilities are being designed with heat export capabilities. Others are considering connections to local district heating systems. Where planning authorities are involved, expectations around environmental performance are rising, and heat reuse can be a strong point in a project’s favor.

Infrastructure is becoming more adaptive

AI is creating new expectations for data centre infrastructure. It needs to be fast, scalable and adaptable. Standardization helps, but flexibility is becoming more important - particularly as AI workloads evolve and spread from central hubs to the edge.

The next generation of data centers will need to manage high loads with minimal waste. They will need to recover energy where possible, stay efficient under pressure, and respond in real time to shifting demand.

This isn’t just about capacity. It’s about designing flexible systems that stay effective as conditions change.

We list the best IT Automation software.

This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro