By Andy Smith
Data centers are the backbone of nearly every modern industry, enabling system operations, data storage, connectivity, and communication worldwide. Telecommunications, e-commerce, media, healthcare, finance, education, security, and manufacturing are all supported by data centers. However, the thermal management of data centers is a massive challenge often overlooked.
This article identifies board, sub-system, and system-level cooling technologies that enable modern data centers to function efficiently and effectively.
Why data center cooling systems are critical
Data center thermal management is essential for equipment reliability, energy efficiency, preventing data loss, performance optimization, safety, cost, operational efficiency, and minimizing environmental impact.
Efficient thermal management ensures that individual electronic components operate within their optimal temperature ranges, also optimizing component-level energy consumption. As component temperatures rise, so does their internal resistance, which can decrease performance or increase the overall power consumption. Undercooling of components can lead to a massive increase in computational energy consumption. In contrast, overcooling or inefficient cooling systems can lead to unnecessary energy expenditure to maintain optimized computation energy consumption.
Data center cost is tied directly to energy consumption. Efficient thermal management strategies reduce overall operation costs. By optimizing cooling system energy costs against computational energy consumption costs, data center operators can minimize operation costs and meet computational performance demands.
The three levels of data center cooling
Board-level cooling
The primary heat generation in data centers comes from processing units such as CPU, GPU, and TPUs. These board-level components require effective heat dissipation to maintain proper internal temperatures.
The most common method of board-level cooling utilizes air as the medium for heat absorption. Data center boards use air-based cooling devices such as passive heat sinks, heat pipes, vapor chambers, active cooling fans, and blowers.
A liquid cooling system can be used when air-cooled thermal management systems don’t provide the desired cooling performance. Liquid-cooled CPUs, GPUs, ASICs, and computational accelerators are found in high-density data centers with specialized hardware requirements that generate heat beyond what traditional cooling methods can handle. Liquid cooling provides more effective heat dissipation for this type of hardware common in high-performance computing (HPC) tasks such as AI training, scientific simulations, complex data analysis, and graphics rendering.
Sub-system cooling
Liquid cooling can be used for entire data centers but is rarely a universal cooling solution due to its higher complexity and cost. Many data centers utilize a variety of cooling techniques from section to section. In the case of localized-liquid cooling systems, heat collected is often disposed of via liquid into the air, which is then transferred throughout the rest of the data center’s air-based cooling systems.
Airflow management is a core technique for overall data center thermal management. Given that most data center boards are arranged into racks, rack-level cooling systems can comprise dedicated cooling units or rack-mounted fans that direct airflow from one section of the data center to the other.
Data centers are commonly organized in hot and cold aisle configurations, whereby one side of a server faces a cold aisle, and the opposite side faces a hot aisle. Rack-level cooling units and fans draw air from the cold aisles, pass it through the rack to cool the boards and components, and then dispose of the air into the hot aisles. Hot Aisle Containment (HAC) solutions enclose aisles using physical barriers and doors to minimize mixing with ambient or cooled air, maintaining efficiency.
As an alternative to air-based rack cooling, chilled water and liquid cooling distribution units (CDUs and LCDUs) circulate chilled water throughout an entire rack to individual board-level liquid cooling devices. CDUs consist of a coolant loop, pump, and control system that can distribute the cooling liquid based on demand. They also consist of a heat exchanger that expels the heat generated within the rack to the data-center level air or liquid cooling system.
Liquid immersion cooling, the rarest form of data center sub-system cooling, is used in high-performance computing and technology sectors. These systems involve submerging entire server hardware systems into dielectric fluids to directly cool all board components simultaneously. This cooling method provides superior cooling capabilities but requires highly specialized infrastructure and is not as scalable as AC or chilled water-cooling systems. Dielectric fluids are also expensive, consumable, complicated to maintain, and can be hazardous to the environment, so most data centers use them sparingly.
System-level cooling
Cooling an entire data center requires substantial foresight for overall infrastructure, energy consumption, and location planning. Several systems are used depending on the overall temperature, humidity, and environmental conditions within the facility. The choice of which system to use largely depends on the industry and the expected demand/utilization of the data center.
Precise air conditioning systems
Precise air conditioning systems such as Computer Room Air Conditioners (CRAC) units are the most common data center cooling systems. CRAC units are utilized in data centers that use cooled, forced air to cool racks and utilize a centralized refrigeration cycle to do so, disposing of heated air into the exterior environment.
CRAC units provide precise temperature and humidity control as well as air filtration, which is crucial for sensitive equipment and data storage but less suited for HPC applications. CRAC units' cooled air distribution systems can vary from hot/cold aisle configuration to up-flow floor-to-ceiling distribution systems. Precision air-conditioned systems are most commonly found in banking, healthcare, technology, e-commerce, and entertainment-oriented data centers, given their high presence of data storage, consistent daily demand, and minimal HPC racks.
Example of an up-flow rack cooling configuration
Chilled water systems
Chilled water systems use water as a cooling medium circulated throughout the entire data center. These systems, also called hydronic cooling systems, are utilized by industries with high computation loads or in very large data centers, given their scalability. Chilled water systems are more efficient and can handle substantially higher heat loads than air conditioning systems, allowing them to manage variable heat demand throughout the day.
Chilled water systems also provide a high number of redundancies, which makes the overall system more reliable. Unfortunately, chilled water systems have a higher upfront cost. The central equipment, distribution piping, and rack-level cooling systems are more expensive than air-conditioned cooling systems. These systems can also be susceptible to leaks, requiring professional installation and maintenance to avoid damaging valuable server hardware.
However, because of their reliability, scalability, and increased system heat capacity, chilled water systems are commonly utilized in telecommunications, government, energy/utilities, security, manufacturing, and scientific sectors. Chilled water systems are the preferred cooling method for high computational load systems, whether by computation density or overall size.
Free cooling systems
Free cooling systems leverage external environmental conditions to cool data centers. Free cooling systems are utilized when energy efficiency is a priority, as they minimize their reliance on mechanical cooling systems. They work by taking in cool outside air, filtering it, using it to cool the data center racks, and then exhausting the warmed air back to the outdoor environment. These systems only exist where the external environment supports them, making these systems less common and usable for only select industries with low computational demand.
Data center cooling is fundamental
Most data center cooling consists of either a traditional air conditioning system or a chilled water system. The data center’s heat densities, available space, geographical location, and cost constraints are the most common deciding factors between the two systems. More advanced HPC data centers can consist of complex liquid immersion thermal management systems, providing the most capable thermal management solution. Still, their cost and complexity make them uncommon. The highly customized design of board, sub-system, and data center-level cooling systems largely depends on the industry demands, scalability, reliability, and energy efficiency goals. Regardless, thermal management solutions are essential for modern data center operations.
See related product
See related product
See related product
See related product
