What’s the right temperature for water in liquid cooled systems?
Water is a core part of many data center cooling systems. But as densities - and therefore temperatures - increase, questions need to be asked about the right temperatures of the water cooling these systems.As the chips running servers become denser and more powerful, operators are faced with questions around whether to lower the temperature of the water going to these chips, to the point we will have to start focusing more on cooling the water systems.
Historically, data centers have been kept at around 20°C to 22°C, but groups such as the American Society of Heating, Refrigeration and Air-Conditioning Engineers (ASHRAE) have been advising organizations to set thermostats higher for years. As a result, data center temperatures have been creeping up: Facebook parent company Meta raised its temperatures to 29.4°C, Google went up to 26.6°C, and Microsoft has published guidelines suggesting temperatures could go up to 27°C. Typical legacy data centers have chilled water set points between 42-45°F (6-7°C). Facilities that have gone through optimization of their cooling systems have successfully raised their chilled water temperatures to 50°F (10°C) or higher.
Cooling has always been the second-largest consumer of energy in the data center after the IT load, and this is mostly energy used to cool whatever the heat transfer medium is - be it air or liquid. So the less energy is spent there, the better the overall efficiency of the facility. The picture is changing as the industry moves towards predominantly liquid-cooled data centers, where a liquid such as water circulates directly over the heat-producing components and removes heat. Water has a much higher thermal capacity than air, meaning data centers can support higher-density chips and use less energy to cool them.
The fluid running over liquid systems has a much higher temperature than that found in chilled water systems, but the industry is yet to standardize on the best approach. At the same time, chips are becoming increasingly dense, and the temperatures of the water being supplied to these systems is coming down. Data center operators have long been accused of being over-cautious by overcooling their air-cooled data centers to protect the IT hardware and avoid even the merest risk of overheating the data halls. Showing too much trepidation on liquid cooling risks the same issue.Higher water temperatures mean less energy used for cooling – great for PUE – but risks running chips closer to their thermal limit.
There is no optimal temperature for water in liquid-cooled systems, because the best temperature will vary depending on the set-up of the facility."This will depend entirely on the type of liquid cooling used as well as the environment the liquid cooling system is in, the type of chip and its TDP as well as the utilization of the chip.water temperatures in liquid-cooled systems today seem to be converging around 32°C (89.6°F) for facility water – what is described as a "good balance" between facility efficiency, cooling capacity, and support for a wide range of DLC systems. The company notes, however, that this often requires additional heat rejection infrastructure either in the form of water evaporation or mechanical cooling for higher-density chips.
Many operators have already opted for conservative water temperatures as they upgrade their facilities to incorporate a blend of air and liquid cooled IT. Others will install DLC systems that are not connected to a water supply but are air-cooled using fans and large radiators. As the chips get more powerful and more power hungry, the internal dissipation that needs to happen to the chip case housing requires colder water to be able to still support reliable cooling of those chips.
