Microfluidics Could Be the Answer to Cooling AI Chips

Data center rack density has risen rapidly in recent years. Operators are cramming more computing power into each server rack to meet the needs of AI and other high-performance computing applications. That means that each rack needs more kilowatts of energy, and ultimately generates more heat. Cooling infrastructure has struggled to keep pace.

Rack densities have gone from an average of 6 kilowatts per rack eight years ago to the point where racks are now shipping with 270 kW," says David Holmes, the global industries CTO at Dell Technologies. Next year, 480 kW will be ready, and megawatt racks will be with us within two years."

Corintis, a Swiss company, is developing a technology called microfluidics, in which water or another cooling liquid is channeled directly to specific parts of a chip to prevent overheating. In a recent test with Microsoft, servers running the company's Teams video conferencing software recorded heat removal rates three times as efficient as other existing cooling methods.Compared to traditional air cooling, microfluidics lowered chip temperatures by more than 80 percent.

Boosting Chip Performance with Microfluidics

A lower chip temperature allows the chip to execute instructions more quickly, increasing its performance. Chips operating at lower temperatures are also more energy efficient and have lower failure rates. Further, the temperature of the air used for cooling can be increased, making the data center more energy efficient by reducing the need for chillers, and lowering liquid consumption.

The amount of water needed to cool a chip can be lowered considerably by targeting the liquid's flow to the locations on the chip that are generating the most heat. Van Erp noted that the current industry standard is approximately 1.5 liters per minute per kilowatt of power. As chips are nearing10 kW, this will soon mean 15 liters per minute to cool one chip-a number that will raise the ire of communities worried about the impact of any supersized AI factories" planned for their areas that could contain a million or more GPUs.

We need optimized chip-specific liquid cooling, to make sure every droplet of liquid goes to the right location," says Remco van Erp, the co-founder and CEO of Corintis.

Corintis' founders Sam Harrison [left] and Remco van Erp hold a cold plate and microfluidic core, respectively.Corintis

The simulation and optimization software developed by Corintis is used to design a network of microscopically small channels on cold plates. Just like arteries, veins, and capillaries in the circulatory system of the body, the ideal cold plate design for each type of chip is a complex network of precisely shaped channels.

Corintis has scaled up its additive manufacturing capabilities to be able to mass-produce copper parts with channels as narrow as a human hair, about 70 micrometers. Its cold plate technology is compatible with today's liquid cooling systems.

The company believes this approach can improve cold plate results by at least 25 percent. By working with chip manufacturers directly to carve channels into the silicon itself, Corintis believes tenfold gains in cooling can eventually be realized.

Advancing Liquid Cooling for AI Chips

Liquid cooling is far from new. The IBM 360 mainframe, for example, was cooled by water more than half a century ago. Modern day liquid cooling has largely been a contest between immersion systems-in which racks and sometimes entire rows of equipment are submerged in a cooling fluid-and direct-to-chip systems-in which a cooling fluid is channeled to a cold plate placed against a chip.

Immersion cooling is not yet ready for prime time. And while direct-to-chip cooling is being widely deployed to keep GPUs cool, it only cools around the surface of the chip.

Liquid cooling in today's form is a one-size-fits-all solution, relying on simplistic designs that are not adapted to the chip, which prevents good heat transfer," says van Erp. The optimal design for each chip is a complex network of precisely shaped micro-scale channels that are adapted to the chip to guide coolant to the most critical regions."

Corintis is already working with chip manufacturers on improved designs. Chip manufacturers are using the company's thermal emulation platform to program heat dissipation on silicon test chips with millimeter-scale resolution, and then sense the resulting temperature on the chip after the selected cooling method is installed. In other words, Corintis acts as the bridge between chip design and cooling system design, enabling chip designers to build future chips for AI applications with superior thermal performance.

The next stage is to go from being a bridge between cooling channel and chip design to unification of those two processes. Modern chips and cooling are currently two discrete elements with the interface between the two being one of the main bottlenecks for heat transfer," says van Erp.

To improve cooling performance by tenfold, Corintis is betting on a future where cooling is tightly coupled as an integral part of the chip itself-the microfluidic cooling channels will be etched directly inside the microprocessor package rather than on cold plates on the perimeter.

Corintis has produced more than 10,000 copper cold plates, and is ramping its manufacturing capabilities to reach a million cold plates by the end of 2026. It has also developed a prototype line in Switzerland where it is developing cooling channels directly within chips rather than onto a cold plate. This is only planned for small quantities to demonstrate basic concepts that will then be turned over to chip makers and cold plate manufacturers.

Corintis announced these expansion plans immediately following the publication of the Microsoft Teams tests. In addition, it is opening U.S. offices to serve U.S. customers and an Engineering office in Munich, Germany. In addition, the company also announced the completion of a US $24 million Series A funding round led by BlueYard Capital and other investors.