Managing 16 Rome CPUs in 1U: Cray’s Shasta Direct Liquid Cooling
The Supercomputing show was a hive of activity, with lots of whispers surrounding the next generation of x86 CPUs, such as AMD’s Rome platform and Intel’s Cascade Lake platform. One of the big announcements at the start of the show is that Cray has developed an AMD Rome based version of its Shasta supercomputer deployments.
Cray had a sizeable presence at the show, given its position as one of the most prominent supercomputer designers in the industry. At its booth, we saw one of the compute blades for the Shasta design:
It’s a bit difficult to see what is going on here, but there are what looks like eight processors in a 1U system. The processors are not facing in the traditional way for a server, in line with airflow, because the system is not air cooled.
The cooling plate and pipes connect to separate reservoirs that also supply additional cooling to the memory. Of course in this situation we’re not in a dense memory configuration, given the cooling required.
This display was being shown off in the CoolIT booth. CoolIT is a well-known specialist of liquid cooling for both the consumer and enterprise worlds, and they had a few interesting points on their implementation for Shasta.
Firstly, the eight CPU design with split memory cooling was obvious, but CoolIT stated that these processor cooling plates were designed to cool CPUs above and below, for a total cooling power of 500W apiece. This means that in this design, the full cooling apparatus could go up to sixteen CPUs and 4kW.
The cooling loops are split between the front and the rear, recombining again near the front. This means that four cooling plates (or 8 CPUs) and memory will be pumping through in each loop. There is a small question about how the memory all fits in, physically, but this is certainly impressive.
CoolIT is claiming that its custom solution for Cray’s Shasta platform helps double the compute density over an aircooled system, offers 20% better rack-level efficiency, and up to 30% better facility-wide energy efficiency.
Shasta is set to be deployed at NERSC as part of its upcoming ‘Perlmutter’ supercomputer, slated for delivery in late 2020.