Nvidia announces new H200 GPU with 141GB of HBM3e memory

Nvidia has recently unveiled the HGX H200, the newest addition to its AI hardware lineup featuring the Nvidia Hopper architecture.

Scheduled for release in Q2 2024, the H200 Tensor Core GPU introduces the new HBM3e memory technology, a significant upgrade from its predecessors, the H100 and the A100.

The new H200 GPU offers an impressive 141GB of HBM3e memory at a remarkable 4.8 terabytes per second, almost doubling the capacity and delivering 2.4 times more bandwidth than the A100. This is also a significant improvement over the H100, which had 80GB of HBM3 and a memory bandwidth of 3.5 TB/s.

Designed to meet the demands of scientific computing and high-performance computing (HPC) workloads, the H200 is poised to make a substantial impact. Ian Buck, the VP of Hyperscale and HPC at Nvidia, emphasises its role in efficiently processing vast data for generative AI and HPC applications.

"To create intelligence with generative AI and HPC applications, vast amounts of data must be efficiently processed at high speed using large, fast GPU memory," wrote Ian Buck in a press release. "With Nvidia H200, the industry’s leading end-to-end AI supercomputing platform just got faster to solve some of the world’s most important challenges."

Prominent supporters of this cutting-edge chip include major server vendors such as Dell Technologies, Hewlett Packard Enterprise, Lenovo, and Supermicro, along with leading cloud service providers such as Amazon Web Services, Google Cloud, Microsoft Azure, and Oracle Cloud. Emerging players in the cloud sector, such as CoreWeave, Lambda, and Vultr, are also part of the roster.

One standout feature of the H200 is its compatibility with existing systems, available in four and eight-way configurations. This flexibility allows for seamless integration into the systems of partner server makers and is a key component of the GH200 Grace Hopper Superchip.

This seamless compatibility enables server and cloud vendors, including the aforementioned entities, to effortlessly upgrade their systems with the new GPU.

Nvidia says that the H200 will significantly boost performance, nearly doubling the speed for tasks on Llama 2, a large language model with 70 billion parameters, compared to the H100. We can also expect more performance improvements with future software updates for the H200.