The NVIDIA H100 NVL 94 GB is a datacenter GPU accelerator ("Hopper" class) designed for the most demanding workloads, including generative AI / LLM inference, HPC, and large-scale analytics. The NVL version is designed to operate in pairs of GPUs connected by NVLink, in order to increase usable memory capacity and inter-GPU throughput, while remaining in a PCIe format suitable for many servers.
Key Points
-
Very large memory per GPU and massive memory bandwidth, ideal for large models and low-latency serving.
-
NVLink between two cards (NVL): allows for creating a pair with a total of 188 GB HBM3 (94 GB + 94 GB) and a high-speed link between the GPUs.
-
Designed for 24/7 server operation (passive cooling depending on integration) and the NVIDIA ecosystem for AI/HPC.
Main Technical Details
-
Architecture: NVIDIA Hopper (H100)
-
Memory: 94 GB HBM3 per GPU (often used in a 188 GB pair via NVLink)
-
Memory Bandwidth: up to 3.9 TB/s per GPU
-
Interface: PCIe Gen5 x16
-
Interconnect: NVLink (NVL pair, high-speed link between two H100 NVL)
-
TDP (max power): up to 400 W per GPU (according to NVL specifications)
-
AI Acceleration (Tensor Cores + Transformer Engine): optimized for formats and computations used by LLMs (FP8/FP16/BF16/TF32 depending on modes)
-
Decoding: mention of 7× NVDEC (and 7× JPEG) on the H100 NVL specs
Who is it for?
-
AI / LLM (inference): serving large models, large contexts, significant batches, controlled latency.
-
HPC / simulation / data analytics: intensive computing and memory-bandwidth bound workloads.
-
Server infrastructures: dense deployments where PCIe format and production efficiency matter.
