Rent NVIDIA H100 SXM cloud GPU
The most powerful data center GPU ever built. 80GB HBM3, 16,896 CUDA cores, Transformer Engine, and 3,350 GB/s memory bandwidth, starting at $2.60/hr on OpenRelay. Train and deploy the largest AI models at a fraction of hyperscaler prices.
NVIDIA H100: the gold standard for enterprise AI
The NVIDIA H100 SXM is the flagship data center GPU built on the Hopper architecture, released in 2023. It represents a generational leap in AI compute, featuring the revolutionary Transformer Engine that automatically applies mixed FP8/FP16 precision to accelerate transformer-based models by up to 3x over the previous-generation A100.
With 80GB of HBM3 high-bandwidth memory delivering 3,350 GB/s of throughput, the H100 eliminates the memory bottleneck that limits smaller GPUs when running large language models. Models that can't fit on 24GB consumer GPUs, such as LLaVA-34B at FP16 or Llama 3 70B in 8-bit, run on a single H100.
The H100 is the GPU that's training the next generation of foundation models. On OpenRelay, you can access this compute at $2.60/hr. Whether you're training custom models, running high-throughput inference, or doing cutting-edge research, the H100 on OpenRelay gives you enterprise-grade performance without enterprise-grade pricing.
H100 SXM technical specifications
| Specification | H100 SXM |
|---|---|
| GPU Architecture | Hopper (GH100) |
| VRAM | 80GB HBM3 |
| CUDA Cores | 16,896 |
| Memory Bandwidth | 3,350 GB/s |
| FP32 Performance | 67 TFLOPS |
| FP16 (Tensor) | 989 TFLOPS (with sparsity: 1,979) |
| FP8 (Tensor) | 1,979 TFLOPS (with sparsity: 3,958) |
| TDP | 700W |
| Memory Type | HBM3 (5th Gen) |
| Tensor Cores | 528 (4th Gen) |
| NVLink | 4th Gen, 900 GB/s |
| Transformer Engine | Yes (FP8 auto-mixed precision) |
The H100's standout feature is its Transformer Engine, which dynamically switches between FP8 and FP16 precision during computation. This delivers up to 3x the throughput of A100 for transformer models without any code changes, the hardware handles precision management automatically. Combined with 3,350 GB/s of HBM3 bandwidth (1.7x the A100 80GB), the H100 is purpose-built for the large language models that define modern AI.
H100 cloud GPU pricing on OpenRelay
OpenRelay's H100 pricing is dramatically lower than hyperscalers. AWS charges $4.76/hr for p5 H100 instances, GCP charges $4.52/hr, and Azure charges $3.67/hr. Even dedicated GPU providers like CoreWeave charge $2.21/hr for A100s (not H100s). At $2.60/hr, OpenRelay offers the best price-to-performance for H100 access in the cloud.
| Provider | GPU | $/hour | $/month |
|---|---|---|---|
| OpenRelay | H100 SXM | $2.60 | $1,872 |
| Azure | H100 (80GB) | $3.67 | $2,642 |
| GCP | H100 (80GB) | $4.52 | $3,254 |
| AWS | H100 (80GB) | $4.76 | $3,427 |
| Lambda Labs | H100 (80GB) | $2.99 | $2,153 |
All OpenRelay pricing includes storage, load balancing, and network egress. No hidden fees.
Best use cases for the H100
The H100 is the right choice when your workload demands the absolute highest performance, the most VRAM, or features only available on the Hopper architecture. Here's where it excels:
Large Language Model Training
Fine-tune 70B models with QLoRA on one GPU, or train larger models across several, with the H100's Transformer Engine, FP8 support, and 80GB HBM3. The H100 delivers up to 3x the training throughput of the A100. Multi-GPU training via NVLink enables scaling to even larger models. Whether you're training domain-specific LLMs or fine-tuning foundation models, the H100 is the fastest path to results.
Large Model Inference (70B+)
Run Llama 3 70B or Mixtral 8x7B in 8-bit, or DBRX in 4-bit, on one GPU. The 3,350 GB/s of memory bandwidth keeps token generation fast on models this size.
Multi-Modal AI & Vision-Language Models
Deploy large vision-language models like LLaVA, CogVLM, and GPT-4V-class models that combine text and image understanding. These models often require 40-80GB VRAM and benefit enormously from the H100's memory bandwidth and Transformer Engine acceleration for both the vision encoder and language decoder components.
High-Throughput Inference Serving
Serve thousands of concurrent users with continuous batching on the H100. The massive memory allows larger batch sizes, the Transformer Engine accelerates every forward pass, and the HBM3 bandwidth keeps the GPU fed with data. For production APIs that need to handle burst traffic at scale, the H100 provides unmatched throughput per GPU.
AI Research & Experimentation
Push the boundaries of AI research with the H100's cutting-edge capabilities. FP8 precision, dynamic parallelism, and the Transformer Engine enable experiments that simply aren't possible on older hardware. Whether you're exploring new architectures, novel training techniques, or scaling laws, the H100 gives you the compute headroom to iterate quickly.
Scientific Computing & HPC
Accelerate molecular dynamics, climate modeling, genomics, and other HPC workloads. The H100's 67 TFLOPS FP32 and massive memory bandwidth make it ideal for double-precision scientific computing. NVLink enables multi-GPU communication at 900 GB/s for distributed simulations that span multiple GPUs.
How to deploy an H100 on OpenRelay
Accessing H100 compute on OpenRelay is just as simple as any other GPU, no months-long waitlists, no capacity negotiations, no enterprise sales calls. Here's the process:
Create your account
Sign up at openrelay.inc/get-started. No upfront commitments or lengthy procurement processes. Get access to the dashboard and CLI immediately upon signup.
Select the H100 SXM
Choose the H100 SXM from the GPU catalog. Configure your deployment: single GPU or multi-GPU, region preferences, and container image. OpenRelay supports any Docker image, including popular ML frameworks like vLLM, TGI, and Triton.
Deploy with full GPU passthrough
OpenRelay provisions your H100 with VFIO passthrough in a lightweight VM. Your workload gets bare-metal GPU performance with strong security isolation. The full 80GB HBM3 and all Tensor Cores are exclusively yours, no GPU sharing or virtualization overhead.
Scale and monitor
Your H100 deployment includes auto-failover, load balancing, and real-time monitoring. Track GPU utilization, memory usage, and inference throughput from the dashboard. Scale up by adding more GPUs, or scale down when demand drops, billing is per-minute, so you only pay for what you use.
H100 vs A100: a generational leap
The H100 represents a massive leap over the A100 across every metric that matters for AI workloads. Here's a direct comparison:
| Feature | H100 SXM | A100 80GB |
|---|---|---|
| VRAM | 80GB HBM3 | 80GB HBM2e |
| Memory BW | 3,350 GB/s | 1,935 GB/s |
| FP16 Tensor | 989 TFLOPS | 312 TFLOPS |
| FP8 Support | Yes (1,979 TFLOPS) | No |
| Transformer Engine | Yes | No |
The H100 delivers up to 3x the throughput of the A100 for transformer workloads. The Transformer Engine alone can double your effective throughput without any code changes, something the A100 simply cannot match.
Frequently asked questions
How much does it cost to rent an H100 on OpenRelay?
OpenRelay offers NVIDIA H100 SXM cloud GPUs at $2.60 per hour with per-minute billing. That works out to approximately $1,872 per month for 24/7 usage. There are no minimum commitments, no egress fees, and no hidden costs. This is significantly less than hyperscalers like AWS ($4.76/hr) and GCP ($4.52/hr) for comparable H100 instances.
What is the difference between H100 SXM and H100 PCIe?
The H100 SXM uses NVIDIA's SXM5 form factor, which provides higher power delivery (700W vs 350W for PCIe) and supports NVLink for high-speed multi-GPU communication at 900 GB/s. It also carries more CUDA cores (16,896 vs 14,592) and HBM3 at 3,350 GB/s instead of HBM2e at 2,000 GB/s. Both have 80GB of VRAM, so the same models fit on either. The SXM is the one to pick for multi-GPU training.
What AI models require an H100?
The H100's 80GB HBM3 runs 30B-class models such as LLaVA-34B at FP16, and Llama 3 70B or Mixtral 8x7B in 8-bit. It's also the preferred GPU for training large models from scratch, as the Transformer Engine and FP8 support dramatically accelerate training throughput.
How does the H100 compare to the A100 for AI workloads?
The H100 is up to 3x faster than the A100 for transformer-based models thanks to its Transformer Engine, FP8 support, and 4th-gen Tensor Cores. Memory bandwidth is 3,350 GB/s (vs 1,935 GB/s on the A100 80GB), which is critical for large language model inference.
Can I use the H100 for model training on OpenRelay?
Yes. The H100 SXM is designed for both training and inference. Its 80GB HBM3, Transformer Engine, and FP8 support make it the fastest single-GPU option for training. OpenRelay supports multi-GPU deployments for distributed training workloads, with auto-failover to ensure your training runs complete even if individual nodes encounter issues.
Ready to deploy on the H100?
Access the most powerful GPU in the world at a fraction of hyperscaler prices. No minimum commitments. No egress fees.