How much VRAM does the NVIDIA A100 80GB have?

The NVIDIA A100 80GB has 80 GB of HBM2e with 2039 GB/s memory bandwidth.

What is the NVIDIA A100 80GB best for?

With 80 GB of VRAM, the NVIDIA A100 80GB is a server-class GPU that runs 70B-class dense models and large MoE models natively, with plenty of room for long context.

What LLMs can the NVIDIA A100 80GB run locally?

The NVIDIA A100 80GB can run 65 of the 97 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Qwen 3.8 27B at BF16, Ornith 1.5 35B-A3B (MoE) at Q8_0, Qwen 3.5 122B-A10B (MoE) at Q3_K_M.

Can the NVIDIA A100 80GB run Gemma 4 31B?

Yes. The NVIDIA A100 80GB runs Gemma 4 31B natively in VRAM at BF16 quantization, achieving approximately 21.1 tokens per second.

Can the NVIDIA A100 80GB run Qwen 3.6 27B?

Yes. The NVIDIA A100 80GB runs Qwen 3.6 27B natively in VRAM at BF16 quantization, achieving approximately 24.3 tokens per second.

Can the NVIDIA A100 80GB run Qwen3 8B?

Yes. The NVIDIA A100 80GB runs Qwen3 8B natively in VRAM at BF16 quantization, achieving approximately 77 tokens per second.

Can I rent the NVIDIA A100 80GB instead of buying it?

Yes: RunPod and similar cloud GPU providers let you rent NVIDIA A100 80GB instances by the hour, with no long-term contract. This is often cheaper than buying if you only need it occasionally, and lets you try the GPU before committing to a purchase.

NVIDIA A100 80GB

The NVIDIA A100 80GB has 80 GB VRAM and 2039 GB/s memory bandwidth. It can run 65 of our 97 tracked models natively in VRAM at 8k context.

With 80 GB HBM2e, the NVIDIA A100 80GB is a datacenter-tier GPU that can run 65 models natively. This site's calculator runs Llama 3.3 70B at Q6_K (67.37 GB, 22.0 tok/s) and GPT-OSS 120B (MoE) at Q3_K_M (70.46 GB, 156.2 tok/s) fully in VRAM: Ampere predates the Transformer Engine, so there's no FP8/FP4 fast path the way there is on H100 and newer; everything here runs on the same FP16/INT8 kernels llama.cpp and vLLM have had years to optimize. GLM-4.6 (355B, MoE) is out of reach on a single card at any quantization; even Q2_K needs 154.94 GB. Across the 97 models this site tracks, 65 fit fully in VRAM at 8k context, the exact same count as the newer H100 80GB, since both share the identical 80GB ceiling; the A100 only trails on speed. The H100 decodes the identical quant roughly 65% faster (48.1 vs 29.2 tok/s on Llama 3.3 70B at Q4_K_M), a gap that tracks almost exactly to the two cards' bandwidth difference (3,350 vs 2,039 GB/s).

The NVIDIA A100 80GB is the high-memory SXM variant of NVIDIA's Ampere datacenter GPU, featuring 80GB HBM2e at 2,039 GB/s. It runs 70B models at Q6_K in a single GPU, and remains widely available in cloud instances (AWS p4de, GCP a2-ultragpu). While superseded by the H100 for new deployments, A100s are heavily discounted on the secondhand market, making them popular for cost-conscious inference setups.

NVIDIA A100 80GB: Unveiled at GTC in May 2020 as Ampere's flagship datacenter GPU, initially only as a 40GB card; this 80GB HBM2e SXM4 variant followed that November (announced at SC20), doubling capacity and raising bandwidth from 1,555 to 2,039 GB/s. Built on the 826mm² GA100 die (TSMC 7nm, 54.2 billion transistors) with 6,912 CUDA cores, it introduced 3rd-generation Tensor Cores with TF32 and structural sparsity, draws up to 400W, and was NVIDIA's first Tensor Core GPU with Multi-Instance GPU (MIG) support, splitting one card into up to seven isolated instances for multi-tenant serving.

This site's calculator runs Llama 3.3 70B at Q6_K (67.37 GB, 22.0 tok/s) and GPT-OSS 120B (MoE) at Q3_K_M (70.46 GB, 156.2 tok/s) fully in VRAM: Ampere predates the Transformer Engine, so there's no FP8/FP4 fast path the way there is on H100 and newer; everything here runs on the same FP16/INT8 kernels llama.cpp and vLLM have had years to optimize. GLM-4.6 (355B, MoE) is out of reach on a single card at any quantization; even Q2_K needs 154.94 GB. Across the 97 models this site tracks, 65 fit fully in VRAM at 8k context, the exact same count as the newer H100 80GB, since both share the identical 80GB ceiling; the A100 only trails on speed. The H100 decodes the identical quant roughly 65% faster (48.1 vs 29.2 tok/s on Llama 3.3 70B at Q4_K_M), a gap that tracks almost exactly to the two cards' bandwidth difference (3,350 vs 2,039 GB/s).

Full CUDA support under compute capability 8.0 (sm_80), the most mature architecture this site tracks, with years of llama.cpp, vLLM, and TensorRT-LLM tuning behind it. Third-generation NVLink connects up to 8 SXM4 cards at 600 GB/s per GPU for multi-card serving; PCIe cards are limited to a slower bridge between pairs. Still the workhorse behind AWS p4d/p4de, Azure NDv4, and GCP A2 instances, and heavily discounted on the used market now that H100/H200 own the frontier tier; MIG partitioning lets a single 80GB card do useful multi-tenant duty even as newer silicon takes the largest jobs.

Vendor	NVIDIA
Architecture	Ampere
VRAM	80 GB
Memory type	HBM2e
Memory bandwidth	2039 GB/s
Compute backend	CUDA
Tier	Datacenter
Released	2020
Models (native)	65 / 97
Models (offload)	6 / 97

Software: Typically cloud-accessed. vLLM and TensorRT-LLM give best batched-inference performance.

The card everything since has been measured against

Plotting NVIDIA's last five datacenter flagships by VRAM and bandwidth together shows how far the lineage has moved since this card started it:

VRAM and memory bandwidth, from each card's real spec sheet. A card further right holds bigger models; a card further up decodes them faster once they fit.

This card (this page) sits at the floor of every axis on this chart: 80GB and 2,039 GB/s. Each step since has traded differently between the two numbers: H100 was bandwidth-led (+64% to 3,350 GB/s, capacity unchanged at 80GB), H200 flipped to capacity-led (+76% to 141GB on a smaller +43% bandwidth gain), B200 went bandwidth-led again (+67% to 8,000 GB/s on a smaller +28% capacity gain), and B300 is capacity-only (+60% to 288GB at the exact same 8,000 GB/s as B200). Add it up and five generations turned this card's 80GB/2,039 GB/s into 288GB/8,000 GB/s (3.6x the capacity, 3.9x the bandwidth), but every one of those generations is still a multiple of what this card shipped first.

This card's HBM2e jump over its own 40GB sibling, and what came next

The 40GB and 80GB A100 aren't different chips; both use the same 826mm² GA100 die. The only thing that changes between them is the memory: older HBM2 on the 40GB card, newer HBM2e on this 80GB card. Tracking bandwidth from the 40GB card through this one to the next NVIDIA generation:

NVIDIA A100 40GB

1555 GB/s

NVIDIA A100 80GB (this page)

2039 GB/s

NVIDIA H100 80GB

3350 GB/s

Switching from the 40GB card's HBM2 to this card's HBM2e is a 31% bandwidth jump (1,555 to 2,039 GB/s) on the exact same GA100 die, a pure memory-technology upgrade, no new architecture involved. The next NVIDIA generation pushed further: the H100's HBM3 adds another 64% on a new GH100 die, to 3,350 GB/s. Since decode is bandwidth-bound, this site's calculator measures that chain directly: Llama 3.1 8B at Q4_K_M runs 170.0 tok/s on the 40GB card, 222.9 tok/s here, and 366.2 tok/s on the H100, each step gaining almost exactly its share of bandwidth.

Cloud GPU Rental

Don't want to buy a NVIDIA A100 80GB? RunPod is a cloud GPU rental service: rent one by the hour instead, no contract, no upfront hardware cost.

Pay by the hour · no contract · pods start in about a minute.

Rent a NVIDIA A100 80GB on RunPod ↗ (+$5 signup credit)

Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.

Popular models for this GPU

Mixtral 8x22B Instruct v0.1 Mistral Medium 3.5 128B Qwen 3.5 122B-A10B (MoE)Nemotron 3 Super 120B GPT-OSS 120B

Models this GPU runs natively in VRAM (65)

Show 60 more

Models that fit with CPU offload (6)

These use system RAM for layers that don't fit in VRAM, so expect much slower inference.

Too large for this GPU (26)

Compare NVIDIA A100 80GB with other GPUs

Continue reading

vram-guides10 min

DeepSeek Family Guide: R1, V3 & Distilled Models

hardware10 min

Used Enterprise GPUs for LLMs: A6000, A100, and Beyond

Frequently asked questions

How much VRAM does the NVIDIA A100 80GB have?: The NVIDIA A100 80GB has 80 GB of HBM2e with 2039 GB/s memory bandwidth.
What is the NVIDIA A100 80GB best for?: With 80 GB of VRAM, the NVIDIA A100 80GB is a server-class GPU that runs 70B-class dense models and large MoE models natively, with plenty of room for long context.
What LLMs can the NVIDIA A100 80GB run locally?: The NVIDIA A100 80GB can run 65 of the 97 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Qwen 3.8 27B at BF16, Ornith 1.5 35B-A3B (MoE) at Q8_0, Qwen 3.5 122B-A10B (MoE) at Q3_K_M.
Can the NVIDIA A100 80GB run Gemma 4 31B?: Yes. The NVIDIA A100 80GB runs Gemma 4 31B natively in VRAM at BF16 quantization, achieving approximately 21.1 tokens per second.
Can the NVIDIA A100 80GB run Qwen 3.6 27B?: Yes. The NVIDIA A100 80GB runs Qwen 3.6 27B natively in VRAM at BF16 quantization, achieving approximately 24.3 tokens per second.
Can the NVIDIA A100 80GB run Qwen3 8B?: Yes. The NVIDIA A100 80GB runs Qwen3 8B natively in VRAM at BF16 quantization, achieving approximately 77 tokens per second.
Can I rent the NVIDIA A100 80GB instead of buying it?: Yes: RunPod and similar cloud GPU providers let you rent NVIDIA A100 80GB instances by the hour, with no long-term contract. This is often cheaper than buying if you only need it occasionally, and lets you try the GPU before committing to a purchase.