How much VRAM does the AMD Instinct MI300X have?

The AMD Instinct MI300X has 192 GB of HBM3 with 5300 GB/s memory bandwidth.

What is the AMD Instinct MI300X best for?

With 192 GB of VRAM, the AMD Instinct MI300X is a server-class GPU designed for running the largest open-weight models (70B–405B) at high quantization with ample context.

What LLMs can the AMD Instinct MI300X run locally?

The AMD Instinct MI300X can run 80 of the 97 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Ornith 1.5 397B (MoE) at Q2_K, Qwen3.8-Flash-Next at Q6_K, GLM-5.3-Flash 320B at Q3_K_M.

Can the AMD Instinct MI300X run Gemma 4 31B?

Yes. The AMD Instinct MI300X runs Gemma 4 31B natively in VRAM at BF16 quantization, achieving approximately 54.8 tokens per second.

Can the AMD Instinct MI300X run Qwen 3.6 27B?

Yes. The AMD Instinct MI300X runs Qwen 3.6 27B natively in VRAM at BF16 quantization, achieving approximately 63.2 tokens per second.

Can the AMD Instinct MI300X run Qwen3 8B?

Yes. The AMD Instinct MI300X runs Qwen3 8B natively in VRAM at BF16 quantization, achieving approximately 200.2 tokens per second.

AMD Instinct MI300X

The AMD Instinct MI300X has 192 GB VRAM and 5300 GB/s memory bandwidth. It can run 80 of our 97 tracked models natively in VRAM at 8k context.

With 192 GB HBM3, the AMD Instinct MI300X is a datacenter-tier GPU that can run 80 models natively. Runs 405B at Q4 native. 70B at FP16 native. ~60-100 t/s for 7B, ~25-40 t/s for 70B.

The AMD Instinct MI300X is AMD's flagship AI data center accelerator, featuring 192GB of HBM3 memory at 5,300 GB/s, the highest memory bandwidth of any GPU in our database. Launched in late 2023, it was AMD's first serious competitor to NVIDIA's H100 at the hyperscaler level. The 192GB capacity allows running full 70B and even 405B models without quantization, and its extraordinary bandwidth makes it exceptionally fast at large-batch inference. vLLM has strong first-class support for the MI300X.

AMD Instinct MI300X: December 2023 CDNA 3 with 192GB HBM3 at 5300 GB/s, highest bandwidth in database.

Runs 405B at Q4 native. 70B at FP16 native. ~60-100 t/s for 7B, ~25-40 t/s for 70B.

ROCm Linux only. vLLM has first-class MI300X support. Exceptional capacity for large models. Cloud-only for most.

Vendor	AMD
Architecture	CDNA 3
VRAM	192 GB
Memory type	HBM3
Memory bandwidth	5300 GB/s
Compute backend	ROCM
Tier	Datacenter
Released	2023
Models (native)	80 / 97
Models (offload)	2 / 97

Software: ROCm Linux only. Exceptional capacity for large models; vLLM has strong MI300X support for batched inference.

Popular models for this GPU

Llama 3.1 405B Instruct Llama 4 Maverick 400B Ornith 1.5 397B (MoE)GLM-4.7 358B GLM-4.5 355B

Models this GPU runs natively in VRAM (80)

Show 75 more

Models that fit with CPU offload (2)

These use system RAM for layers that don't fit in VRAM, so expect much slower inference.

Too large for this GPU (15)

Compare AMD Instinct MI300X with other GPUs

Explore

Browse all guides →Browse all models →Browse all GPUs →

Frequently asked questions

How much VRAM does the AMD Instinct MI300X have?: The AMD Instinct MI300X has 192 GB of HBM3 with 5300 GB/s memory bandwidth.
What is the AMD Instinct MI300X best for?: With 192 GB of VRAM, the AMD Instinct MI300X is a server-class GPU designed for running the largest open-weight models (70B–405B) at high quantization with ample context.
What LLMs can the AMD Instinct MI300X run locally?: The AMD Instinct MI300X can run 80 of the 97 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Ornith 1.5 397B (MoE) at Q2_K, Qwen3.8-Flash-Next at Q6_K, GLM-5.3-Flash 320B at Q3_K_M.
Can the AMD Instinct MI300X run Gemma 4 31B?: Yes. The AMD Instinct MI300X runs Gemma 4 31B natively in VRAM at BF16 quantization, achieving approximately 54.8 tokens per second.
Can the AMD Instinct MI300X run Qwen 3.6 27B?: Yes. The AMD Instinct MI300X runs Qwen 3.6 27B natively in VRAM at BF16 quantization, achieving approximately 63.2 tokens per second.
Can the AMD Instinct MI300X run Qwen3 8B?: Yes. The AMD Instinct MI300X runs Qwen3 8B natively in VRAM at BF16 quantization, achieving approximately 200.2 tokens per second.