How much VRAM does the AMD Instinct MI300X have?

The AMD Instinct MI300X has 192 GB of HBM3 with 5300 GB/s memory bandwidth.

What is the AMD Instinct MI300X best for?

With 192 GB of VRAM, the AMD Instinct MI300X is a server-class GPU designed for running the largest open-weight models (70B–405B) at high quantization with ample context.

What LLMs can the AMD Instinct MI300X run locally?

The AMD Instinct MI300X can run 64 of the 70 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Llama 3.3 70B Instruct at BF16, Llama 3.1 8B Instruct at FP32, Llama 3.2 3B Instruct at FP32.

Can the AMD Instinct MI300X run Llama 3.3 70B Instruct?

Yes. The AMD Instinct MI300X runs Llama 3.3 70B Instruct natively in VRAM at BF16 quantization, achieving approximately 37.9 tokens per second.

Can the AMD Instinct MI300X run Qwen 3.6 27B?

Yes. The AMD Instinct MI300X runs Qwen 3.6 27B natively in VRAM at FP32 quantization, achieving approximately 49.1 tokens per second.

Can the AMD Instinct MI300X run Llama 3.1 8B Instruct?

Yes. The AMD Instinct MI300X runs Llama 3.1 8B Instruct natively in VRAM at FP32 quantization, achieving approximately 165.6 tokens per second.

AMD Instinct MI300X

The AMD Instinct MI300X has 192 GB VRAM and 5300 GB/s memory bandwidth. It can run 64 of our 70 tracked models natively in VRAM at 8k context.

With 192 GB HBM3, the AMD Instinct MI300X is a datacenter-tier GPU that can run 64 models natively. It handles 70B-class models at Q4 quantization.

The AMD Instinct MI300X is AMD's flagship AI data center accelerator, featuring 192GB of HBM3 memory at 5,300 GB/s — the highest memory bandwidth of any GPU in our database. Launched in late 2023, it was AMD's first serious competitor to NVIDIA's H100 at the hyperscaler level. The 192GB capacity allows running full 70B and even 405B models without quantization, and its extraordinary bandwidth makes it exceptionally fast at large-batch inference. vLLM has strong first-class support for the MI300X.

AMD Instinct MI300X: December 2023 CDNA 3 with 192GB HBM3 at 5300 GB/s — highest bandwidth in database.

Runs 405B at Q4 native. 70B at FP16 native. ~60-100 t/s for 7B, ~25-40 t/s for 70B.

ROCm Linux only. vLLM has first-class MI300X support. Exceptional capacity for large models. Cloud-only for most.

Vendor	AMD
Architecture	CDNA 3
VRAM	192 GB
Memory type	HBM3
Memory bandwidth	5300 GB/s
Compute backend	ROCM
Tier	Datacenter
Released	2023
Models (native)	64 / 70
Models (offload)	0 / 70

Software: ROCm Linux only. Exceptional capacity for large models; vLLM has strong MI300X support for batched inference.

Popular models for this GPU

MiniMax M1 456B Llama 3.1 405B Instruct Llama 4 Maverick 400B GLM-4.7 358B GLM-4.5 355B

Models this GPU runs natively in VRAM (64)

Too large for this GPU (6)

Compare AMD Instinct MI300X with other GPUs

Frequently asked questions

How much VRAM does the AMD Instinct MI300X have?: The AMD Instinct MI300X has 192 GB of HBM3 with 5300 GB/s memory bandwidth.
What is the AMD Instinct MI300X best for?: With 192 GB of VRAM, the AMD Instinct MI300X is a server-class GPU designed for running the largest open-weight models (70B–405B) at high quantization with ample context.
What LLMs can the AMD Instinct MI300X run locally?: The AMD Instinct MI300X can run 64 of the 70 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Llama 3.3 70B Instruct at BF16, Llama 3.1 8B Instruct at FP32, Llama 3.2 3B Instruct at FP32.
Can the AMD Instinct MI300X run Llama 3.3 70B Instruct?: Yes. The AMD Instinct MI300X runs Llama 3.3 70B Instruct natively in VRAM at BF16 quantization, achieving approximately 37.9 tokens per second.
Can the AMD Instinct MI300X run Qwen 3.6 27B?: Yes. The AMD Instinct MI300X runs Qwen 3.6 27B natively in VRAM at FP32 quantization, achieving approximately 49.1 tokens per second.
Can the AMD Instinct MI300X run Llama 3.1 8B Instruct?: Yes. The AMD Instinct MI300X runs Llama 3.1 8B Instruct natively in VRAM at FP32 quantization, achieving approximately 165.6 tokens per second.