NVIDIA RTX 5050
The NVIDIA RTX 5050 has 8 GB VRAM and 320 GB/s memory bandwidth. It can run 24 of our 70 tracked models natively in VRAM at 8k context.
The NVIDIA RTX 5050 is the most affordable Blackwell desktop GPU at $249 MSRP, with 2,560 CUDA cores and 8GB GDDR6 on a 128-bit bus (320 GB/s). Unlike the rest of the 50-series it still uses GDDR6. Suitable only for very small LLMs (3B–4B params) and entry-level 1080p gaming.
The NVIDIA RTX 5050 is a consumer-grade NVIDIA GPU based on the Blackwell architecture. Released in 2025. It features 8 GB of GDDR6 at 320 GB/s memory bandwidth. Full llama.cpp and Ollama support out of the box. CUDA 12.x recommended; driver ≥ 525 required.
For local LLM inference, this GPU runs 24 of the 70 models we track natively in VRAM at 8K context. The largest model it handles in VRAM is Qwen3 14B (65.7 t/s at Q2_K). Its VRAM limits it to smaller models (1-8B parameters), which makes it suitable for prototyping and edge inference. On Llama 3.1 8B Instruct, it achieves approximately 80 tokens per second at NVFP4 quantization. An additional 23 models fit with CPU offload — slower but usable.
NVIDIA's CUDA ecosystem provides broad out-of-the-box support across llama.cpp, Ollama, vLLM, and TensorRT-LLM. Among consumer GPUs, it sits above NVIDIA RTX 4060 and NVIDIA RTX 4060 Ti 16GB in performance, but below NVIDIA RTX 3060 12GB.
| Vendor | NVIDIA |
| Architecture | Blackwell |
| VRAM | 8 GB |
| Memory type | GDDR6 |
| Memory bandwidth | 320 GB/s |
| Compute backend | CUDA |
| Tier | Consumer |
| Released | 2025 |
| Models (native) | 24 / 70 |
| Models (offload) | 23 / 70 |
Models this GPU runs natively in VRAM (24)
- Qwen3 14B14.8B · MMLU-Pro —Q2_K · ~65.7 t/s
- Qwen 2.5 14B Instruct14.7B · MMLU-Pro 51.2Q2_K · ~66.2 t/s
- Phi-4 14B Instruct14B · MMLU-Pro 56.1Q2_K · ~69.5 t/s
- Mistral Nemo 12B Instruct12.2B · MMLU-Pro 35.6Q3_K_M · ~61 t/s
- Gemma 3 12B Instruct12.2B · MMLU-Pro —Q3_K_M · ~61 t/s
- Gemma 2 9B Instruct9.2B · MMLU-Pro 32.0Q3_K_M · ~80.9 t/s
- Llama 3.1 8B Instruct8B · MMLU-Pro 37.5NVFP4 · ~80 t/s
- DeepSeek R1 Distill Llama 8B8B · MMLU-Pro 41.0NVFP4 · ~80 t/s
- Qwen3 8B8B · MMLU-Pro —NVFP4 · ~80 t/s
- Qwen 2.5 7B Instruct7.6B · MMLU-Pro 36.5NVFP4 · ~84.2 t/s
- Mistral 7B Instruct v0.37.25B · MMLU-Pro 30.0NVFP4 · ~88.3 t/s
- Gemma 3 4B Instruct4B · MMLU-Pro —NVFP4 · ~160 t/s
- Gemma 4 E4B4B · MMLU-Pro —NVFP4 · ~160 t/s
- Phi-3.5 Mini Instruct3.8B · MMLU-Pro 35.6NVFP4 · ~168.4 t/s
- Llama 3.2 3B Instruct3.2B · MMLU-Pro 24.0NVFP4 · ~200 t/s
- Qwen 2.5 3B Instruct3.1B · MMLU-Pro 32.4BF16 · ~51.6 t/s
- Gemma 2 2B Instruct2.6B · MMLU-Pro 17.8BF16 · ~61.5 t/s
- Gemma 4 E2B2B · MMLU-Pro —BF16 · ~80 t/s
- SmolLM2 1.7B Instruct1.7B · MMLU-Pro 19.0BF16 · ~94.1 t/s
- Qwen 2.5 1.5B Instruct1.5B · MMLU-Pro 16.8FP32 · ~53.3 t/s
- Llama 3.2 1B Instruct1.24B · MMLU-Pro 12.5FP32 · ~64.5 t/s
- Gemma 3 1B Instruct1B · MMLU-Pro —FP32 · ~80 t/s
- Qwen 2.5 0.5B Instruct0.5B · MMLU-Pro 10.0FP32 · ~160 t/s
- SmolLM2 360M Instruct0.36B · MMLU-Pro 8.0FP32 · ~222.2 t/s
Models that fit with CPU offload (23)
These use system RAM for layers that don't fit in VRAM — expect much slower inference.
- Qwen 2.5 72B Instruct72B · MMLU-Pro 58.1Q2_K · ~3.4 t/s
- Llama 3.3 70B Instruct70B · MMLU-Pro 68.9Q2_K · ~3.5 t/s
- DeepSeek R1 Distill Llama 70B70B · MMLU-Pro 70.0Q2_K · ~3.5 t/s
- Llama 3.1 70B Instruct70B · MMLU-Pro 66.4Q2_K · ~3.5 t/s
- Mixtral 8x7B Instruct v0.146.7B · MMLU-Pro 29.7NVFP4 · ~12.4 t/s
- Command-R 35B35B · MMLU-Pro 33.0NVFP4 · ~4.6 t/s
- Qwen 3.5 35B-A3B (MoE)35B · MMLU-Pro —NVFP4 · ~53.3 t/s
- Qwen 3.6 35B35B · MMLU-Pro —NVFP4 · ~4.6 t/s
- Yi 1.5 34B Chat34.4B · MMLU-Pro 37.0NVFP4 · ~4.7 t/s
- Qwen3 32B32.8B · MMLU-Pro —NVFP4 · ~4.9 t/s
- Qwen 2.5 32B Instruct32.5B · MMLU-Pro 55.1NVFP4 · ~4.9 t/s
- Qwen 2.5 Coder 32B Instruct32.5B · MMLU-Pro 50.4NVFP4 · ~4.9 t/s
- DeepSeek R1 Distill Qwen 32B32.5B · MMLU-Pro 65.0NVFP4 · ~4.9 t/s
- Nemotron 3 Nano 30B32B · MMLU-Pro —NVFP4 · ~53.3 t/s
- Gemma 4 31B31B · MMLU-Pro —NVFP4 · ~5.2 t/s
- Qwen3 30B-A3B (MoE)30B · MMLU-Pro —NVFP4 · ~53.3 t/s
- Gemma 2 27B Instruct27.2B · MMLU-Pro 38.0NVFP4 · ~5.9 t/s
- Gemma 3 27B Instruct27B · MMLU-Pro —NVFP4 · ~5.9 t/s
- Qwen 3.6 27B27B · MMLU-Pro —NVFP4 · ~5.9 t/s
- Gemma 4 26B (MoE)26B · MMLU-Pro —NVFP4 · ~42.1 t/s
- Mistral Small 3.1 24B Instruct24B · MMLU-Pro —NVFP4 · ~6.7 t/s
- Mistral Small 22B22.2B · MMLU-Pro 49.2NVFP4 · ~7.2 t/s
- GPT-OSS 20B21B · MMLU-Pro —NVFP4 · ~40 t/s
Too large for this GPU (23)
- Mixtral 8x22B Instruct v0.1
- Llama 3.1 405B Instruct
- DeepSeek V3 671B
- DeepSeek R1 671B
- Llama 4 Scout 109B
- Llama 4 Maverick 400B
- Qwen3 235B-A22B (MoE)
- MiniMax M1 456B
- GPT-OSS 120B
- GLM-4.5 355B
- GLM-4.5 Air 106B
- GLM-4.6 355B
- GLM-4.6V 106B
- GLM-4.7 358B
- Qwen 3.5 122B-A10B (MoE)
- MiniMax M2.5 229B
- GLM-5 744B
- MiniMax M2.7 229B
- Nemotron 3 Super 120B
- Kimi K2.6
- GLM-5.1 754B
- DeepSeek V4 Pro 1.6T
- DeepSeek V4 Flash 284B
Frequently asked questions
- How much VRAM does the NVIDIA RTX 5050 have?
- The NVIDIA RTX 5050 has 8 GB of GDDR6 with 320 GB/s memory bandwidth.
- What LLMs can the NVIDIA RTX 5050 run locally?
- The NVIDIA RTX 5050 can run 24 of the 70 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Llama 3.1 8B Instruct at NVFP4, Llama 3.2 3B Instruct at NVFP4, Llama 3.2 1B Instruct at FP32.
- Can the NVIDIA RTX 5050 run Llama 3.3 70B Instruct?
- The NVIDIA RTX 5050 can run Llama 3.3 70B Instruct with CPU offload at Q2_K quantization, but inference will be slower than native VRAM execution.
- Can the NVIDIA RTX 5050 run Qwen 3.6 27B?
- The NVIDIA RTX 5050 can run Qwen 3.6 27B with CPU offload at NVFP4 quantization, but inference will be slower than native VRAM execution.
- Can the NVIDIA RTX 5050 run Llama 3.1 8B Instruct?
- Yes. The NVIDIA RTX 5050 runs Llama 3.1 8B Instruct natively in VRAM at NVFP4 quantization, achieving approximately 80 tokens per second.