Qwen3.8 2.4T-A95B
Qwen3.8 2.4T-A95B needs roughly 1669.3 GB VRAM at Q4_K_M quantization (5480.3 GB at FP16). 0 GPUs we track can run it fully in VRAM at 8k context.
0 GPUs run this natively · 0 with CPU offload
- Q2_K total
- 1044.7 GB
- at 8k context
- KV cache, full context
- 24.7 GB
- 23 of 92 layers cache
- Inputs
- Text
- Qwen3.8-Max, released August 2026
- Active experts
- 10 routed + 1 shared
- of 512 total
- Active parameters
- 95B
- 3.9% of the 2.4T total
- Context ceiling
- ~1.01M tokens
- 262,144 native, extended
Qwen3.8 2.4T-A95B is a Mixture of Experts (MoE) model with 2446.18B total parameters but only 95B active per token developed by Alibaba. Released 8 August 2026 under the custom Qwen3.8-Max license, the first time Alibaba has released open weights for a Qwen-Max-class model. It's a mixture of experts: 2.4 trillion parameters total, but a router activates only 95 billion per token (512 experts, 10 routed plus 1 shared). The 92 layers are hybrid, the same idea Qwen used in Qwen 3.6 27B and Qwen 3.5 35B-A3B scaled up: 23 repeats of three Gated DeltaNet layers followed by one Gated Attention layer, so only 23 of the 92 layers keep a growing KV cache. Context is 262,144 tokens natively and up to 1,010,000 with extended settings. Thinking mode can't be turned off, and the checkpoint is text-only.
To run Qwen3.8 2.4T-A95B locally: There's no getting around the scale: even the most aggressive standard quantization this site tracks, Q2_K, needs about 932GB of weights alone, and about 1,045GB once KV cache and overhead are added at a modest 8k context. That's more than a full 8-GPU node of NVIDIA H100 80GB cards can pool (640GB raw) or even 8x H200 141GB cards (1,128GB raw, ~1,015GB after multi-GPU pooling overhead). It takes at least 7x NVIDIA B200 180GB cards (1,260GB raw) to clear it, and even then decode lands around 38.5 tok/s at Q2_K. Weight-native FP8 (Qwen's own quantized release) and NVFP4 community builds roughly halve or quarter that footprint respectively, which is the realistic path to running this outside a datacenter, but this is a model built for Qwen Cloud's API or a serious multi-GPU cluster, not a home lab. As a MoE model, inference speed depends on active parameters (95B) rather than total size.
Qwen's own comparison table reports GPQA Diamond 92.6, SWE-bench Pro 67.7, and Terminal-Bench 2.1 86.6, placing it ahead of GPT-5.6 Sol (max) and Qwen's own prior-generation Qwen3.7-Max on several of the same agentic coding benchmarks, though behind Claude Opus 4.8 and Claude Fable 5 on some general-capability suites like HLE. These are Alibaba's own launch figures, measured with its preferred harness (Claude Code for the coding-agent rows); independent third-party reproduction was still thin in the first days after release.
A 2.4 trillion-parameter model that reads 95 billion per token
This model stacks the same two efficiency tricks Qwen 3.5 35B-A3B introduced, just at a much larger scale. It's a mixture of experts: 512 experts exist in the weights, but a router picks only 10 plus one shared expert for each token, so 95 of the 2.4 trillion parameters actually do the work. Underneath that routing, the 92 layers aren't all the same kind of layer either: the same hybrid attention idea from Qwen 3.6 27B, scaled up and combined with MoE routing on every layer, not just the attention ones.
23 of 92 layers keep a KV cache that grows with the full context. The other 69 hold a fixed-size recurrent state, so their memory cost does not move as the context window grows.
Only the 23 Gated Attention layers keep a per-token KV cache; the 69 Gated DeltaNet layers carry a constant-size recurrent state instead, and every one of the 92 layers routes through its own bank of 512 experts regardless of which attention type it uses. The MoE routing is what makes 2.4 trillion parameters tractable to run at all: decode speed tracks the 95B active count, not the full total. Hybrid attention shrinks the KV cache on top of that. Two independent savings, stacked.
Why the hybrid attention barely moves the needle at this scale
On Qwen 3.6 27B, cutting caching layers from 64 to 16 cut the KV cache to a quarter of an all-attention design and that mattered, because the weights and the KV cache were close enough in size for the difference to change which GPU tier the model needed. At 2.4 trillion parameters, that's no longer true.
KV cache only, at FP16. Weights and activation overhead sit on top of these figures.
Filling the full 262,144-token window costs 24.7 GB of KV cache with the real hybrid stack, versus 98.8 GB if every layer cached. That's a genuine 4x reduction, but stacked on Q2_K's 932.0 GB of weights, the full-context totals only move from 1071.5 GB to 1154.5 GB, an 8% difference. The lesson runs the opposite way from the 27B page: hybrid attention still saves real gigabytes, but at this parameter count the weights were always going to dominate the bill, not the context window.
VRAM at each quantization
Calculated at 8k context. Because only part of this model's stack keeps a growing KV cache, longer sessions cost far less extra VRAM than a full-attention model of the same size.
| Quant | Weights | KV cache | Total |
|---|---|---|---|
| FP32 | 9784.7 GB | 0.77 GB | 10959.8 GB |
| BF16 | 4892.4 GB | 0.77 GB | 5480.3 GB |
| FP16 | 4892.4 GB | 0.77 GB | 5480.3 GB |
| Q8_0 | 2600.3 GB | 0.77 GB | 2913.2 GB |
| Q6_K | 2008.3 GB | 0.77 GB | 2250.2 GB |
| Q5_K_M | 1741.7 GB | 0.77 GB | 1951.5 GB |
| Q4_K_M | 1489.7 GB | 0.77 GB | 1669.3 GB |
| Q3_K_M | 1176.6 GB | 0.77 GB | 1318.7 GB |
| Q2_Krec | 932.0 GB | 0.77 GB | 1044.7 GB |
| NVFP4cuda | 1223.1 GB | 0.77 GB | 1370.7 GB |
Shown at 8k context with FP16 KV cache. NVFP4 needs a CUDA GPU to run. Toggle TurboQuant in the calculator to view compressed KV cache numbers.
Quantization × context: where it fits
The table above holds context fixed at 8k. Add context as a second axis and the picture barely changes: every cell here needs a multi-GPU datacenter node regardless of which row or column is picked, since even the smallest quantization this ladder estimates is still hundreds of gigabytes.
| Quant | 8k ctx | 32k ctx | 128k ctx | 256k ctx |
|---|---|---|---|---|
| Q8_0 | 2913.2 GB80 GB+ card | 2915.8 GB80 GB+ card | 2926.2 GB80 GB+ card | 2940.0 GB80 GB+ card |
| Q6_K | 2250.2 GB80 GB+ card | 2252.8 GB80 GB+ card | 2263.1 GB80 GB+ card | 2277.0 GB80 GB+ card |
| Q5_K_M | 1951.5 GB80 GB+ card | 1954.1 GB80 GB+ card | 1964.5 GB80 GB+ card | 1978.3 GB80 GB+ card |
| Q4_K_M | 1669.3 GB80 GB+ card | 1672.0 GB80 GB+ card | 1682.3 GB80 GB+ card | 1696.2 GB80 GB+ card |
| Q3_K_M | 1318.7 GB80 GB+ card | 1321.3 GB80 GB+ card | 1331.6 GB80 GB+ card | 1345.5 GB80 GB+ card |
Benchmarks
Bigger, but not proportionally smarter
Plotted by total parameter count against every other model tracked here that reports GPQA Diamond, this model sits at the far right of the chart by a wide margin, roughly 2.4x the size of the next-largest MoE release. Its score doesn't lead by nearly that much: Qwen's own 92.6 is only a few points past Kimi K2.6's 90.5, a model roughly a quarter of the total size and a third of the active-parameter count.
GPUs that run Qwen3.8 2.4T-A95B natively (0)
No single GPU in our list fits this model at Q4 with 8k context. Browse all GPUs or try multi-GPU setups.
Notes
The first Qwen-Max-class model Alibaba has released with open weights, built on the Qwen3.5 architectural foundation. Thinking mode is mandatory and cannot be disabled: every response opens with reasoning inside <think> tags. Its depth is tunable via reasoning_effort (xhigh by default, medium, or low), and preserve_thinking carries reasoning context across turns. The Qwen3.8-Max license permits free commercial use up to $20M/month revenue or 100M MAU (with attribution required above that), and requires a separate license only for a Model-as-a-Service or AI-assistant business over $50M/year.
How to run Qwen3.8 2.4T-A95B locally
Q2_K needs 1044.7 GB — needs multiple datacenter-class GPUs (80 GB+ each).
llama.cpp or vLLM/SGLang (thinking mode is mandatory, can't be disabled)
./llama-server \
-hf unsloth/Qwen3.8-2.4T-A95B-GGUF:UD-Q1_0 \
-c 262144 -ngl 99 \
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0llama.cpp
# UD-Q1_0 (~397GB) is the smallest build unsloth has shipped so far.
# No Q2_K-Q6_K middle ground exists yet for this model, unlike smaller Qwen
# releases. UD-IQ1_S (~508GB) is the next step up.
./llama-server \
-hf unsloth/Qwen3.8-2.4T-A95B-GGUF:UD-IQ1_S \
-c 262144 -ngl 99 \
--temp 1.0 --top-p 0.95 --top-k 20 --min-p 0.0 --presence-penalty 0.0LM Studio: Search for 'Qwen3.8 2.4T-A95B GGUF'. At launch, unsloth's repo only has BF16, Q8_0, UD-IQ1_S, and UD-Q1_0. None of the usual Q3_K-Q6_K middle tiers this site's calculator estimates for the standard ladder exist as real downloadable files yet, so treat those middle rows on this page as size estimates, not confirmed builds.
Why this quantization? There isn't a realistic single-GPU quant here. UD-Q1_0's real ~397GB is the only build that fits inside a single machine this site tracks at all: a 512GB Mac Studio M3 Ultra, with the rest of its unified memory as headroom for KV cache and context. Every other config on this site needs the weights split across multiple datacenter GPUs: even 8x NVIDIA H200 141GB (about 1,015GB pooled) falls short of the generic Q2_K ladder estimate's ~1,045GB total at a modest 8k context, so it takes at least 7x NVIDIA B200 180GB to clear it.
Who is Qwen3.8 2.4T-A95B for?
Teams with a multi-GPU datacenter node (or a 512GB Mac Studio willing to run the sub-1.5-bit UD-Q1_0 build) who specifically need open weights rather than Qwen Cloud's hosted API, for auditability, fine-tuning, or air-gapped deployment. Everyone else is better served by the API. This is not a model to download on a whim.
Best for
- Self-hosted, auditable deployment of a Qwen-Max-class model behind an existing multi-GPU inference cluster
- Fine-tuning or distillation research that needs the open weights rather than API access
- Long-context agentic coding and research tasks, using the full 262,144-to-1,010,000-token window
Not ideal for
- Anything short of a multi-GPU datacenter node or a 512GB unified-memory Mac, since there is no practical single consumer or prosumer GPU path
- Users who just want to try the model: Qwen Cloud's API serves the same weights without a multi-terabyte download
- Disabling thinking mode or supplying image/audio input, since this checkpoint is text-only and reasoning-only
So, should you actually run this?
For almost everyone, no. Not the open weights, anyway. Qwen Cloud's hosted Qwen3.8-Max API serves the identical model with vision input, a 1M-token context by default, and none of the multi-terabyte download or multi-GPU cluster this repository demands. The open weights make sense for a specific, narrower group: teams that need to self-host a Qwen-Max-class model for auditability, fine-tuning, or an air-gapped environment, and already have (or can rent) a multi-GPU datacenter node or a 512 GB Mac Studio willing to run the sub-1.5-bit UD-Q1_0 build. Everyone else gets the same weights, none of the infrastructure, through the API.
Frequently asked questions
- What are the VRAM requirements for Qwen3.8 2.4T-A95B?
- Qwen3.8 2.4T-A95B requires approximately 1669.4 GB of VRAM at Q4_K_M quantization, 2913.2 GB at Q8, and 5480.3 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.
- How many parameters does Qwen3.8 2.4T-A95B have?
- Qwen3.8 2.4T-A95B has 2446.18 billion total parameters, but only 95 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
- Can Qwen3.8 2.4T-A95B run on a 16 GB GPU?
- No. At Q4_K_M, Qwen3.8 2.4T-A95B needs 1669.4 GB of VRAM — more than 16 GB. You will need a multi-GPU server.
- Can Qwen3.8 2.4T-A95B run on a 24 GB GPU?
- No. Even at Q4_K_M, Qwen3.8 2.4T-A95B needs 1669.4 GB. Consider a multi-GPU server with 1670 GB+ of combined VRAM.
- What is the smallest quantization for Qwen3.8 2.4T-A95B that fits in 24 GB of VRAM?
- Qwen3.8 2.4T-A95B cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 1044.7 GB at Q2_K.
- What GPU do I need to run Qwen3.8 2.4T-A95B locally?
- You need a multi-GPU server. At Q4_K_M, Qwen3.8 2.4T-A95B needs 1669.4 GB VRAM, more than any single consumer GPU. That's roughly 21x 80 GB datacenter GPUs (H100, A100, or similar) pooled together.
- Can Qwen3.8 2.4T-A95B run on a single GPU?
- No. Even the most aggressive standard quantization this site tracks, Q2_K, needs about 932 GB of weights alone. No single consumer, prosumer, or even single datacenter GPU on the market holds that much. The smallest single-machine option this site tracks is a 512 GB Mac Studio M3 Ultra running the community UD-Q1_0 GGUF build (about 397 GB); everything else needs several GPUs pooled together.
- Why is only 95B of the 2.4T parameters active?
- It's a mixture-of-experts model: 512 experts exist in the weights, but a router selects only 10 routed experts plus 1 shared expert for each token, so decode only has to read about 95 billion parameters off memory per step instead of the full 2.4 trillion. That's what makes a model this large tractable to serve at all, though the full weights still have to be resident in VRAM (or RAM) even though most of them sit unused on any given token.
- How does this compare to Kimi K2.6 or DeepSeek V4 Pro?
- It's the largest of the three by total parameters (2.4T versus Kimi K2.6's 1T and DeepSeek V4 Pro's 1.6T) and the largest by active parameters too (95B versus 32B and 49B). On GPQA Diamond, the score gap is much narrower than the size gap: 92.6 here against Kimi K2.6's 90.5, both self-reported by their respective vendors. All three need a multi-GPU datacenter setup; none is a realistic home-lab download.