What are the VRAM requirements for Ornith 1.5 397B (MoE)?

Ornith 1.5 397B (MoE) requires approximately 270.9 GB of VRAM at Q4_K_M quantization, 472.7 GB at Q8, and 889.1 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.

How many parameters does Ornith 1.5 397B (MoE) have?

Ornith 1.5 397B (MoE) has 396.8 billion total parameters, but only 14 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.

Can Ornith 1.5 397B (MoE) run on a 16 GB GPU?

No. At Q4_K_M, Ornith 1.5 397B (MoE) needs 270.9 GB of VRAM, more than 16 GB. You will need a multi-GPU server.

Can Ornith 1.5 397B (MoE) run on a 24 GB GPU?

No. Even at Q4_K_M, Ornith 1.5 397B (MoE) needs 270.9 GB. Consider a multi-GPU server with 271 GB+ of combined VRAM.

What is the smallest quantization for Ornith 1.5 397B (MoE) that fits in 24 GB of VRAM?

Ornith 1.5 397B (MoE) cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 169.6 GB at Q2_K.

What GPU do I need to run Ornith 1.5 397B (MoE) locally?

You need a multi-GPU server. At Q4_K_M, Ornith 1.5 397B (MoE) needs 270.9 GB VRAM, more than any single consumer GPU. That's roughly 4x 80 GB datacenter GPUs (H100, A100, or similar) pooled together.

Ornith 1.5 397B (MoE)

Name: Ornith 1.5 397B (MoE)
Author: Ornith AI

Ornith 1.5 397B (MoE) needs roughly 270.9 GB VRAM at Q4_K_M quantization (889.1 GB at FP16). 6 GPUs we track can run it fully in VRAM at 8k context.

6 GPUs run this natively · 0 with CPU offload

Ornith AI396.8B params14B active (MoE)256k contextMITCommercial use ok

Ornith 1.5 397B (MoE) is a Mixture of Experts (MoE) model with 396.8B total parameters but only 14B active per token developed by Ornith AI. Released 19 August 2026 under the MIT license, the flagship of the three Ornith-1.5 sizes launched the same day. A 512-expert MoE (10 routed plus 1 shared active per token) built on the same hybrid attention idea as the smaller two sizes: 60 layers, fifteen repeats of three Gated DeltaNet layers followed by one Gated Attention layer, so only 15 of 60 layers keep a growing KV cache. config.json is byte-identical in shape to Qwen 3.5's MoE line, scaled up; safetensors metadata puts the exact total at 396.8B parameters, the figure behind Ornith AI's own "397B" rounding. Context is 262,144 tokens natively and roughly 1M with YaRN scaling.

To run Ornith 1.5 397B (MoE) locally: At bf16 the full weights run roughly 800GB, so Ornith AI's own reference serving setup uses 8-way tensor parallelism across 141GB-class datacenter GPUs; this is not a model anyone runs at full precision on a workstation. Aggressive quantization is what makes local use realistic at all: GGUF, FP8, and NVFP4 builds are available from ornith-ai on Hugging Face, and this site's own quantized-fit numbers below show what's actually achievable on consumer and prosumer hardware rather than the vendor's own multi-GPU datacenter recipe. As a MoE model, inference speed depends on active parameters (14B) rather than total size.

Ornith AI's own launch post puts it on par with Claude Opus 4.8 on Terminal-Bench 2.1 (86.1 vs 85.0) and DeepSWE (56.0 vs 59.0), and ahead of similarly-sized open-weight peers GLM-5.2 (753B) and DeepSeek-V4-Flash-0731 (284B) on the same suite; GPQA Diamond lands at 92.8. These are Ornith AI's own comparison figures from its launch blog and model card, not independently reproduced here, and reflect the same self-improvement training loop (the model proposes its own tasks, scaffolds, and rollouts for reinforcement learning) used across all three Ornith-1.5 sizes.

VRAM at each quantization

Numbers here are computed at 8k context. This model's hybrid attention stack means KV cache grows much more slowly than context length, unlike a conventional full-attention model.

Quant	Weights	KV cache	Total
FP32	1587.2 GB	0.25 GB	1778.0 GB
BF16	793.6 GB	0.25 GB	889.1 GB
FP16	793.6 GB	0.25 GB	889.1 GB
Q8_0	421.8 GB	0.25 GB	472.7 GB
Q6_K	325.8 GB	0.25 GB	365.1 GB
Q5_K_M	282.5 GB	0.25 GB	316.7 GB
Q4_K_M	241.7 GB	0.25 GB	270.9 GB
Q3_K_M	190.9 GB	0.25 GB	214.1 GB
Q2_Krec	151.2 GB	0.25 GB	169.6 GB
NVFP4cuda	198.4 GB	0.25 GB	222.5 GB

KV cache is calculated at 8k context (FP16). Note that NVFP4 only runs on CUDA GPUs. Turn on TurboQuant in the calculator above for lower KV cache estimates.

Benchmarks

92.8

86.0

65.1

86.1

GPUs that run Ornith 1.5 397B (MoE) natively (6)

NVIDIA B300 288GBNVFP4 · 220.5 t/s
NVIDIA B200 180GBQ2_K · 288.4 t/s
AMD Instinct MI300XQ2_K · 191.1 t/s
Apple M3 Ultra (512GB)Q8_0 · 13.1 t/s
Apple M3 Ultra (256GB)Q3_K_M · 28.9 t/s

Show 1 more

Apple M2 Ultra (192GB)Q2_K · 35.5 t/s

Notes

Flagship of the Ornith-1.5 family, launched 19 August 2026 alongside the 9B and 35B-A3B models, trained with the same self-improvement loop described on the 9B entry. Ornith AI's own launch post puts it on par with Claude Opus 4.8 on Terminal-Bench 2.1 (86.1 vs 85.0) and DeepSWE (56.0 vs 59.0), and ahead of similarly-sized open-weight peers GLM-5.2 (753B) and DeepSeek-V4-Flash-0731 (284B) on the same suite; those are Ornith AI's own comparison numbers, not independently reproduced here. 512 experts, 10 routed + 1 shared active per token. At bf16 the full weights run roughly 800GB, so full-precision serving needs multi-GPU tensor parallelism (Ornith AI's own reference recipe uses 8x 141GB GPUs); this site's quantized-fit calculations are what make single-workstation use realistic. MIT licensed. GGUF, FP8, and NVFP4 builds are available from ornith-ai on Hugging Face.

Hugging Face ↗Ollama ↗Released 2026-08-19

How to run Ornith 1.5 397B (MoE) locally

816244880160320

Q2_K needs 169.6 GB: needs multiple datacenter-class GPUs (80 GB+ each).

vLLM or SGLang, multi-GPU tensor parallelism (llama.cpp only below the official quant floor)

vllm serve ornith-ai/Ornith-1.5-397B \
  --served-model-name Ornith-1.5-397B \
  --tensor-parallel-size 8 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.90 \
  --enable-auto-tool-choice --tool-call-parser qwen3_xml \
  --reasoning-parser qwen3

llama.cpp

# ornith-ai's own GGUF repo starts at Q4_K_M (~242GB); this site's Q2_K/Q3_K
# ladder rows below that are size estimates, not files anyone has published yet.
./llama-server \
  -hf ornith-ai/Ornith-1.5-397B-GGUF:Q4_K_M \
  -c 262144 -ngl 99 \
  --temp 0.6 --top-p 0.95 --top-k 20

LM Studio: LM Studio can load the Q4_K_M GGUF if you have roughly 250GB+ of pooled VRAM or unified memory, but this is squarely multi-GPU-node or high-end Mac Studio territory, not a single consumer card. Most people are better served by ornith-ai's FP8 or NVFP4 builds through vLLM/SGLang, which is also what Ornith AI's own reference recipe uses.

Why this quantization? There's no realistic single-consumer-GPU quant here. At bf16 the full weights run roughly 800GB; ornith-ai's own GGUF repo's smallest published build is Q4_K_M at roughly 242GB, still well past a single 80GB or 96GB GPU. This site's generic Q2_K ladder estimate (roughly 152GB of weights alone) is a size projection, not a confirmed file, the same caveat that applies to other frontier-scale MoE releases this site tracks without a full quant ladder yet.

Who is Ornith 1.5 397B (MoE) for?

Teams with a multi-GPU node (Ornith AI's own reference setup is 8x 141GB-class GPUs) or a very large unified-memory machine who specifically want open weights rather than a hosted API, for auditability, fine-tuning, or air-gapped deployment.

Best for

Self-hosted, auditable deployment of a Claude-Opus-class open-weight coding model behind an existing multi-GPU inference cluster
Fine-tuning or distillation research that needs the open weights rather than API access
Long-context agentic coding and research tasks using the full 262,144-to-1,000,000-token window

Not ideal for

Anything short of a multi-GPU datacenter node or a very large unified-memory Mac, since there is no practical single consumer or prosumer GPU path even at Q4_K_M
Anyone who just wants to try the model: the smaller 9B or 35B-A3B Ornith-1.5 sizes are the realistic entry points on consumer hardware
Anyone who needs benchmark claims independently verified before adopting a model, this released within days of when this page was written and the comparisons against Claude Opus 4.8, GLM-5.2, and DeepSeek-V4-Flash are Ornith AI's own

Explore

Browse all guides →Browse all models →Browse all GPUs →

Frequently asked questions

What are the VRAM requirements for Ornith 1.5 397B (MoE)?: Ornith 1.5 397B (MoE) requires approximately 270.9 GB of VRAM at Q4_K_M quantization, 472.7 GB at Q8, and 889.1 GB at FP16. These numbers assume 8k context window; its hybrid attention stack caches far fewer than all layers, so VRAM grows much slower than linearly with context.
How many parameters does Ornith 1.5 397B (MoE) have?: Ornith 1.5 397B (MoE) has 396.8 billion total parameters, but only 14 billion are active per token thanks to its Mixture of Experts (MoE) architecture. This makes inference significantly faster than the total parameter count suggests.
Can Ornith 1.5 397B (MoE) run on a 16 GB GPU?: No. At Q4_K_M, Ornith 1.5 397B (MoE) needs 270.9 GB of VRAM, more than 16 GB. You will need a multi-GPU server.
Can Ornith 1.5 397B (MoE) run on a 24 GB GPU?: No. Even at Q4_K_M, Ornith 1.5 397B (MoE) needs 270.9 GB. Consider a multi-GPU server with 271 GB+ of combined VRAM.
What is the smallest quantization for Ornith 1.5 397B (MoE) that fits in 24 GB of VRAM?: Ornith 1.5 397B (MoE) cannot fit in 24 GB of VRAM at any standard quantization level. The minimum needed is 169.6 GB at Q2_K.
What GPU do I need to run Ornith 1.5 397B (MoE) locally?: You need a multi-GPU server. At Q4_K_M, Ornith 1.5 397B (MoE) needs 270.9 GB VRAM, more than any single consumer GPU. That's roughly 4x 80 GB datacenter GPUs (H100, A100, or similar) pooled together.