Mixture of Experts LLMs

Mixture of Experts (MoE) models have a large total parameter count but only activate a fraction per token. This means you need enough VRAM to load all the weights, but inference speed is determined by the active parameter count — often dramatically faster than a dense model of equivalent quality. The trade-off: MoE models can be hard to fit on a single GPU.