How to Run LLMs Locally on Mac (M1–M4 Guide)

The Unified Memory Advantage

Apple Silicon's biggest advantage for LLM inference is not raw compute but unified memory architecture. On a discrete GPU system, only the dedicated VRAM is available for model weights. An RTX 4090 with 24 GB of VRAM can only hold 24 GB of model data, regardless of how much system RAM the machine has. On Apple Silicon, the GPU and CPU share the same memory pool. An M4 Max with 128 GB of unified memory can hold a 128 GB model entirely in GPU-accessible memory. This is why Apple Silicon has become the platform of choice for running large models locally on consumer hardware: an M4 Max MacBook Pro can run Llama 3.3 70B at Q8_0, which would require a dual-GPU workstation setup on NVIDIA. The trade-off is memory bandwidth. The M4 Max delivers about 546 GB/s of bandwidth, which is excellent for a laptop but roughly half what an RTX 4090 provides (1,008 GB/s). This means token generation is slower on Apple Silicon, typically 10-30 tokens per second for models that would hit 30-60 tokens per second on equivalent NVIDIA hardware. For most interactive use cases, this is fine. For batch processing large document sets, the speed difference adds up.

What Each M-Series Chip Can Run

The M-series lineup covers a wide capability range. The base M1 and M2 with 8 GB can run small models only: 3-4B class at Q4_K_M, or a 7B model at Q3_K_M with very short context. This is functional for quick queries and lightweight coding assistance but frustrating for anything demanding. With 16 GB (M1 Pro, M2 base 16GB, or M3), you enter the same tier as 16 GB NVIDIA cards: comfortable with 7-8B at Q8_0 and 14B at Q4_K_M. The M3 Pro with 36 GB and M4 Pro with 48 GB are where it gets interesting. At 48 GB, you can run Llama 3.3 70B at Q4_K_M with moderate context, or Gemma 3 27B at Q8_0 with long context. These configurations represent a sweet spot for professional use. The M4 Max at 128 GB is the enthusiast tier: it runs 70B models at Q8_0, or even two different models loaded simultaneously. The M4 Ultra with 192 GB or more opens the door to 100B+ parameter models at practical quantization levels, approaching what was previously only possible on datacenter hardware.

Setting Up Ollama on Mac

Ollama is the fastest path from zero to running an LLM on Mac. It handles model downloading, quantization selection, and Metal GPU acceleration automatically. Installation takes under a minute, and the first model pull takes only as long as your internet connection needs to download the weights. Ollama uses the Metal backend by default on Apple Silicon, which means GPU acceleration works without any additional configuration. After installation, running your first model is a single command. Ollama will download the model on first run and cache it locally. Subsequent runs load the model from cache in seconds. The Ollama model library includes pre-quantized versions of most popular models, so you do not need to deal with raw GGUF files unless you want to. For Mac users specifically, there are two things to note. First, Ollama runs as a background service, so you can interact with it from any terminal window or through its API. Second, Activity Monitor will show high memory usage when a model is loaded, which is expected and not a memory leak. The memory is released when the model is unloaded (which Ollama does automatically after a period of inactivity).

# Install Ollama on Mac
brew install ollama

# Or download from https://ollama.ai and drag to Applications

# Start the Ollama service
ollama serve

# In another terminal, run your first model
ollama run llama3.1:8b

# List downloaded models
ollama list

# Remove a model you no longer need
ollama rm llama3.1:8b

MLX: Apple's Native Framework

MLX is Apple's machine learning framework designed specifically for Apple Silicon. Unlike llama.cpp (which Ollama uses under the hood), MLX is written to exploit Apple Silicon's unified memory and hardware features natively. The practical difference is that MLX can be slightly faster than llama.cpp on Mac for certain model architectures, particularly for prompt processing (prefill). For token generation speed, the difference is typically small (5-15%). MLX is worth using if you want Python-level control over inference, are doing research or development work, or want to use the MLX ecosystem of fine-tuning tools. For day-to-day model running, Ollama provides a more convenient interface. The MLX community maintains a large library of pre-converted models on Hugging Face under the mlx-community organization. These models are in MLX's native format and load directly without conversion. To get started with MLX, install the mlx-lm package via pip and use the provided generation script.

# Install MLX for LLM inference
pip install mlx-lm

# Run a model with MLX
mlx_lm.generate \
  --model mlx-community/Llama-3.1-8B-Instruct-4bit \
  --prompt "Explain quantum computing briefly"

# Start an interactive chat session
mlx_lm.chat --model mlx-community/Llama-3.1-8B-Instruct-4bit

# For serving via OpenAI-compatible API
mlx_lm.server --model mlx-community/Llama-3.1-8B-Instruct-4bit

Performance Expectations by Chip

Token generation speed on Apple Silicon is primarily limited by memory bandwidth, not compute. This means the speed depends more on your chip tier than on the model size (within VRAM limits). The base M1 delivers roughly 68 GB/s of bandwidth, translating to about 5-15 tokens per second for a 7B model at Q4_K_M. The M2 Pro bumps this to about 200 GB/s (15-25 tok/s). The M3 Max at 400 GB/s delivers 25-40 tok/s for 7B models, but drops to 10-20 tok/s for 27B models since more bytes need to move per token. The M4 Max at 546 GB/s is the current sweet spot: roughly 30-50 tok/s for 7B and 15-25 tok/s for 27B models. Prompt processing (the initial ingestion of your input) is much faster than generation on all chips because it benefits from parallelism. A 2000-token prompt might process in 1-2 seconds even on models that generate at 15 tok/s. One important setting: in Ollama, the num_gpu parameter controls how many model layers are placed on the GPU. On Apple Silicon, this should always be set to the maximum (all layers on GPU) unless you are deliberately experimenting with CPU-only inference.

Managing Memory on Mac

Since Apple Silicon shares memory between the system and GPU, you need to be mindful of total memory pressure. macOS itself typically consumes 4-8 GB depending on your configuration and open applications. A web browser can use 2-10 GB depending on open tabs. This means an M4 Pro with 48 GB might only have 32-38 GB available for model inference in typical use. Check available memory before loading a large model. If macOS starts swapping to disk (visible in Activity Monitor under Memory Pressure), inference speed will degrade dramatically because the SSD, while fast, is orders of magnitude slower than memory. Close unnecessary applications before loading models that approach your memory limit. For power users, consider creating a dedicated macOS user account with minimal login items for LLM work. This ensures maximum memory availability. Also note that Ollama keeps models loaded in memory for a configurable timeout (default: 5 minutes of inactivity). If you are switching between multiple large models, you can reduce this timeout or manually unload models to free memory between switches.

# Check available memory on Mac
vm_stat | head -10

# Or use the memory_pressure command
memory_pressure -l warn

# Set Ollama to unload models faster
export OLLAMA_KEEP_ALIVE=60s  # Unload after 60 seconds

# Manually unload a model
curl http://localhost:11434/api/generate -d '{"model": "llama3.1:8b", "keep_alive": 0}'

Recommended Configurations by Budget

For budget-conscious users entering local LLMs, the M2 MacBook Air with 16 GB (often available refurbished) runs 7-8B models comfortably and handles 14B at Q4_K_M. It is the cheapest viable Mac for meaningful local inference. For the best balance of capability and cost, the M4 Pro Mac Mini with 48 GB is hard to beat: it runs Llama 3.3 70B at Q4_K_M, fits on a desk, and costs less than an equivalent NVIDIA setup. For maximum local capability, the M4 Max MacBook Pro with 128 GB runs 70B models at near-lossless quantization and can even handle some 100B+ class models at Q4_K_M. The Mac Studio with M4 Ultra and 192 GB is the final tier, capable of running virtually any single open-weight model at reasonable quantization. When buying a Mac specifically for LLM work, always maximize memory over other specs. A base M4 Pro with 48 GB is far more useful than a maxed-out M4 Pro with 24 GB, even though the latter has more GPU cores. Memory is the constraint that determines which models you can run. GPU core count only affects speed, not capability.