How much VRAM does the NVIDIA L40S have?

The NVIDIA L40S has 48 GB of GDDR6 with 864 GB/s memory bandwidth.

What is the NVIDIA L40S best for?

With 48 GB of VRAM, the NVIDIA L40S is ideal for running 70B-class models at Q3-class quantization and large MoE models, a workstation sweet spot for local inference.

What LLMs can the NVIDIA L40S run locally?

The NVIDIA L40S can run 57 of the 97 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Qwen 3.8 27B at Q8_0, Ornith 1.5 35B-A3B (MoE) at Q8_0, Ornith 1.5 9B at BF16.

Can the NVIDIA L40S run Gemma 4 31B?

Yes. The NVIDIA L40S runs Gemma 4 31B natively in VRAM at Q8_0 quantization, achieving approximately 16.4 tokens per second.

Can the NVIDIA L40S run Qwen 3.6 27B?

Yes. The NVIDIA L40S runs Qwen 3.6 27B natively in VRAM at Q8_0 quantization, achieving approximately 19.2 tokens per second.

Can the NVIDIA L40S run Qwen3 8B?

Yes. The NVIDIA L40S runs Qwen3 8B natively in VRAM at BF16 quantization, achieving approximately 32.6 tokens per second.

Can I rent the NVIDIA L40S instead of buying it?

Yes: RunPod and similar cloud GPU providers let you rent NVIDIA L40S instances by the hour, with no long-term contract. This is often cheaper than buying if you only need it occasionally, and lets you try the GPU before committing to a purchase.

NVIDIA L40S

The NVIDIA L40S has 48 GB VRAM and 864 GB/s memory bandwidth. It can run 57 of our 97 tracked models natively in VRAM at 8k context.

With 48 GB GDDR6, the NVIDIA L40S is a datacenter-tier GPU that can run 57 models natively. 70B at Q3_K_M native (Q4_K_M's ~51GB total is just past this card's effective capacity). 405B at Q2 with CPU offload. ~25-40 t/s for 7B, ~15 t/s for 70B at Q3_K_M.

The NVIDIA L40S is a 2023 data center GPU built on the Ada Lovelace architecture with 48GB of GDDR6 VRAM. Unlike HBM-based chips, its 864 GB/s bandwidth is lower, but the higher VRAM capacity over the A100 40GB lets it hold larger models entirely in GPU memory. The L40S also includes hardware-accelerated video encode and decode, making it a popular choice for mixed multimedia and LLM inference workloads in cloud deployments.

NVIDIA L40S: October 2022 Ada Lovelace datacenter GPU with 48GB GDDR6 at 864 GB/s.

70B at Q3_K_M native (Q4_K_M's ~51GB total is just past this card's effective capacity). 405B at Q2 with CPU offload. ~25-40 t/s for 7B, ~15 t/s for 70B at Q3_K_M.

Full CUDA support. Popular for mixed multimedia + LLM workloads. 48GB is the sweet spot capacity.

Vendor	NVIDIA
Architecture	Ada Lovelace
VRAM	48 GB
Memory type	GDDR6
Memory bandwidth	864 GB/s
Compute backend	CUDA
Tier	Datacenter
Released	2023
Models (native)	57 / 97
Models (offload)	8 / 97

Software: Typically cloud-accessed. vLLM and TensorRT-LLM give best batched-inference performance.

Cloud GPU Rental

Don't want to buy a NVIDIA L40S? RunPod is a cloud GPU rental service: rent one by the hour instead, no contract, no upfront hardware cost.

Pay by the hour · no contract · pods start in about a minute.

Rent a NVIDIA L40S on RunPod ↗ (+$5 signup credit)

Affiliate link: CanItRun may earn a commission. Doesn't affect the fit calculation above.

Popular models for this GPU

Qwen 2.5 72B Instruct Llama 3.3 70B Instruct DeepSeek R1 Distill Llama 70B Llama 3.1 70B Instruct Mixtral 8x7B Instruct v0.1

Models this GPU runs natively in VRAM (57)

Show 52 more

Models that fit with CPU offload (8)

These use system RAM for layers that don't fit in VRAM, so expect much slower inference.

Too large for this GPU (32)

Compare NVIDIA L40S with other GPUs

Explore

Browse all guides →Browse all models →Browse all GPUs →

Frequently asked questions

How much VRAM does the NVIDIA L40S have?: The NVIDIA L40S has 48 GB of GDDR6 with 864 GB/s memory bandwidth.
What is the NVIDIA L40S best for?: With 48 GB of VRAM, the NVIDIA L40S is ideal for running 70B-class models at Q3-class quantization and large MoE models, a workstation sweet spot for local inference.
What LLMs can the NVIDIA L40S run locally?: The NVIDIA L40S can run 57 of the 97 open-weight models tracked by CanItRun natively in VRAM at 8k context. Top options include: Qwen 3.8 27B at Q8_0, Ornith 1.5 35B-A3B (MoE) at Q8_0, Ornith 1.5 9B at BF16.
Can the NVIDIA L40S run Gemma 4 31B?: Yes. The NVIDIA L40S runs Gemma 4 31B natively in VRAM at Q8_0 quantization, achieving approximately 16.4 tokens per second.
Can the NVIDIA L40S run Qwen 3.6 27B?: Yes. The NVIDIA L40S runs Qwen 3.6 27B natively in VRAM at Q8_0 quantization, achieving approximately 19.2 tokens per second.
Can the NVIDIA L40S run Qwen3 8B?: Yes. The NVIDIA L40S runs Qwen3 8B natively in VRAM at BF16 quantization, achieving approximately 32.6 tokens per second.
Can I rent the NVIDIA L40S instead of buying it?: Yes: RunPod and similar cloud GPU providers let you rent NVIDIA L40S instances by the hour, with no long-term contract. This is often cheaper than buying if you only need it occasionally, and lets you try the GPU before committing to a purchase.