LLM VRAM Calculator

Pick a model, a quantization level, and a context length — see the VRAM required to run it, and which tracked GPUs have enough.

Model

Quantization

Context length

Weights = params × bytes-per-parameter for the selected quantization.KV cache is always computed at FP16, independent of weight quantization — the default in vLLM/llama.cpp and most serving stacks — and assumes batch size 1 (concurrent requests multiply the KV-cache term roughly linearly; not modeled here). A flat +15% is added on top for CUDA context/activation overhead — a documented rule of thumb, not a measured figure. See each model's own page for MoE and other model-specific caveats (e.g. DeepSeek-V3's compressed KV cache, which this calculator does not model and will overestimate).