VRAM calculator for LLMs
Enter the model size, weight precision, context length and task. The calculator shows how much video memory you need and the cheapest GPUs that fit, with their price per hour.
You need about
- Weights
- —
- Context (KV cache)
- —
- Training (gradients, optimiser, activations)
- —
- Buffers
- —
Cheapest fits
An estimate: real use depends on the model architecture (attention heads, GQA), the engine (vLLM reserves most free memory for the cache by default), batch size and sequence length. Image and video models: see the cards on the GPU pages.
Frequently asked questions
How do I calculate VRAM for an LLM?
Multiply the parameters in billions by the bytes per weight: 2 in FP16, 1 in 8-bit, about 0.6 in 4-bit. Add the KV cache for your context and number of parallel requests, and 10–15% for buffers.
What is the KV cache?
Memory where the model keeps the context it has already processed. It grows with context length and with the number of requests served at once, so long chats and many users need more memory than the weights alone.
How much memory does fine-tuning need?
QLoRA adds roughly 30–50% to the 4-bit weights, LoRA with a 16-bit base about 20–40% to the FP16 weights. Full fine-tuning with AdamW needs about 16 bytes per parameter, roughly 8 times the FP16 weights, plus activations.
Does it work for Stable Diffusion and Flux?
No, the formula is for language models. For image and video models see the table on the Stable Diffusion and video pages: SDXL needs about 12 GB, Flux 16 GB, Qwen-Image 24 GB.