11 November 2026 · VRAMfarm team

How much VRAM do I need? A table for LLMs, Flux, Wan 2.2 video and training

A 7–8B chat model in 4-bit fits on a 12 GB card, a 32B model needs 24 GB, and Llama 3.3 70B needs about 43 GB just for its weights, so 48 GB is the realistic minimum. Image models sit between 10 GB (SDXL) and 24 GB (Qwen-Image in fp8), and Wan 2.2 video needs 22–48 GB. The same estimates drive the VRAM picker on the home page, and the formula they come from works for any model you want to check yourself.

The formula: weights, KV cache and overhead

Video memory holds three things while a model runs:

VRAM ≈ parameters × bytes per parameter + KV cache (or activations) + 1–2 GB overhead

The first part is the weights. One billion parameters at 2 bytes each is 2 GB, so an 8B model in fp16 needs about 16 GB before it has processed a single word. The bytes per parameter depend on precision:

The second part depends on what the model does. For a language model it is the KV cache: the model keeps keys and values for every token in the conversation, and that grows with context length. For image and video models it is the activations: intermediate tensors that grow with resolution, number of frames and batch size.

The third part is the CUDA context, the framework’s buffers and memory fragmentation. Budget 1–2 GB, more with vLLM, which reserves 90% of the card up front by default.

Quantisation is how most people fit bigger models onto smaller cards. fp8 halves the weights with almost no visible difference, and it is the default for Flux, Qwen-Image and Wan in ComfyUI. 4-bit halves them again. For chat and summarising, a 4-bit 32B model (about 20 GB) usually gives better answers than an 8B model in fp16, which takes almost as much memory. For code and maths in small models the loss is easier to notice. Below 4 bits, quality drops quickly.

One hardware detail: fp8 is computed natively from the Ada generation onwards (RTX 40 and 50 series, L40S, H100, B200). Ampere cards (RTX 3090, RTX A6000, A100) can load fp8 weights, but they compute in fp16, so you save memory without gaining speed.

Language models: from 8B to 235B

The numbers are approximate: weights plus working buffers at short context. Long context comes on top, see the KV cache section below. The “4-bit” column matches what our picker shows, because that is how most people run these models on one card.

A few notes on reading it:

• 24 GB runs a 32B model, with a short context. A 4-bit 32B model leaves 2–3 GB for the KV cache on a 24 GB card, enough for roughly 8–10k tokens. The 32 GB RTX 5090 gives the same model room for something like 30–40k, depending on the quantisation and the runtime’s own buffers.
• 70B on one card means 48 GB or more. On 48 GB, a 4-bit 70B runs with 8–12k context, about twice that with an 8-bit cache. For long documents, several users at once or vLLM serving, take an 80 GB A100 or H100.
• The 141 GB H200 looks right for a 136 GB model, but it is too tight. After the CUDA context almost nothing is left for the cache. Take a B200 or two 80 GB cards.
• MoE models need all their experts in memory. Qwen3 235B-A22B activates 22B parameters per token, so it generates text about as fast as a 22–30B dense model, yet it needs the memory of a 235B one.

Speed is a separate question. Generating each token reads the whole model from memory, so a rough ceiling for tokens per second is memory bandwidth divided by model size. An RTX 4090 (1,008 GB/s) with a 20 GB model tops out around 50 tokens per second, and real numbers land a bit lower. That is why an H100 with 3,350 GB/s feels so much faster with the same model.

Image and video models

Image and video models in ComfyUI are usually run in fp8, and ComfyUI moves the text encoder and VAE in and out of VRAM as needed. The “minimum” column means “runs at 1024 × 1024 (or 720p video) without errors”; the recommended card gives you headroom for LoRAs, ControlNet and higher resolutions. Treat the figures as estimates for a typical workflow: resolution, frame count and ComfyUI’s offloading settings can move them by several gigabytes either way.

Video is the category where “fits” and “fits comfortably” differ most. Each extra second of video adds frames, and every frame adds activations. Wan 2.2 14B in fp8 needs about 30 GB at 720p for a 5-second clip: a 32 GB RTX 5090 handles it, but longer clips or higher resolution push you to 48 GB. That is why our ready-made Wan 2.2 14B app asks for a 48 GB card, and the 5B app for 24 GB.

GGUF quantisations (Q4, Q5) of Flux and Qwen-Image exist and squeeze them onto 8–12 GB cards, at the price of slower steps and some detail. If you only need a few images, that is fine; for real work the extra memory pays for itself in time. The GPU guide for Flux, SDXL and ComfyUI goes card by card.

Speech and training

Speech models are small. Training needs far more memory than running the same model, because gradients and optimiser states sit next to the weights.

The full fine-tune line needs explaining. With the standard AdamW optimiser in mixed precision, each parameter costs about 16 bytes: bf16 weights (2), gradients (2), an fp32 master copy (4) and two optimiser states (8). For 8B parameters that is about 128 GB before activations, which means two or more 80 GB cards, an H200 or a B200. An 8-bit optimiser and gradient checkpointing bring it down to roughly 70 GB, depending on sequence length and the trainer, which fits on a single A100 or H100. QLoRA avoids almost all of it: the base model stays frozen in 4-bit and only small adapter matrices are trained.

Why a model that fits still crashes on long context

This is the most common surprise: the model loads, answers the first questions, then fails with “CUDA out of memory” when you paste a long document. The weights did fit. The KV cache did not.

The cache per token is 2 × layers × KV heads × head size × bytes. In fp16 that gives:

So a 4-bit 70B on a 48 GB card is fine at 8k tokens and runs out somewhere around 10–12k. At its advertised 128k context the cache alone is as big as the weights. Serving several users multiplies the cache by the number of parallel conversations.

How to fix it without a bigger card:

• Cap the context. In vLLM set --max-model-len 16384. vLLM refuses to start if the context you ask for can’t fit, which is better than crashing later. In Ollama, the context length (num_ctx) is what decides memory, so set it on purpose rather than relying on the default, which has changed between versions.
• Quantise the cache. --kv-cache-dtype fp8 in vLLM, or -ctk q8_0 -ctv q8_0 in llama.cpp (with flash attention on), halves the cache with little effect on answers.
• Limit parallel requests if you serve an API: fewer simultaneous conversations, less cache.

If you need the long context every day, the cheaper fix is usually the next card up: an 80 GB card for 70B, or a 32 GB RTX 5090 instead of 24 GB for 32B.

Picking the card from the number

Take the number from the tables, add 10–20% for headroom, and pick the smallest card above it. The cards come in fixed steps: 10–12 GB, 16 GB, 24 GB, 32 GB, 48 GB, 80 GB, 96 GB, 141 GB and 192 GB. Within one step the difference is speed, not what you can run: an RTX 3090 and an RTX 4090 both hold 24 GB. The 4090 makes images about 1.6–2.5 times faster, but generates LLM text only a little faster, because their memory bandwidth is similar (936 vs 1,008 GB/s). Compare their hourly prices, from €0.30 and €0.59, on the pricing page.

If you are unsure, start one step bigger than you think. Billing is per minute, so a card that is too large costs you a few cents for the test, while one that is too small costs you an evening of out-of-memory errors.