LLM infrastructure / 7B versus 13B

The model size sets the floor. The cache decides the fit.

Q: How much VRAM does a 7B LLM need?

Seven billion parameters require about 3.26 GiB at exactly four bits, 6.52 GiB at eight bits, or 13.04 GiB at sixteen bits for raw weights. The illustrative 32K GQA serving profiles on this page reach 8.71, 12.62, and 20.45 GiB after an explicit four GiB KV cache and 20 percent runtime headroom.

Q: How much VRAM does a 13B LLM need?

Thirteen billion parameters require about 6.05 GiB at exactly four bits, 12.11 GiB at eight bits, or 24.21 GiB at sixteen bits for raw weights. The illustrative 32K GQA serving profiles on this page reach 13.26, 20.53, and 35.06 GiB after an explicit five GiB KV cache and 20 percent runtime headroom.

Q: Can a 7B model run on an 8 GB GPU?

A four-bit 7B weight floor is about 3.26 GiB, so some short-context, lean-runtime, or partially offloaded workloads can fit. The full 32K GQA example on this page plans for 8.71 GiB and therefore does not fit when only 90 percent of an eight GiB GPU is usable.

Q: Why can two 13B models need different amounts of VRAM?

The same parameter count fixes only the raw weight floor. Layer count, KV-head count, head dimension, cached context, concurrency, quantization format, and serving runtime can change the total. In this page's 13B examples, replacing eight GQA KV heads with forty MHA KV heads raises the 32K cache from five GiB to twenty-five GiB.

Compare 7B and 13B weight memory at INT4, INT8, and FP16, then include architecture, 32K cached context, runtime headroom, and actually usable GPU memory.

Open the editable 13B model →

Raw weight floors

The arithmetic is exact. The deployment is not.

7B
INT4 3.26 · INT8 6.52 · FP16 13.04 GiB

13B
INT4 6.05 · INT8 12.11 · FP16 24.21 GiB

KV cache
2 × layers × KV heads × head dim × tokens × bytes

Planning target
(weights + explicit KV cache) × 1.20 headroom

Illustrative 32K GQA profiles

Hold the cache policy constant. Change the weights.

The 7B examples use 32 layers and the 13B examples use 40 layers. Both use eight KV heads, 128 dimensions per head, one full 32,768-token sequence, 16-bit KV cache, 20% runtime headroom, and 90% usable VRAM. Replace those inputs with the exact model configuration and workload.

7B INT4 · 1 × 12 GiB 8.71 GiB 3.26 weights + 4.00 KV + headroom → 7B INT8 · 1 × 16 GiB 12.62 GiB 6.52 weights + 4.00 KV + headroom → 7B FP16 · 1 × 24 GiB 20.45 GiB 13.04 weights + 4.00 KV + headroom → 13B INT4 · 1 × 16 GiB 13.26 GiB 6.05 weights + 5.00 KV + headroom → 13B INT8 · 1 × 24 GiB 20.53 GiB 12.11 weights + 5.00 KV + headroom → 13B FP16 · 1 × 48 GiB 35.06 GiB 24.21 weights + 5.00 KV + headroom →

The architecture trap

GQA and MHA can change the answer before concurrency starts.

Both comparisons keep model size, INT4 weights, 32K context, 16-bit cache, and 20% headroom constant. Only the number of KV heads changes.

Swipe the matrix horizontally →

Profile	GQA example	MHA example	Difference
7B · 32 layers	8 KV heads · 4 GiB cache 8.71 GiB target	32 KV heads · 16 GiB cache 23.11 GiB target	+14.40 GiB
13B · 40 layers	8 KV heads · 5 GiB cache 13.26 GiB target	40 KV heads · 25 GiB cache 37.26 GiB target	+24.00 GiB

These are architecture examples, not universal 7B or 13B specifications. Read num_hidden_layers, num_key_value_heads, and head dimension from the exact configuration before reserving a GPU.

Do not collapse the decision

Fit is not speed.
Capacity does not prove useful latency or tokens per second.

Context is allocated memory.
A longer or fuller cache can erase the saving from quantized weights.

Concurrency repeats cache.
Every fully allocated sequence multiplies the KV-cache budget.

Build the exact plan

Use the checkpoint and workload, not the label.

Enter the exact parameter count or checkpoint size, then copy the layer count, KV heads, and head dimension from the model configuration. Set cached context and concurrent sequences from the serving target—not the advertised maximum context window.

Open the LLM GPU memory calculator → Check common 8 GB to 141 GB GPU profiles → Start with your VRAM and find the largest model tier → Calculate GQA, MHA, context, and concurrency directly → Compare the 70B deployment boundary → Check the Hugging Face model memory estimator → Review current Transformers KV-cache strategies →

How much VRAM does a 7B LLM need?

The raw floor is 3.26 GiB at INT4, 6.52 GiB at INT8, or 13.04 GiB at FP16. This page's one-sequence 32K GQA examples plan for 8.71, 12.62, and 20.45 GiB after cache and headroom.

How much VRAM does a 13B LLM need?

The raw floor is 6.05 GiB at INT4, 12.11 GiB at INT8, or 24.21 GiB at FP16. This page's one-sequence 32K GQA examples plan for 13.26, 20.53, and 35.06 GiB after cache and headroom.

Can a 7B model run on an 8 GB GPU?

The INT4 weights can fit mathematically, but this page's full 32K GQA profile does not fit in 7.2 GiB of usable capacity. A shorter cache, leaner runtime, lower cache precision, or CPU offload can change the result.

Why can two 13B models need different amounts of VRAM?

Parameter count fixes only the weight floor. In the examples above, changing from eight GQA KV heads to forty MHA KV heads raises the 32K cache from 5 to 25 GiB before concurrency.