Local LLM GPU and VRAM requirements, without the one-number shortcut.

The honest answer is not a single lookup number. For GPU inference, memory planning begins with the model weights, then adds the key-value cache, runtime allocations, and a safety margin. The GPU must have enough usable capacity for the resulting plan—or the runtime must offload part of the work to slower system memory.

A transparent weight floor is parameters × bits per parameter ÷ 8. Seven billion parameters at exactly four bits are about 3.26 GiB. Thirteen billion are about 6.05 GiB. Thirty-two billion are about 14.90 GiB. Seventy billion are about 32.60 GiB. These figures are arithmetic floors, not promises that a packaged checkpoint or live server will fit.

Real checkpoints carry metadata and quantization overhead. Live inference needs workspace memory. Long context creates a KV cache whose size depends on layers, KV heads, head dimension, cache precision, tokens, and concurrent sequences. A runtime can also reserve memory before it serves the first request. That is why two models with the same parameter count can cross different hardware boundaries.

If you already know the model architecture, use the LLM GPU memory calculator. If you only know your available memory, use What LLM can I run? to find a starting tier, then refine it with the exact checkpoint.

1. The checkpoint, not just the parameter label. A model name is a routing clue. The actual files reveal the resident artifact, precision, tensors, and packaging overhead. For a dense model, parameter math is a useful cross-check. For a mixture-of-experts model, do not confuse the smaller active-parameter count with the larger resident weight set. Read the release artifact before choosing hardware.

2. Quantization. Moving from sixteen-bit to eight-bit or four-bit weights lowers the arithmetic floor. Real formats are not always exactly four or eight bits per parameter because scales, metadata, and mixed-precision tensors add overhead. Quantization can also change output quality and runtime compatibility. Treat precision as both a memory choice and a model-quality experiment.

3. KV-cache architecture and precision. Autoregressive inference stores keys and values for previous tokens. The cache grows with the number of layers, KV heads, head dimension, cached tokens, bytes per element, and concurrent sequences. Grouped-query attention can use fewer KV heads than multi-head attention, producing dramatically different cache totals for models with similar parameter counts. The KV-cache calculator exposes every term.

4. Runtime headroom and usable capacity. Model weights and KV cache do not own the whole GPU. Kernels, temporary buffers, graph captures, the display stack, and allocator fragmentation consume memory. Evidence Lab examples often reserve 10% of advertised VRAM and then apply 20% headroom to the weights-plus-cache target. Those are explicit planning defaults; replace them with measured values from the runtime and machine you will operate.

5. Concurrency and serving policy. A personal chat with one active sequence is not the same deployment as an API serving many users. A runtime may preallocate a cache pool or reserve enough blocks for a target concurrency. The cache can therefore become a service-capacity decision, not merely a context-length choice. Enter the simultaneous sequences you intend to support, not the traffic volume you hope never arrives.

System RAM is the offload and loading boundary. When a checkpoint does not fit entirely in VRAM, runtimes such as llama.cpp can keep some layers or tensors in system memory. That can make an otherwise impossible model run, but the data path is now constrained by system-memory bandwidth and transfers between CPU and GPU. Reserve enough RAM for the model data, the operating system, the runtime, and the rest of the workload. A machine that barely holds the checkpoint can spend its time paging rather than generating.

The CPU matters differently for full-GPU and hybrid inference. When the whole model and active cache stay on the GPU, the processor is usually not the main token-generation engine. It still handles tokenization, request routing, sampling, networking, storage, and application code. With partial offload, CPU core performance, memory channels, and RAM bandwidth become much more visible. Plan around the actual runtime split instead of buying a processor from a generic “AI PC” label.

Storage changes startup and iteration, not the weight floor. Model repositories can contain several quantizations, shards, and revisions. Fast NVMe storage reduces download extraction, verification, and model-loading friction, but it does not turn disk capacity into GPU memory. Keep enough free space for the original artifacts, converted formats, temporary files, and a known-good rollback checkpoint. If a model reload is part of the production path, measure cold-start time rather than assuming the storage label is sufficient.

Multi-GPU capacity is not perfectly additive. Tensor or pipeline parallelism can split a model, and simple layer offload can distribute weights, but each runtime has communication costs and may replicate data. Consumer GPUs without a fast peer-to-peer path can spend meaningful time moving tensors through the host. Use the calculator's GPU-count result as a capacity floor, then confirm that the chosen runtime supports the topology and precision. Two 24 GB cards are not automatically one seamless 48 GB accelerator.

Apple unified memory needs a separate capacity rule. Apple Silicon exposes one memory pool to the CPU and GPU, so the relevant number is usable unified memory after macOS and applications—not a dedicated VRAM label. The same weight-plus-cache logic still applies, but the available fraction and runtime implementation differ from a discrete NVIDIA card. Enter the memory you can actually allocate and leave operating-system margin rather than treating the full installed RAM as model capacity.

Power and cooling are operating constraints. A GPU that can load a model may throttle, destabilize, or become uneconomic under sustained serving. Include the card, CPU, memory, fans, power supply losses, and idle periods when comparing a local machine with rented inference. For a personal workstation, noise and heat may matter as much as peak throughput. For a service, measure completed requests per watt and per dollar instead of quoting the GPU's board specification.

ResearchAudio calculates dense weight floors directly, labels illustrative inputs, and routes model-specific claims to the exact release evidence used by each guide. This page is a map of the decision system; the linked calculators are where you replace defaults with your own architecture and workload.

For model-level inspection, begin with the configuration and weight artifacts published by the model author. The existing guides link directly to the relevant Hugging Face Transformers cache documentation, vLLM engine configuration, OpenAI gpt-oss release, Qwen3 release, and Google Gemma 4 model card. Runtime behavior changes; recheck the current documentation before purchasing hardware.

The guide deliberately avoids universal tokens-per-second promises. Speed depends on the exact checkpoint, quantization, prompt length, decode length, batch, runtime version, kernels, operating system, power limit, and hardware. A capacity worksheet should narrow the candidates, not replace a benchmark on the target stack.