Local LLM hardware / RTX 4090

What LLM can a 24 GB RTX 4090 run?

Q: What LLM can an RTX 4090 run?

With 24 GB of VRAM, 90 percent usable memory, and 20 percent runtime headroom, an RTX 4090 clears this site's generic weight floor for a 32B tier at INT4, 14B at INT8, or 8B at FP16. Named models still need architecture-aware context and runtime checks.

Q: Can an RTX 4090 run Qwen3 32B?

Not under this page's fully GPU-resident 32K planning profile. Qwen3 32B at INT4 reaches 27.93 GiB after an eight GiB BF16 KV cache and 20 percent headroom, above the 21.6 GiB usable-memory budget used for one 24 GB card.

Q: Can an RTX 4090 run gpt-oss-20b?

The official 12.8 GiB checkpoint reaches a 14.29 GiB planning target for one 4K sequence and fits this page's RTX 4090 budget. A conservative full 128K cache profile reaches 22.56 GiB and exceeds the 21.6 GiB default usable-memory allowance.

Q: Can an RTX 4090 run a 70B model entirely in VRAM?

No at the ordinary four-bit weight floor used here. Seventy billion parameters require about 32.6 GiB for raw four-bit weights before KV cache, runtime workspace, metadata, and fragmentation.

A 24 GB card clears a generic 32B INT4 weight floor. The useful answer comes after adding the exact checkpoint, architecture, context window, runtime reserve, and the memory the driver leaves available.

Open the 24 GB model finder →

Fast capacity answer

Start with the weight floor. Do not stop there.

INT4
32B listed tier · 38.7B arithmetic ceiling

INT8
14B listed tier · 19.3B arithmetic ceiling

FP16 / BF16
8B listed tier · 9.7B arithmetic ceiling

Default reserve
24 × 0.90 ÷ 1.20 = 18 GiB for weights

Named-model checks

Models near the RTX 4090 boundary.

These are fully GPU-resident planning estimates, not speed claims. Open any card to change context, cache precision, headroom, or GPU count.

Qwen3 14B · INT4 · 32K 14.27 GiB Fits the 21.6 GiB planning budget → Qwen3 30B-A3B · INT4 · 32K 20.64 GiB Fits with a narrow default margin → Qwen3 32B · INT4 · 32K 27.93 GiB Exceeds one-card planning capacity → gpt-oss-20b · MXFP4 · 4K 14.29 GiB Fits the default one-card budget → gpt-oss-20b · MXFP4 · 128K 22.56 GiB Exceeds the 21.6 GiB default budget → 70B · raw INT4 weights 32.60 GiB More than 24 GB before cache →

Read the boundary

Why “32B fits” can still be false.

The generic tier finder excludes KV cache and artifact-specific overhead. The named-model profiles include architecture-aware cache and an explicit runtime reserve.

Question	Fast answer	Deployment answer
Largest INT4 tier	32B weight floor	Inspect the exact checkpoint and context
Qwen3 30B-A3B	20.64 GiB at 32K	Fits narrowly; benchmark the runtime
Qwen3 32B	15.27 GiB raw INT4 weights	27.93 GiB with 32K cache and headroom
gpt-oss-20b	14.29 GiB at 4K	22.56 GiB in the conservative 128K profile

Capacity is only gate one

Context consumes VRAM.
The KV cache grows with tokens, layers, KV heads, head dimension, precision, and concurrency.

MoE still loads experts.
Active parameters reduce compute per token; they do not erase resident checkpoint memory.

Fit is not throughput.
Bandwidth, kernels, offload, batching, and power limits decide delivered speed.

Primary references and next checks

Keep the hardware claim auditable.

NVIDIA specifies 24 GB of GDDR6X memory for the RTX 4090. The model scenarios use the same architecture and checkpoint inputs documented in the ResearchAudio Qwen3 and gpt-oss worksheets.

Official NVIDIA RTX 4090 specifications → Open the Qwen3 GPU worksheet → Open the gpt-oss hardware worksheet → Compare the 32 GB RTX 5090 boundary →

What LLM can an RTX 4090 run?

At the default reserve, the generic one-card floor reaches a 32B INT4, 14B INT8, or 8B FP16 tier. Named models still need exact checkpoint, cache, context, and runtime inputs.

Can an RTX 4090 run Qwen3 32B?

Not under this page's 32K fully GPU-resident profile: the 27.93 GiB planning target exceeds the 21.6 GiB usable-memory budget.

Can an RTX 4090 run gpt-oss-20b?

The 4K profile reaches 14.29 GiB and fits. The conservative 128K profile reaches 22.56 GiB and exceeds the default one-card allowance.

Can an RTX 4090 run a 70B model entirely in VRAM?

No at the ordinary four-bit floor used here. Raw INT4 weights alone require about 32.6 GiB before cache and runtime allocations.