RTX 4090 for Local LLMs in 2026: Great Card, Broken Price

The RTX 4090 is still a faster single 24GB consumer card in the community 7B figures this page already cited. At ~104 tokens per second on a 7B Q4_K_M model, it outpaces a used RTX 3090’s ~87 tokens per second (community-cited via r/LocalLLaMA and modelfit.io, 2024–2025, not independently verified by LocalRig, and not re-run on 2026-10-06). 104 / 87 is about 20% more throughput on that unreverified pair. Both cards hold 24GB.

The price gap changed. CompSniper, fetched 2026-10-06, shows a used 4090 sold median of $3,000 across 21 sales, typical $2,850–$3,299. The recent table still includes a heatsink-without-GPU, a parts-only card, and a brand-new outlier. CompSniper’s 3090 page the same day shows a sold median of $1,499 across 204 sales, typical $1,329–$1,599, and the recent table includes some 3090 Ti cards. $3,000 / $1,499 is 2.00× for the same 24GB. The June 29, 2026 line on this page — 4090 at $2,000+ and 3090 at $600–$800, called 2.5× — is history. Asking prices for the 4090 were not fetched on October 6.

The performance vs. price problem

Decode is memory-bandwidth-bound. The hardware catalog, verified 2026-06-27 and not re-read from NVIDIA on 2026-10-06, lists 936 GB/s for the 3090 and 1,008 GB/s for the 4090. That is about an 8% bandwidth gap, not the 900 vs 1,456 GB/s (62%) pair this page used to print. Prompt processing is more compute-heavy, so the 4090’s extra FLOPS show up there. For a long generation after a short prompt, the community 7B gap above is about 20% and was not re-measured. That is not 2.00× the sold median.

Compare the constraint geometry:

Aspect	RTX 4090	RTX 3090
VRAM	24 GB GDDR6X	24 GB GDDR6X
Memory bandwidth (catalog, 2026-06-27)	1,008 GB/s	936 GB/s
~7B Q4_K_M tok/s (2024–2025, not re-run)	~104	~87
Sold median, CompSniper, 2026-10-06	$3,000 (21 sales)	$1,499 (204 sales)
Sold median vs the 3090	2.00×	baseline
Community tok/s per sold-median dollar	0.035	0.058

87 / $1,499 is 0.058 tok/s per dollar. 104 / $3,000 is 0.035. That is about 1.7× the tokens per purchase dollar on those unreverified speeds, not the old 2–3×. It is a planning ratio, not a new benchmark. $3,000 / 24 = $125.00/GB. $1,499 / 24 = $62.46/GB.

When a 4090 makes sense

The 4090 is the right buy if any of these apply:

Prompt processing speed matters more than decode speed

Running inference on a large batch of input tokens (e.g., processing a document or codefile all at once, before generation starts) is compute-bound, not memory-bound. The 4090’s higher FLOPS deliver real gain here. If your workload is text embedding, document analysis, or code completion on large files, the 4090 shaves meaningful time. For pure open-ended generation, it does not.

You are running diffusion or video generation

Local LLM inference is only part of what people load GPUs for. Stable Diffusion XL, video generation (Luma, Runway), and image upscaling are not memory-bandwidth-limited in the same way as LLM decode. The 4090’s compute capacity is a genuine advantage for these workloads. If you are splitting time between LLMs and diffusion, the 4090 becomes more interesting. Honestly, though: if you need diffusion, rent. You will not run it daily.

You are serving multiple concurrent users

A single 24GB card handles one inference stream well. When multiple users hit it at once, batching matters more than single-stream speed. The 4090’s FLOPS do help here—it can push more batches through per second. But this is not a local-inference problem; it is a serving problem. If you are building a service, you need vLLM or SGLang, not just a faster GPU. And at that scale, renting datacenter GPUs or smaller cloud instances is cheaper than ownership. See how to run LLMs locally for the batching calculus.

Non-LLM compute is your real bottleneck

Some workflows mix LLM inference with other operations: synthetic data generation (LLM + diffusion), local video summarization (video decode + LLM), retrieval-augmented generation (embedding + LLM). If the LLM is 30% of the workload and diffusion/video is 70%, the 4090’s general compute advantage is real. For pure LLM work, it is not.

The rent alternative: ~$0.20–$0.37 per hour

The price spine’s RTX 4090 rental row is $0.19–$0.35/hr, observed 2026-08-01 from RunPod and Vast.ai. It was not re-fetched on 2026-10-06. The June community band of $0.20–$0.37/hr is the older row. Local electricity for a 4090 uses the catalog 450 W and the EIA rate of $0.15/kWh (dated 2026-06-29): 0.45 × $0.15 = $0.0675/hr.

A $3,000 sold median then breaks even at:

• $3,000 ÷ ($0.19 − $0.0675) = $3,000 ÷ $0.1225 ≈ 24,490 hours
• $3,000 ÷ ($0.35 − $0.0675) = $3,000 ÷ $0.2825 ≈ 10,619 hours

The June worksheet ($2,000 ÷ $0.20 = 10,000 hours, and $2,000 ÷ $0.37 ≈ 5,400 hours) left electricity out and used a lower card price. At 8 hours a day, 10,619 hours is about 3.6 years and 24,490 hours is about 8.4 years. Intermittent use does not get there. The old “~6 months at 8 hours a day” does not survive this price.

For occasional or bursty workloads, check current cloud rental pricing before buying. The math often favors renting.

The honest constraint logic

Here is how to think about the decision:

Buy a used RTX 3090 if:

• The model fits in 24GB and you do not want to pay about twice the sold median for the same memory ceiling
• A budget under $1,200 does not clear CompSniper’s typical sold band, which starts at $1,329. That June “under ~$1,200” line is not an October buy signal
• You are not doing heavy diffusion, video, or other compute-heavy work where the 4090’s extra FLOPS are the point

Browse used RTX 3090 24GB on eBay →

Buy a used RTX 4090 if:

• You will pay the October sold median, about $3,000 (typical $2,850–$3,299), and the sample problems above are acceptable
• You need prompt processing speed on large inputs or non-LLM workloads
• You are building a multi-user serving system and have the engineering to match
• You have confirmed the seller’s reputation and verified the card’s thermals
• Four hours a day does not, by itself, pay the card back against the August rental band on the math above

Browse used RTX 4090 24GB on eBay →

Rent a 4090 or 3090 if:

• Your usage is intermittent (a few hours per week or less)
• You need the speed for a specific project and will not use it after
• You want to avoid the capital and thermal/electrical overhead
• You are testing a model or workload before committing to hardware

Rent a 4090 on RunPod →

Consider alternatives if:

• You need bigger than 24GB models: two RTX 3090s (48GB) or Apple Silicon
• You want the best single-card speed at any price: the 4090 is it, but understand the cost
• You need to understand why GPU prices are inflated right now: see why GPU prices are so high in 2026

A note on supply and used-market noise

The used RTX 4090 sample CompSniper published on 2026-10-06 is thin: 21 sales, and the recent table still includes a heatsink-without-GPU, a parts-only card, and a brand-new outlier. The June “$2,000+ , some auctions $2,400, others $1,800” line is not the October sold median. Before paying $3,000, use the checks in the used GPU buying guide. Demand photos and confirm the return policy. A paste-and-clean budget was not re-priced on October 6, so this page does not quote one.

Bottom line

For local LLM decode on a model that fits in 24GB, a used 4090 at the $3,000 sold median is about $1,501 more than a used 3090 at $1,499. The community 7B gap this page cites is about 20%, and the catalog bandwidth gap is about 8%. Neither was re-measured on October 6. Prompt processing, diffusion, video, or multi-user serving is where the extra card money has a clearer job. If the work is single-stream chat or coding on a model that fits, the used RTX 3090 is the lower bill. The June “pocket the $1,200 you save” line assumed a $2,000 4090 and a $600–$800 3090.

If you need to test the speed before buying, rent a 4090. This page is not quoting a fresh weekly rental total. The August hourly band is above, and it was not re-fetched on October 6.