Best GPU for Local LLM Inference (October 2026): Used 3090 for 24GB, Not for Every Budget

For a model that fits in 24GB, the value GPU in October 2026 is a used RTX 3090: CompSniper’s eBay sold median on 2026-10-06 was $1,499 (typical $1,329–$1,599, 204 sales). That is $1,499 / 24 = $62.46 per gigabyte for the card alone, with 936 GB/s of GDDR6X. Step up to a used RTX 4090 when you will pay a $3,000 sold median for faster single-card decode of the same 24GB. Move to a listed RTX 5090, a Strix Halo mini PC, a Mac Studio, or a DGX Spark when 24GB is not enough or when you want one new box. The rest of this guide is that split, with sold prices, asking prices, and whole-system prices kept apart.

This page used to open with a used 3090 at about $500–$800 as the GPU nothing else could match. That string was the June 28, 2026 top pick. The price spine’s June 29 listing range was $600–$800, and the August 1 sold range was $580–$780. Those rows are history. The October median is about double.

Most “best GPU” lists still rank cards by gaming FLOPS. For a language model, two specs decide almost everything: VRAM — whether the model fits — and memory bandwidth — how fast it decodes once it fits. Compute matters less than the marketing implies, because token generation is dominated by reading weights out of memory.

This guide is the product-decision layer of the GPU cluster. The sizing math is in What Is Quantization and The Local-AI Hardware Buying Framework. The single-card checklist is the used RTX 3090 buying guide.

Buying too little VRAM means the model does not load, or it spills into system RAM and decode collapses. Buying a 128GB computer to run a 7B model means paying for capacity and then living with less bandwidth than a 3090. Match the constraint you actually have.

The core principle: VRAM = fit, bandwidth = speed

• VRAM determines what you can run. Weights, runtime overhead, and the KV cache for your context have to fit. If they do not, the model fails to load or offloads layers over PCIe, and throughput falls off.
• Memory bandwidth determines how fast it runs. Each token re-reads the weights. More GB/s produces more tokens per second than a higher FLOPS number does, for this workload.

A 7B model is roughly 4–5 GB at Q4_K_M, ~8 GB at Q8_0, and ~14–16 GB at F16. The quant sets the VRAM, and the VRAM sets the card. See What Is Quantization for the per-quant table.

24GB is the meaningful discrete-card tier for common local models. It holds a 7B at Q8 with context, a 13B at Q4 with headroom, and the 27B-class quants people were loading on 24GB NVIDIA cards in September 2026. It does not hold a 70B at a usable quant. That ceiling is why a used 3090 and a used 4090 are the same size of decision, and why a 128GB box is a different decision.

October 2026 prices, in the units they were measured

Every dollar below carries the date it was fetched. The 3090, 4090, mini-PC, Mac, and Spark figures are the 2–6 October checks and were not re-fetched on 15 October. The two RTX 5090 street rows are the 15 October checks. Tok/s in the next section are older community cites and are not in this table, because they were not re-measured. Bandwidth for the 3090 and 4090 is the hardware catalog, verified 2026-06-27. Empty cells were not fetched. Apple’s Mac Studio shop page, fetched 2026-10-06, returned no dollar prices; Mac dollars are ComputingForGeeks’ October 3 configurator check only.

Option	Price kind and date	Figure	What the price buys	$/GB	Bandwidth
Used RTX 3090	eBay sold median, 2026-10-06	$1,499 (typical $1,329–$1,599)	24GB card	$62.46 at the median; $55.38–$66.63 across the typical band	936 GB/s
Used RTX 3090	Asking going rate, 2026-10-06	$1,655	24GB card, not a sale	$68.96	936 GB/s
Used RTX 4090	eBay sold median, 21 sales, 2026-10-06	$3,000 (typical $2,850–$3,299)	24GB card	$125.00	1008 GB/s
RTX 5090	vga.live store-sold low, fetched 2026-10-15	$5,700	32GB, tracker header	$178.13
RTX 5090	videocardprices.com cheapest live street, as of 2026-10-15	$6,900.00	Index row has no memory size. $/GB uses vga.live’s 32GB	$215.63
Framework Desktop 64GB	Maker-store check, updated 2026-10-02	$1,959	Whole system, 64GB	$30.61	256 GB/s class
Bosgame M5 128GB/2TB	Amazon US, re-checked 2026-10-02	$3,324.04	Whole system, 128GB	$25.97	256 GB/s class
Mac Studio M5 Max, 40-core, 128GB, 512GB SSD	Configurator check, 2026-10-03	$5,099	Whole system	$39.84	614 GB/s
Mac Studio M5 Ultra, 80-core, 256GB, 1TB	Configurator check, 2026-10-03	$10,799	Whole system	$42.18	1.2 TB/s
DGX Spark 64GB	NVIDIA starting price, on sale 2026-10-23	$4,999	Whole appliance	$78.11	273 GB/s
DGX Spark 128GB	ComputingForGeeks vendor-store figure, 2026-10-03	$6,950	Whole appliance	$54.30	273 GB/s
RTX 3060 12GB			12GB on this page’s June 28 spec cite
RTX 5090 launch MSRP	Both trackers, 2026-10-15. NVIDIA’s product page was not re-opened	$1,999	Launch price, not a shop receipt
RTX 5090 sold median	Not fetched on 2026-10-15

CompSniper’s 3090 recent table includes 3090 Ti sales (including $1,749 and $1,799 cards in the first days of October). The $580 and $3,650 outliers are not the planning band. About $19 shipping sits on top of CompSniper’s sale figures. RigPrice’s $1,655 is asking: 92 active used listings, average asking $1,550–$1,843. A delisted asking price is not a sold price. RigPrice’s new-condition bucket near $2,500 and its for-parts rate near $820 are not used as the price of a working 3090.

The 4090 sample is 21 sales and still includes a heatsink-without-GPU near $239.99, a parts-only card near $1,700, and a brand-new listing near $5,210.87. Cite $3,000 with that mess named. $3,000 / $1,499 is about 2.00×.

The 15 October 5090 rows are two street reads, not a sold median and not a low-high pair. vga.live, fetched 2026-10-15, prints Lowest $5,700 (Gigabyte Infinity OC Dual-Fan at Micro Center $5,700; 40 of 48 cards in stock; launch MSRP $1,999). videocardprices.com, as of 2026-10-15, prints a cheapest live street price of $6,900.00 (+245.2% vs $1,999.00). $5,700 / 32 = $178.13/GB and $6,900 / 32 = $215.63/GB. The 32GB figure is vga.live’s header. The index row does not print a memory size, so the $6,900 division uses that same 32GB. The 6 October PC Tech Vault reading — in-stock low $4,100, 90-day median $4,900, other listing $6,899 — is the previous reading and was not re-fetched. A 15 October sold median was not fetched. NVIDIA’s product page was not re-opened; both trackers print the $1,999 launch MSRP.

Other 128GB Strix Halo checks from ComputingForGeeks, updated 2026-10-02, so the Bosgame row is not the only box: Framework Desktop 128GB DIY with no SSD $3,449 (out of stock); GMKtec EVO-X2 $3,499.99 (128GB/1TB) and $3,649.99 (128GB/2TB); Minisforum MS-S1 Max 128GB/2TB $3,799; Beelink GTR9 Pro 128GB/2TB $4,349. GMKtec 64GB + 1TB was $2,199.99. The base Mac Studio M5 Max at $2,499 in the October 3 check is 32-core, 36GB, 460 GB/s ($2,499 / 36 = $69.42/GB, whole computer). The M5 Ultra starts at $5,499 with 96GB in that check. A 512GB Ultra had no price.

The $6,950 Spark figure is ComputingForGeeks on October 3, which says NVIDIA raised the 128GB price on October 2. The NVIDIA blog post fetched here (Allen Bourgoyne, October 2) states the 64GB partner systems at $4,999 and does not state a 128GB dollar price. Bandwidth of 273 GB/s is the NVIDIA product-page spec extract (the page rendered as JavaScript) and that ComputingForGeeks article. Strix Halo at 256 GB/s is ComputingForGeeks, attributed to AMD, not a page this refresh opened on amd.com.

Older speed ranges, labeled so they are not October measurements

These tok/s figures are community-cited (r/LocalLLaMA, llama.cpp threads, 2024–2025), not independently verified by LocalRig, and not re-run on 2026-10-06. They assume a 7B model at Q4_K_M, roughly 4,096-token context, single user. They are planning ranges from the 7B/8B hardware guide. They do not update the price table.

GPU	VRAM	~7B Q4_K_M tok/s (2024–2025 community)	Power
RTX 4090	24 GB	~120–160	450W catalog spec
RTX 3090	24 GB	~80–110	350W catalog spec
RTX 3060	12 GB	~40–60	~170W as previously described on this page; October spec not re-fetched
Apple M3 Max 128GB	128 GB unified	~50–65	The June 28 page cited Apple for 400 GB/s. Not re-checked. October price not fetched.

A separate, newer community cluster — Qwen 3.8 27B on a 3090, about 41 tok/s with llama.cpp MTP in August and September 2026 — is documented on the used 3090 buying guide and was not re-measured here. modelfit.io’s ~87 tok/s on 32B-class models is the same kind of carry-forward.

LocalRig’s own first-party row is still the base Apple M4 (16 GB): 18.4 tok/s (llama.cpp b9820) and 19.5 tok/s (Ollama 0.30.11) on Llama 3.1 8B Q4_K_M, measured 2026-06-27. That is a small-Mac data point. It is not a 3090 measurement and it is not an M5 Max measurement.

The picks, by the constraint you actually have

Model fits in 24GB, and you want CUDA bandwidth: used RTX 3090

This is the value pick against a used 4090, not against every computer in the table. At a $1,499 sold median you get 24GB and 936 GB/s. The 4090’s catalog bandwidth is 1008 GB/s, about 8% higher on the spec sheet, at about twice the sold median. The 2024–2025 community tok/s say the 4090 is faster. They do not say it runs a bigger model. Nothing else fetched this week is a 24GB discrete card at this sold price. A new 4090 street price was not fetched, so this page does not rank the used 3090 against one.

Browse used RTX 3090 24GB on eBay →

Same 24GB, faster single card: used RTX 4090

Buy it when the watts (450W catalog spec) and the $3,000 sold median are acceptable and the model already fits. $3,000 / 24 = $125.00/GB, about double the 3090’s $62.46. The sample is 21 sales and imperfect. Treat the median as a noisy sold observation, not a clean index.

Check RTX 4090 24GB pricing on Amazon →

One new card up to 32GB: RTX 5090, at listed retail

Buy it when the model needs more than 24GB and at most 32GB, and you will pay a 15 October street read rather than a sold median. vga.live’s store-sold low was $5,700 ($178.13/GB at 32GB). videocardprices.com’s cheapest live street price was $6,900.00 ($215.63/GB). Both are worse dollars per gigabyte than the 6 October 3090 sold median, which was not re-fetched, and both are list prices, not completed eBay sales. The 6 October in-stock low of $4,100 and 90-day median of $4,900 are the previous reading. Bandwidth was not re-fetched, so this page does not claim a GB/s win or loss against 936.

The model does not fit in 24GB, and you want one box

• Strix Halo mini PCs. Framework Desktop 64GB at $1,959 ($30.61/GB, whole system) and Bosgame M5 128GB/2TB at $3,324.04 ($25.97/GB, whole system) are cheaper per gigabyte than a 3090 card because the price includes the computer and a much larger memory pool. Bandwidth at 256 GB/s does not beat 936 GB/s. They win on fit and on “one plug,” not on decode speed for a model that already fits in 24GB.
• Mac Studio. The 128GB M5 Max at $5,099 has 614 GB/s, still under the 3090. The 256GB M5 Ultra at $10,799 has 1.2 TB/s, which is the row that beats the 3090 on bandwidth and on capacity. The $2,499 base studio is 36GB at 460 GB/s: more memory than 24GB, less bandwidth, $69.42/GB for the whole machine.
• DGX Spark. 64GB from $4,999 ($78.11/GB) beats a 3090 when you need 25–64GB in one new CUDA appliance. It does not beat it on dollars per gigabyte or on bandwidth (273 GB/s). Two 3090 cards at $2,998 are less card spend than that $4,999 box for 48GB, before the host, which was not re-priced. The 128GB Spark at $6,950 ($54.30/GB) is the ComputingForGeeks figure, not the NVIDIA post.

Two used 3090s: 48GB of cards, not double speed

2 × $1,499 = $2,998 for 48GB, still $62.46/GB, card only. The typical pair is $2,658–$3,198. The asking pair is 2 × $1,655 = $3,310 and is not a sold total. That card spend matches one used 4090 and buys twice the VRAM, then you still buy a board, a 1000W-class PSU, and a case. Those host prices were not fetched in October. Two cards do not double single-stream tok/s. See Two RTX 3090s vs One RTX 4090.

Budget card under the 3090: RTX 3060 12GB, price left blank

An October price for a 12GB 3060 was not fetched, so this page will not repeat the old “under about $300” line as if it were current. The June 28 page treated 12GB as the floor where a 7B fits and a 13B Q4 is tight, with community tok/s around 40–60 that were not re-measured. If a real 3060 quote is still far below $1,329, it can be the short-term card. This refresh does not assert that quote.

RTX 3060 12GB on Amazon → · Browse used RTX 3060 12GB on eBay →

Multi-GPU reality: capacity, not linear speed

Two GPUs in a box do not double tokens per second for single-stream inference. Each token reads the weights, and weights split across two cards have to be coordinated over PCIe. PCIe is a fraction of on-card bandwidth, so it becomes the bottleneck. A second 3090 is a second $1,499 (sold median) of VRAM, which is the right purchase when the model does not fit, and the wrong purchase when you wanted 2× speed. Concurrent users are a different problem — batching and the engine matter more than a second gaming card. See how to run LLMs locally.

Buying used: the checks that matter more at $1,499

The 3090 is years past a manufacturer warranty. The higher sold price makes a bad card a larger loss than it was in June. A few habits still do the work:

• Assume as-is. Read the seller’s return window before you bid. There is no NVIDIA warranty riding along.
• Demand photos of the actual card, including the heatsink and the outputs. Confirm it is a 3090 and not a 3090 Ti. Public sold tables are already mixing the Ti in.
• Budget for fresh thermal paste on a card this old. Dried paste throttles a long run.
• Ask about the previous workload. Ex-mining cards can be fine, and they can be worn out. A specific answer is more useful than a stock photo.
• Run memtest_vulkan for a full pass before you trust the memory. The procedure is on the used 3090 buying guide.

Who this is not for

• Training or fine-tuning as the main job. This page sizes inference. A used 3090 can do light fine-tuning; the dollars-per-gigabyte split above is not a training budget.
• Many concurrent users. One 24GB card serves one stream well. Multi-user serving wants a batching engine and a different hardware budget.
• A model past 24GB where you also want 3090-class bandwidth in one card. The fetched one-box options clear the capacity and give up bandwidth, except the M5 Ultra at $10,799. A dual-3090 build keeps 936 GB/s per card and adds a host plus a split.
• A shopper who has not sized the model. The card is the wrong first question. Read the quantization guide and the buying framework first.

Frequently asked questions

What is the best GPU for running local LLMs in October 2026?

For a model that fits in 24GB, a used RTX 3090 at the $1,499 sold median (2026-10-06) is the value pick: 24GB, 936 GB/s, $62.46/GB for the card. It is not the best dollar per gigabyte of memory against a 128GB mini PC, and it is not the pick past 24GB. Asking price the same day was $1,655, labeled asking.

Is the RTX 4090 worth it over a used RTX 3090 for local AI?

When you want faster single-card decode and will pay the $3,000 sold median for the same 24GB. Catalog bandwidth is 1008 GB/s against 936 GB/s. Community 7B ranges of ~120–160 versus ~80–110 tok/s are 2024–2025 and were not re-measured. The 4090 does not unlock a larger model.

How much VRAM do I need to run a 7B or 13B model?

A 7B model needs roughly 4–5 GB at Q4_K_M and about 8 GB at Q8_0. A 13B model at Q4_K_M is tight on 12 GB. 24GB is the comfortable discrete tier for those sizes.

Does adding a second GPU double local inference speed?

No. A second card buys VRAM capacity. Two 3090s at the sold median are $2,998 for 48GB of cards, not twice the tok/s of one card.

Should I buy a used GPU for local AI?

For 24GB of CUDA bandwidth, yes, with a memory test and no expectation of a warranty. For a model that does not fit in 24GB, price a whole system from the October table before you buy a second used card.

Methodology

• Sold 3090 and 4090 figures: CompSniper, fetched 2026-10-06. Sample problems are stated next to the medians. The spine stores the 3090 typical band $1,329–$1,599; the median is additional.
• Asking: RigPrice, fetched 2026-10-06, not treated as sold.
• 5090 street: vga.live and videocardprices.com, fetched 2026-10-15. Two cheapest-style reads, not one range. The 6 October PC Tech Vault $4,100 / $4,900 median is history. A 15 October sold median was not fetched. 32GB is the capacity those trackers print.
• Mini PCs, Macs, and the 128GB Spark dollar figure: ComputingForGeeks, updated 2026-10-02 and 2026-10-03. The 64GB Spark starting price is NVIDIA’s October 2, 2026 post. Apple’s shop page did not return prices.
• Specs for 3090 and 4090: catalog verified 2026-06-27. No new tok/s. Community ranges stay dated 2024–2025. First-party M4 row stays 2026-06-27.
• RTX 3060 October price, 5090 bandwidth, new 4090 street price, and host-PC parts for a dual-3090 build were not fetched and are not guessed.

Sources

• CompSniper, RTX 3090, fetched 2026-10-06: https://compsniper.com/worth/rtx-3090
• RigPrice, RTX 3090 asking, fetched 2026-10-06: https://rigprice.com/gpu/rtx-3090/
• CompSniper, RTX 4090, fetched 2026-10-06: https://compsniper.com/worth/rtx-4090
• vga.live, RTX 5090 store-sold low, fetched 2026-10-15: https://vga.live/gpus/geforce-rtx-5090
• videocardprices.com GPU Price Index, RTX 5090 cheapest live street, as of 2026-10-15: https://videocardprices.com/gpu-price-index/
• PC Tech Vault, RTX 5090 listed retail, fetched 2026-10-06, previous reading: https://pctechvault.com/market-watch/geforce-rtx-5090/
• NVIDIA, Allen Bourgoyne, October 2, 2026: https://blogs.nvidia.com/blog/local-ai-dgx-spark-64gb-sync/
• NVIDIA DGX Spark product page, spec extract 2026-10-06: https://www.nvidia.com/en-us/products/workstations/dgx-spark/
• ComputingForGeeks, updated 2026-10-02: https://computingforgeeks.com/ryzen-ai-max-395-mini-pc-comparison/
• ComputingForGeeks, updated 2026-10-03: https://computingforgeeks.com/best-mini-pc-for-local-ai-llm/
• LocalRig first-party, base Apple M4, 18.4 tok/s and 19.5 tok/s, Llama 3.1 8B Q4_K_M, 2026-06-27, methodology in the 7B/8B hardware guide.

Prices that carry a 2026-10-06 date were fetched that day and were not re-fetched on 2026-10-15. The RTX 5090 street rows are the 2026-10-15 fetches. dataDate stays 2026-10-06 so the unrechecked 3090 and 4090 comps are not redated. Older rows are labeled with their own dates. Verify a live listing before you pay; a median is not a quote from one seller.