Benchmarks

Benchmark guides with methodology

Dated guides remain available here when they document a method but do not provide a complete structured record suitable for a side-by-side comparison.

• Can I Run LFM2.5-24B-A2B Locally? Official Q4 Is 13.529 GiB
  Data: Oct 22, 2026 Liquid AI's LFM2.5-24B-A2B is a text MoE. The card lists 24.0B total and 2.45B active, with a 131,072-token context. The official Q4_K_M GGUF is 13.529 GiB, so the file is under 16 GiB and over 12 GiB. NVIDIA named day-zero support on 21 October 2026 and did not publish a VRAM figure.
• Nemotron 3.5 Bolt: what the 74B '24GB' and the E4B '4GB' really need
  Data: Oct 22, 2026 NVIDIA's Bolt 74B-A9B GGUF is 22.018 GB (20.506 GiB). With the projector and drafter the three files are 23.050 GiB, under a 24 GiB card before KV. The published llama.cpp recipes start at 32 GB. E4B's 4 GB line is announced for 23 October 2026 and has no public weights yet.
• Can I Run Recursive Flint Locally? 24 GB Holds One Q4 File, Not the Other
  Data: Oct 22, 2026 Recursive Flint is a 39.8B MoE coding model with about 2.5B parameters active per token and a 262,144-token context. ggml-org's Q4_K_M is 23.39 GB. Bartowski's is 25.35 GB. A 24 GB GPU holds the smaller file and Unsloth's 18.49 GB 3-bit file. The 125+ and 150+ tok/s lines are vendor claims.
• Hardware for a Local Coding Agent: What Needs to Be Always On?
  Data: Oct 22, 2026 Separate the always-on agent gateway from the model machine, then size a box that can stay on all day. Idle and load watts for a 24–32GB GPU, a used dual-GPU rig, M5 Mac Studio, Strix Halo, and DGX Spark, each tied to a dated source, plus a monthly bill at the July 2026 U.S. residential rate.
• Can I Run DeepSeek-V4.1-Flash-1017 Locally? 763B Total, 552B Backbone
  Data: Oct 21, 2026 DeepSeek-V4.1-Flash-1017 is about 763B logical weights: a 552B backbone plus 196.6B Engram tables. Unsloth's 1-bit band is 128–145 GB with Engram on SSD; 3-bit is 256–290 GB. Not the 284B Flash-0731 page. A 24 GB GPU does not fit.
• Can I Run GLM-5.3-Flash Locally? Stock llama.cpp v0.6.0, Not a 24 GB Card
  Data: Oct 21, 2026 GLM-5.3-Flash is a 320B-A18B MoE. llama.cpp v0.6.0 (2026-10-05) is the first tagged release whose notes add it. ggml-org GGUFs are Q2_K at 124.62 GiB and Q4_K at 168.92 GiB. Unsloth's smaller dynamic files still have a fork snippet on their docs. Not a 24 GB card. Mac tok/s added 2026-10-21 are labeled community or vendor figures. LocalRig has not measured tok/s.
• Can I Run Muse Spark Open Locally? Two 512 GB Macs, or Four 256 GB
  Data: Oct 21, 2026 Meta's card states Muse Spark Open at 816B total and 42B active, with a 1M context. The Hub safetensors total is 822,036,529,664. Official NVFP4 weight shards are 533.011 GB. The Mac cookbook requires two 512 GB or four 256 GB M5 Ultra Studios. Apple lists 36.73 tok/s on four M5 Ultras at 8,192 tokens. That speed is a vendor table. LocalRig has not measured tok/s.
• Can I Run Ornith-1.5-122B Locally? Official Q4 Is 70.6 GiB
  Data: Oct 21, 2026 Ornith-1.5-122B is a 122B-class MoE with a 262,144-token config window. The official Q4_K_M GGUF is 70.635 GiB (70.6 GiB). A 24 GB card cannot hold that file. An 80 GB GPU is a Q4 file-size fit, not a fit for the 232 GiB BF16 file.
• Can You Run a Model Bigger Than Your RAM? SSD Expert Streaming on a Mac
  Data: Oct 21, 2026 SSD expert streaming keeps a bounded cache of routed MoE experts in memory and reads the rest from a local SSD. DwarfStar measured GLM-5.3-Flash Q4_K at 11.9 and 14.9 tok/s on a 128 GB M5 Max. MLX-Serve's about 35 tok/s on a 128 GB Mac is a vendor pre-release claim. LocalRig has not measured either.
• Is the RTX 5090 Worth It for Local AI in October 2026? $1,999 MSRP, $5,700 and $6,900 Street
  Data: Oct 15, 2026 Launch MSRP is still $1,999 on the 15 October 2026 trackers. vga.live's store-sold low that day was $5,700. videocardprices.com's cheapest live street price was $6,900.00. Those are separate reads, not one range. This page reprices 32GB against a used 3090, an RTX PRO 6000, a 128GB Strix Halo box, a Mac Studio, and DGX Spark.
• RTX Spark Laptops for Local AI: Which Config Is Worth It, and Which Is a Trap
  Data: Oct 15, 2026 RTX Spark laptop preorders opened 7 October 2026 and the first machines ship 16 October. Buy the 6,144-core 128GB configuration only if you need a Windows CUDA laptop above 27B-class 4-bit. The 24GB and 32GB configs are the memory trap. Independent reviews published 15 October 2026; the tok/s and agent times below are community-cited.
• Can I Run Kolibri-1 Locally? ~78 GB FP8, 3.46B Active, Not a 24 GB Card
  Data: Oct 6, 2026 Aleph Alpha Kolibri-1 (3 October 2026) is a 78B MoE with 3.46B active per token. Official FP8 weights are about 78 GB and want 2× A100 80GB or 1× H200 plus a vLLM plugin. Community GGUF and MLX builds need patched runtimes.
• Can I Run Muse Glimmer 30B Locally? 24GB Floor, DFlash, and the Agent Tax
  Data: Oct 6, 2026 Meta's Muse Glimmer 30B is a ~29.6B dense multimodal agent model. The published K-Quant-17GB stack with vision measures 19.0 GiB at 131,072 tokens. A 24 GB GPU is that cell. Vision, the DFlash drafter, and a 32 GB Mac each change the bill. LocalRig has not measured tok/s.
• Can I Run Qwen3.5 Locally? Picking Your Size from 3B to 235B
  Data: Oct 6, 2026 Qwen3.5 spans 3B to 235B parameters. This guide pairs your hardware to the right model size using requirement math, not marketing claims. A single table from 8GB cards to 256GB Macs, with Unsloth Dynamic 2.0 quantization recommendations.
• Hardware to Run a 32B Model Locally: The Sweet Spot Tier
  Data: Oct 6, 2026 32B models hit the 2026 sweet spot: near-70B quality at half the hardware cost and power draw. This guide covers the VRAM math, which cards qualify (used 3090, A6000, 4090, some Macs), and when 32B at Q5 outperforms 70B at Q2 in practical work.
• Hardware to Run a 70B Model Locally: VRAM, the 48GB Wall, and Your Real Options
  Data: Oct 6, 2026 What it actually takes to run a 70B model at home: the VRAM math, why 48GB is the practical floor at Q4, and the five hardware paths (dual 3090, used A6000, Apple Silicon, DGX Spark, or cloud).
• The Local-AI Hardware Buying Framework
  Data: Oct 6, 2026 A constraint-first framework for choosing hardware to run AI models locally. Covers VRAM, memory bandwidth, quantization, Apple Silicon, DGX Spark, and budget paths — so you buy once and regret nothing.
• Quantization: What It Means for Local AI and Why It Matters
  Data: Oct 6, 2026 Quantization reduces the numerical precision of a model's weights to shrink its memory footprint — the single technique that determines whether a 7B or 70B model fits in your GPU's VRAM and how fast it will run.
• Which Quant Should You Download? Q4_K_M vs Q8_0 vs F16, Decided by Your Hardware
  Data: Oct 6, 2026 The download page forces a choice — Q4_K_M, Q8_0, F16, or an IQ variant — and most guides answer it with vibes. This is a constraint selector: your VRAM/unified memory sets the ceiling, your quality tolerance sets the floor, and for almost everyone the answer is Q4_K_M.
• AMD 7900 XTX for Local LLMs in 2026: ROCm Finally Grew Up
  Data: Oct 6, 2026 ROCm 7.2 (March 2026) closed the compatibility gap that used to make AMD a hard pass for local LLM inference. This guide covers what "parity" actually means for the 7900 XTX's 24GB, where it beats the RTX 4090 on price, where it loses on speed, and who should still buy CUDA.
• AMD MI50 32GB: The $150 HBM2 Wildcard for Local LLMs
  Data: Oct 6, 2026 The MI50's June 2026 eBay band was $120–$210, about 8–14% of CompSniper's 6 October 2026 used RTX 3090 sold median of $1,499. That MI50 dollar was not re-checked. ROCm support is in maintenance mode, so drivers will not improve and may decay. A knowing bet for tinkerers; explicitly not a recommendation for anyone's only GPU.
• Best GPU for Local LLM Inference (October 2026): Used 3090 for 24GB, Not for Every Budget
  Data: Oct 6, 2026 A used RTX 3090 at the October 6, 2026 sold median of $1,499 is the value pick when the model fits in 24GB and you want CUDA bandwidth. A used 4090, a listed 5090, a Strix Halo mini PC, a Mac Studio, or a DGX Spark beats it in specific cases, and several of those prices are whole computers rather than cards.
• Best GPU Under $500 for Local LLMs in 2026, Ranked by Constraint
  Data: Oct 6, 2026 At sub-$500, the GPU decision shifts from "what will fit" to "what will actually run without exploding the power supply." A constraint-ranked guide to the used 3060 12GB (safest default), Tesla P40 (VRAM-dense but cooling-complex), and the RTX 5060 Ti 16GB (best new-card path, just over budget).
• GPU VRAM Tiers for Local AI: What 8, 12, 16, 24, 48GB Actually Buy You
  Data: Oct 6, 2026 A constraint-first selector that maps VRAM tiers—8GB, 12GB, 16GB, 24GB, 48GB—to the model classes they serve at usable quantization, with the heuristic to size your tier and the best-value GPU card per tier. No guesswork: know exactly which models fit where.
• How to Buy a Used GPU Without Getting Burned: Vetting, Testing, and Red Flags
  Data: Oct 6, 2026 The used GPU market holds real value — and real risk. This guide walks through seller red flags, the first-48-hours test protocol (VRAM testing, sustained load, thermals), and return-window discipline. Persona 2's defective VRAM module risk is why this page exists.
• RTX 3060 12GB for Local LLMs: The Honest Entry Point Under $300
  Data: Oct 6, 2026 A 12GB VRAM card for under $300 can run 7B–8B models comfortably at Q4 quantization. This is the right first GPU when budget rules everything; the wrong choice if you already know you want bigger models. We show the model-fits math, the speed ceiling, and the upgrade path.
• RTX 4090 for Local LLMs in 2026: Great Card, Broken Price
  Data: Oct 6, 2026 On October 6, 2026 a used RTX 4090 sold for a $3,000 eBay median and a used RTX 3090 for $1,499. Same 24GB. About 2.00× the card money, not the June 2.5× gap. Community 7B tok/s from 2024–2025 were not re-run.
• Tesla P40 in 2026: 24GB of VRAM for $150 — Legit Bargain or Trap?
  Data: Oct 6, 2026 The Tesla P40 costs $150–200 used and holds 24GB of VRAM. It runs local LLMs, but passive cooling throttles hard without forced airflow, compute is ancient, and there is no video output. A constraint guide: right for the tinkering homelabber who enjoys the project; wrong for anyone who wants inference to work on day one.
• Two Used RTX 3090s vs One RTX 4090: Same Card Money, Twice the VRAM
  Data: Oct 6, 2026 On October 6, 2026 two used RTX 3090s at the sold median were $2,998 in cards for 48GB, and one used RTX 4090 at the sold median was $3,000 for 24GB. The decision is still whether the model fits in 24GB. The old "pair costs less" claim does not survive the October sold medians.
• Used RTX 3090 Buying Guide (October 2026): $1,499 Sold, and When It Is Not the Value Pick
  Data: Oct 6, 2026 On October 6, 2026 a used RTX 3090 sold for a $1,499 eBay median, about double the June and August observations still sitting on older LocalRig pages. It is still the value pick when the model fits in 24GB and you want discrete CUDA bandwidth. It is not the cheapest dollar per gigabyte once 128GB mini PCs and Macs are in the comparison.
• Used RTX A6000 48GB for Local AI: The Single-Card Ceiling for 32B Models
  Data: Oct 6, 2026 As enterprises refresh to Blackwell, used RTX A6000 prices on this page were last observed at $600–$1,200 and were not re-checked on 2026-10-06. Two used 3090s at CompSniper's $1,499 sold median are $2,998 in cards. That June claim that the A6000 has worse VRAM-per-dollar than the pair does not survive the October card price, and this page does not name an October winner.
• Why GPU Prices Spiked Again in 2026: The DRAM Shortage, Explained
  Data: Oct 6, 2026 2026's GPU price spike is rooted in DRAM memory shortage, not a repeat of the 2021–22 crypto boom. AI datacenters are locking up supply years ahead; the timeline to clearance is uncertain. Your honest options: buy used, switch to AMD, rent instead, or wait—and the right choice depends on your workload and timeline, not scarcity pressure.
• Why VRAM (and Memory Bandwidth) Matters More Than Compute for Local LLMs
  Data: Oct 6, 2026 The single concept that explains the used-GPU market: LLM inference—especially token generation—is memory-bandwidth-bound, not compute-bound. Older cards with fat VRAM beat newer cards with thin VRAM. Learn capacity vs. bandwidth vs. compute, and why a 2020 Tesla P40 or 2021 AMD MI50 often outperform newer GPUs for local inference on a budget.
• How Much Unified Memory Do You Need for Local LLMs? 48GB to 256GB, Mapped to Models
  Data: Oct 6, 2026 A constraint-first guide to Apple Silicon unified memory sizing: the heuristic that maps each memory tier (48/64/128/256GB) to the models it can actually run, and why capacity alone does not guarantee usable speed. Avoids the two costliest Apple mistakes.
• Mac Studio M5 Max and M5 Ultra for Local LLMs: Which Configuration to Buy
  Data: Oct 6, 2026 M5 Max and M5 Ultra Mac Studio have been on sale since September 22, 2026. This is the configuration guide: which unified-memory tier fits which model class, what Apple's bandwidth and list prices actually are, and when an older M3 Ultra or a non-Mac box is the better buy. M5 tokens per second are not yet measured.
• Mac Mini M4 Pro for Local LLMs: The Quiet Entry Point to Apple Inference
  Data: Oct 6, 2026 The Mac Mini M4 Pro with 48GB unified memory is the budget door into local LLM inference: runs 7B–32B models quietly at 30–45W, but unified memory bandwidth limits it to small-to-medium models. A practical guide to sizing, headless setup, and the hard ceiling where larger models hit the wall.
• Mac Studio M3 Ultra for Local LLMs: 800GB/s, 256GB, and the Prefill Problem
  Data: Oct 6, 2026 Apple's top inference chip: 800GB/s bandwidth and 256GB unified memory. But the split personality is real—excellent decode throughput on large models, brutal prefill latency that turns long-context work into minutes of waiting. Plus March 2026: the 512GB config is discontinued, capping new Studios at 256GB.
• Beelink GTR9 Pro for Local LLMs: 128GB Unified Memory, Price Checked Oct 2026
  Data: Oct 6, 2026 The GTR9 Pro is a 128GB Ryzen AI Max+ 395 mini PC. Beelink's own store listed it at $4,349 on 2026-10-06, not the $1,899 figure from June. Bandwidth for this platform is AMD's 256 GB/s Strix Halo number. Throughput on this page was not re-measured.
• Best Mini PC for Local LLMs (Oct 2026): Strix Halo 128GB vs Gorgon Halo 192GB
  Data: Oct 6, 2026 A 128GB Strix Halo mini PC is still the capacity buy when the model fits in the memory the GPU can actually use. A 192GB Gorgon Halo box costs about 1.8× a $3,799 128GB listing for 1.5× the installed RAM and only about 7% more bandwidth. This page prices that choice from manufacturer listings checked 2026-10-06 and leaves unmeasured tok/s cells empty.
• DGX Spark 64GB vs 128GB: Which to Buy After the October Price Split
  Data: Oct 6, 2026 NVIDIA's 64GB DGX Spark ships from partners on 23 October 2026 from $4,999, while The Register reports the 128GB list at $6,950. This page maps which models fit each pool, and when a used 3090, Strix Halo mini PC, or Mac is the better buy.
• NVIDIA DGX Spark vs 2× RTX 3090 for Local AI: Buy the Box or Build the Rig?
  Data: Oct 6, 2026 NVIDIA DGX Spark and a dual-RTX-3090 workstation solve the same 70B local-AI problem in very different ways. Compare memory, software, power, multi-device scaling, cost, and the two purchase paths before choosing a turnkey Spark or a DIY 3090 rig.
• How Do You Run Qwen 3.8 Flash-Next on Two DGX Sparks in 2026?
  Data: Oct 6, 2026 Follow Mia's public dual-DGX-Spark vLLM NVFP4 runbook for Qwen3.8-Flash-Next. Two GB10 boxes are tensor-parallel serving, not one 256GB pool. Public benches only — LocalRig has no first-party tok/s for this stack.
• Dual RTX 3090 Build Guide (October 2026): 48GB Costs About One Used 4090 in Cards
  Data: Oct 6, 2026 Two used RTX 3090s at the October 6, 2026 sold median are $2,998 in cards for 48GB, about the same card spend as one used RTX 4090 at a $3,000 sold median. The board, PSU, and case are extra, and they were not re-priced in October. This is the parts list, with sold prices and asking prices labeled separately.
• Framework Desktop as a Local AI Server: Strix Halo, ROCm, and the 128GB Question
  Data: Oct 6, 2026 The Framework Desktop with Strix Halo CPU offers up to 128GB unified memory, full repairability, and x86 Linux native support. But unified memory bandwidth is not GDDR6X, and ROCm tooling is still catching up. This is a guide to what you actually get: the constraints, the community-reported 70B feasibility, and when it trades off speed for serviceability.
• Strix Halo vs Mac Studio for Local LLMs: The Unified-Memory Showdown
  Data: Oct 6, 2026 Strix Halo (AMD Ryzen AI Max, 128GB unified memory, x86) and Mac Studio (M3 Ultra, up to 256GB unified memory, MLX) are the two open paths to running big models without a rack of GPUs. Neither wins outright — this scores both on bandwidth, ecosystem maturity, price-per-GB, and homelab fit.
• Tesla V100 Budget AI Homelab: Datacenter Cast-Offs as the Value Multi-GPU Path
  Data: Oct 6, 2026 2× Tesla V100 32GB with HBM2 bandwidth and NVLink capability at half the cost of new consumer cards — but earned only with cooling mods, blower management, and SXM2 adapter tinkering. For builders, not plug-and-play buyers.
• DigitalOcean GPU Droplets Review 2026: Beginner-Friendly, Not the Cheapest
  Data: Oct 6, 2026 DigitalOcean's GPU Droplets are the beginner-friendliest tier-1 cloud GPU option: predictable per-GPU-hour pricing, no spot preemption, and tight integration with existing DO infrastructure. Paperspace (acquired 2023) still runs as a separate subscription-gated product, not a merged one. For hobbyists escaping ChatGPT costs.
• GPUMart Review 2026: Bare-Metal GPU Servers and the Real Rent-a-4090-Monthly Math
  Data: Oct 6, 2026 GPUMart's monthly bare-metal RTX 4090 rental creates one of the few apples-to-apples rent-vs-buy comparisons in the market. When high utilization and no upfront capital meet a sub-one-year horizon, monthly bare metal can beat both hourly cloud and ownership. This guide cuts through the pricing layers.
• Lambda Cloud Review 2026: The $4.29 H100 Standard-Bearer for Serious Training
  Data: Oct 6, 2026 Lambda Cloud is the reference point builders compare against: clean datacenter H100s at ~$4.29/hr on-demand (single GPU) or ~$4.09/hr per GPU in an 8x cluster, with no marketplace variance. Wins decisively for multi-hour training and fine-tuning where interruption risk is high. Overkill for inference workloads where cheaper marketplaces undercut, and pricier than it used to be.
• Rent vs Buy a GPU for Local AI: The Break-Even Math Nobody Shows You
  Data: Oct 6, 2026 A full total-cost-of-ownership model for the rent-vs-buy GPU decision — purchase price, electricity, depreciation, idle time, and cloud egress — worked in both directions with real, attributed numbers. The answer is a utilization threshold, not a winner.
• RunPod Review 2026: Secure Cloud vs Community Cloud, Pricing, and the Data-Loss Gotcha
  Data: Oct 6, 2026 RunPod is the only tier-1 provider renting a real consumer RTX 4090. This guide covers Secure Cloud vs Community Cloud, when interruptions cost more than savings, the network-volume data-loss failure mode, and whether RunPod makes sense for fine-tuning or inference.
• TensorDock Review 2026: The $0.37/hr RTX 4090 Marketplace, Honestly Assessed
  Data: Oct 6, 2026 TensorDock's spot RTX 4090 rentals at $0.20–$0.37/hr are among the cheapest on the market. But they're marketplace listings with no quality guarantees — identical constraint logic to Vast.ai. When they work, the unit economics are real. When they don't, you have no recourse. Honest pricing, zero affiliation, pure editorial trust piece.
• How to Run llama.cpp on an RTX 3090 (CUDA, Step by Step)
  Data: Oct 6, 2026 A step-by-step guide to building llama.cpp with CUDA and running a GGUF model on an RTX 3090. Covers driver and toolkit prerequisites, the CUDA build, full GPU offload with -ngl, a throughput check, and an OpenAI-compatible server.
• Can I Run DeepSeek-V4-Flash-0731 Locally? 284B MoE, Not a 14B Card
  Data: Sep 10, 2026 DeepSeek-V4-Flash-0731 is a 284B-class MoE with 13B active, not a 14B dense download. Unsloth's 3-bit GGUF is ~103 GB and wants ~110 GB total memory. Spark or Mac 128 at 3-bit, or rent; a used 3090 is the wrong buy.
• Can I Run DeepSeek-V4-Pro-0813 Locally? 1.57T Skip/Rent, Not Flash-0731
  Data: Sep 10, 2026 DeepSeek-V4-Pro-0813 is 1.57T / 48B active, not Flash-0731's ~103 GB 3-bit. Unsloth Q4 GGUF is a ~20-shard ~850 GB-class download. Skip 24 GB and 128 GB. Rent a datacenter node. Do not size a 3090.
• Can I Run Gemma 4 26B-A4B Locally? 16–18 GB 4-bit MoE vs 31B Dense
  Data: Sep 10, 2026 Gemma 4 26B-A4B is a 256K MoE (4B active), not the 31B dense SKU. Unsloth's 4-bit band is 16–18 GB. That is a 16 GB maybe and a 24 GB used-3090 yes for weights; 8-bit wants 28–30 GB.
• Can I Run Gemma 4 31B Locally? Unsloth 4-bit 17–20 GB on a 24 GB Card
  Data: Sep 10, 2026 google/gemma-4-31B-it is a ~31B dense multimodal SKU with 256K context, not the 26B-A4B MoE. Unsloth wants 17–20 GB total memory at 4-bit. A used 3090 is the 24 GB cell; 16 GB is offload-or-skip at that band.
• Can I Run GLM-4.7-Flash Locally? 30B-A3B on 24 GB, Not GLM-5.3
  Data: Sep 10, 2026 GLM-4.7-Flash is Z.ai's 30B-A3B MoE, not GLM-5.3-Flash and not GLM-5.2. Unsloth's 4-bit path wants ~18 GB; they say it runs on 24 GB. UD-Q4_K_XL is 17.5 GB. A used 3090 is the cell; 5.3-Flash is 100 GB-class.
• Can I Run GLM-5.2 Locally? Hardware Requirements for the 1M-Context Flagship
  Data: Sep 10, 2026 GLM-5.2 (released June 2026) is open-weight but not runnable on most home hardware. The 1M-context model demands a 256GB-class machine and hits single-digit tokens/sec even then. Honest constraints, the math, and where to go instead.
• Can I Run gpt-oss-20b Locally? Native MXFP4 in 16 GB, Harmony Required
  Data: Sep 10, 2026 openai/gpt-oss-20b is 21B / 3.6B active with native MXFP4. OpenAI says it runs within 16 GB. Unsloth GGUFs are ~11.5–13.8 GB. A 16 GB card is the floor; a used 3090 is KV headroom. Harmony format is required.
• Can I Run Kimi K2.6 Locally? 1T MoE, 350 GB Floor, Not a 48 GB Card
  Data: Sep 10, 2026 moonshotai/Kimi-K2.6 is a 1T-parameter multimodal MoE (32B active), not a 48–72 GB 14B peer. Unsloth's Dynamic 2-bit GGUF wants ≥350 GB. Skip 24 GB and 128 GB boxes; rent or a 512 GB-class machine.
• Can I Run Kimi K2.7-Code Locally? Still 1T / ~595 GB Q8, Not a 3090
  Data: Sep 10, 2026 moonshotai/Kimi-K2.7-Code is a 1T / 32B-active coding SKU built on K2.6. Unsloth lossless Q8 is 595 GB; Q4 is ~10 GB smaller. Skip 24 GB and 128 GB. Same memory planet as K2.6. Rent, do not size a 3090.
• Can I Run Kimi K3 Locally? 2.8T / 594 GB 1-bit, Not K2.6
  Data: Sep 10, 2026 moonshotai/Kimi-K3 is 2.8T / 104B active with 1M context. Unsloth Dynamic 1-bit is 594 GB and they want ~610 GB total memory. Skip 24 GB, 128 GB, and K2.6's 350 GB class. Rent. Do not size a 3090.
• Can I Run MiniMax M3 Locally? 428B MoE, Not an 8–13B KV Essay
  Data: Sep 10, 2026 MiniMax-M3 is a ~428B / 23B-active multimodal MoE with 1M context, not an 8–13B model. Unsloth's 1-bit GGUF wants ~133 GB; 3-bit 164–200 GB. Spark or Mac 128 is a tight 1-bit maybe; a 3090 is a skip. GGUF is experimental.
• Can I Run Ornith-1.0-35B Locally? Q4 ~22 GB on 24 GB, Q8 Wants Dual 3090
  Data: Sep 10, 2026 Ornith-1.0-35B is the MoE sibling of Ornith 9B, not a 9B download. Unsloth UD-Q4_K_XL is 22.3 GB; Q8_0 is 36.9 GB. A used 3090 is a tight Q4 cell; dual 3090 is the Q8 conversation. Hub parameter overview is a shortfall.
• Can I Run Ornith-1.0-9B Locally? 9B Dense on 16 GB vs a Used 3090
  Data: Sep 10, 2026 Ornith-1.0-9B is a dense ~9B coding SKU, not a 100 GB MoE. Unsloth's Q4_K_M GGUF is 5.7 GB; BF16 is 17.9 GB. 16 GB and used 3090 both hold Q4; 24 GB is the longer-context card. Not Ornith-35B.
• Can I Run Qwen3.6-27B Locally? Prior 24 GB Driver, Not Qwen 3.8
  Data: Sep 10, 2026 Qwen/Qwen3.6-27B is the prior 27B multimodal daily driver, not Qwen 3.8. Unsloth's 4-bit band is 18 GB; 6-bit 24 GB. A used 3090 is the Q4 cell. MTP ~160 tok/s is Unsloth on RTX 6000, not a 3090 number.
• Can I Run Qwen 3.8 Flash Locally? 75 GB GGUF, Not the 27B Homepage SKU
  Data: Sep 10, 2026 Qwen 3.8 Flash is the 125B MoE / large-RAM path, not Qwen3.8-27B on a 24 GB card. Unsloth's Flash-Next GGUF is 75 GB at 1-bit and they ask for a 96 GB machine. Public speed cites are RTX PRO 6000 MTP and a 4090-plus-RAM offload run — not LocalRig benches.
• Qwen 3.8 27B Q4 at 200k: Spark, Mac 128, or Dual 3090?
  Data: Sep 10, 2026 At a 200k agent window, Qwen3.8-27B Q4 is a KV-cache decision, not a weights decision. This page maps Spark 128 GB, Mac 128 GB, and dual RTX 3090 against named public cite packets. A single 24 GB 3090 is the short-session path.
• Qwen 3.8 27B Q4 on a Used RTX 3090: The Short-Session Path
  Data: Sep 10, 2026 A used RTX 3090 still runs Qwen3.8-27B Q4 at 4k–32k. This page is the named 24 GB cell: Unsloth weight bands, llama.cpp fit limits, and community tok/s with URL, date, quant, and runtime. It is not the 200k agent default.
• Qwen 3.8 27B at 200k: Ollama vs llama.cpp vs vLLM
  Data: Sep 10, 2026 The same 27B Q4 weights are not the same job on Ollama, llama.cpp, and vLLM once context is 200k. This page routes by stack: GGUF on Ampere, NVFP4 on Blackwell/Spark, and what each runtime actually publishes for a long window.
• Can I Run Qwen 3.8 Locally? 27B vs Flash-Next vs the Giant MoE
  Data: Sep 7, 2026 Qwen 3.8 is three different local SKUs. This guide maps Qwen3.8-27B dense to 24 GB cards, Flash-Next to Spark and large unified memory, and the 125B / 2.4T MoEs to machines you probably should not buy for this model. Requirement math first; community tok/s second.
• Best GPU Cloud for Fine-Tuning vs Inference: Two Different Problems, Two Different Providers
  Data: Sep 7, 2026 Fine-tuning and inference have opposite constraint profiles. Fine-tuning is interruption-intolerant and hours-long; inference is bursty and restart-tolerant. A single provider does not win both. This guide routes you by workload: reliable datacenter tiers (RunPod Secure, Lambda) for training, cheaper marketplace and serverless tiers (RunPod Community, Vast.ai, Modal) for inference, with the price penalty of each mismatch stated clearly.
• Cheapest Place to Rent an RTX 4090 in 2026, Ranked by What You Actually Pay
  Data: Sep 6, 2026 Every "cheapest RTX 4090 cloud" list ranks by sticker price. This one ranks by realized cost — sticker plus variance risk, bandwidth fees, and interruption probability by provider class. The cheapest listing and the cheapest outcome are usually different providers.
• The Hidden Costs of Cloud GPUs: Bandwidth Fees, Preemption Multipliers, and Silent Throttling
  Data: Sep 6, 2026 Cloud GPU invoices diverge from the advertised $/hr rate in three places: bandwidth overage, preemption/interruption multipliers, and undisclosed throttling. This guide catalogs documented cases from Vast.ai, Modal, and RunPod and gives you a pricing framework to run before you rent, not after the bill arrives.
• Quiet Cooling for a Home GPU Server: Taming Blower Cards and 24/7 Noise
  Data: Aug 3, 2026 Home GPU servers are silent until they are not. A guide to the escalation ladder: undervolting, fan curves, passive datacenter shrouds, case selection, and closet airflow math — with honest expectations about dB trade-offs at each step.
• PSU Sizing for Multi-GPU AI Rigs: The Wattage Math That Keeps Your House Safe
  Data: Aug 2, 2026 PSU sizing for multi-GPU AI workloads is a safety issue, not a performance optimization. This guide covers transient-spike headroom, the 15A wall-circuit ceiling, and per-build wattage math for single 5090, dual 4090, and dual 3090 setups.
• NAS for AI Model Storage: Sizing a Library of GGUFs, Checkpoints, and Datasets
  Data: Jul 31, 2026 A GGUF collection plus fine-tuning checkpoints hits terabytes in months, not years. This guide sizes NAS capacity, explains why network speed (not caching) is the real constraint, and covers backup strategy for irreplaceable checkpoints.
• RTX 4060 Ti 16GB vs 5060 Ti 16GB: The Budget Pick That Changed in Months
  Data: Jul 30, 2026 The 4060 Ti 16GB was the default sub-$500 local LLM recommendation for two years. The 5060 Ti 16GB arrived near the same price with meaningfully better memory bandwidth, and bandwidth — not compute — is what token decode actually spends. Here is the price-conditional verdict, not a blanket "buy the new one."
• Minisforum vs Beelink vs GMKtec: Which Mini-PC Brand Can You Actually Trust?
  Data: Jul 30, 2026 Hardware reviewers compare specs. This compares the three mini-PC brands on after-sales support, firmware cadence, and community trust — because a 24/7 inference box failing at year two is a real cost. Honest breakdown of support history and sentiment.
• Why Prompt Processing Is Slow on Apple Silicon (and What Actually Helps)
  Data: Jul 29, 2026 Apple Silicon excels at token generation (decode) but struggles with prompt processing (prefill) on long contexts. The difference between TTFT and tok/s explains why headline throughput hides 10-minute waits, and which optimizations actually work.
• How to Avoid Losing Your RunPod Data When Your Balance Runs Low
  Data: Jul 29, 2026 RunPod terminates pods and network volumes with zero recovery when your account balance hits zero and you have no backup payment method. A five-minute prevention checklist: automatic payments, balance alerts, checkpoint-to-object-storage backups, and a decision rule for network volume costs.
• Is NVLink Worth It in 2026? The Dual-3090 Bridge Question, Answered
  Data: Jul 25, 2026 NVLink is a $40–80 bridge with outsized mythology. For inference workloads that layer-split models across cards, it barely matters. For tensor-parallel serving it can. This guide separates the workloads where the bridge pays for itself from those where it's cargo cult tech.
• Best UPS for a Home AI Server: Sizing for GPU Loads, Not Just Runtime
  Data: Jul 20, 2026 GPU rigs need UPS sizing for peak transient draw and pure sine wave inverters, not just runtime hours. This guide sizes by load tier (575W single-GPU, 1000–1200W dual-GPU), explains VA vs watts, and ranks by safety and workload constraint.
• Mac Mini vs Mac Studio for Local LLMs: Which Should You Actually Buy?
  Data: Jul 19, 2026 The Mac Mini and Mac Studio don't split on speed the way Apple's chip names imply — they split on memory bandwidth and unified memory ceiling. This is the constraint logic: which model sizes the Mini genuinely serves well, where the Studio actually earns its price, and the crossover line between them.
• Best Server Racks and Rack Accessories for a Home AI Lab
  Data: Jul 17, 2026 GPU servers need 30+ inches of usable depth, weight ratings above 500 lbs, and often-forgotten accessories like PDUs and cable management. A constraint-driven guide to 12U through 32U racks, sized for the builds LocalRig actually recommends.
• Cudo Compute Review 2026: Distributed GPU Cloud Without the Marketplace Roulette
  Data: Jul 9, 2026 Cudo Compute occupies the middle ground between Vast.ai's bargain chaos and hyperscaler lock-in: distributed GPU supply with SMB-grade account management. Honest assessment of who it fits, pricing anchored to H100 market rates, and when RunPod or Vast is the better choice.
• Modal Review 2026: 12-Second Cold Starts, and the Pricing Multipliers Nobody Reads
  Data: Jul 9, 2026 Modal's GPU snapshotting cut serverless cold starts from 118 seconds to 12 seconds. But non-preemptible, non-default-region pricing can quietly stack to 5x+ list cost. When serverless GPU makes sense, and when a rented pod wins.
• Salad Cloud Review 2026: $0.20/hr 4090s on Gaming PCs — Too Cheap to Be True?
  Data: Jul 9, 2026 Salad rents compute time on 60,000+ consumer gaming PCs at $0.20/hr for an RTX 4090. The catch is reliability: you get stateless, checkpointed, interruption-tolerant inference at a massive price advantage, or long stateful jobs fail mid-way. This guide shows when Salad wins and when to rent from a datacenter instead.
• Is Vast.ai Safe? An Honest 2026 Review of the Cheapest GPU Marketplace
  Data: Jul 9, 2026 Vast.ai is legitimate, not a scam. But it is a peer-to-peer GPU marketplace with a variance problem: host reputations vary widely, pricing can creep past advertised rates, and unattended workloads silently fail. This guide separates real risks from rumors and tells you when Vast makes sense.
• Vultr GPU Cloud Review 2026: Established IaaS for AI Workloads — Who It's Actually For
  Data: Jul 9, 2026 Vultr occupies the middle ground between hyperscaler pricing and GPU marketplace volatility. Datacenter-grade GPUs (A100, H100), hourly billing, no consumer chips, and a familiar control plane. The guide: when Vultr wins, why reliability costs, and who should look elsewhere.
• RunPod vs Vast.ai in 2026: Which Is Actually Cheaper (and When Cheap Isn't the Point)
  Data: Jul 7, 2026 RunPod and Vast.ai get compared on list price by almost every affiliate site, and almost none of them mention that Vast's cheaper number and its realized cost are not the same thing. The honest answer is bimodal: it depends on whether your workload can tolerate variance, not on which provider pays a better commission.
• Mac Studio Clusters over Thunderbolt 5: Trillion-Parameter Models at Home?
  Data: Jun 29, 2026 Four stacked Mac Studios over Thunderbolt 5, running macOS Tahoe RDMA, reportedly unlock trillion-parameter inference at home. A frontier-hobbyist path with honest cost-per-token math, attribution of unverified claims, and clarity on when cloud rental wins.
• Mac Studio vs RTX 5090 for Local AI: Unified Memory vs Raw Speed
  Data: Jun 29, 2026 A widely-shared 2026 thread claims the RTX 5090 "can't keep up" with Apple Silicon — and on giant MoE models, it's true: 32GB of VRAM simply can't hold what 128-256GB of unified memory can. But the 5090 crushes anything that fits in its VRAM and owns fine-tuning outright. The real answer is a decision tree by model size, not a winner.
• 10GbE vs 2.5GbE for an AI Homelab: When Model Loading Justifies the Upgrade
  Data: Jun 29, 2026 The math on pulling a 40GB 70B-Q4 GGUF from NAS: ~2.6–2.8 minutes at 2.5GbE versus ~42–43 seconds at 10GbE. A constraint-first decision tree: local NVMe first, network second, and why used enterprise SFP+ gear is the honest value path.
• Cheapest RTX 5090 Cloud Rental in 2026: Prices, Availability, and Why Supply Is the Story
  Data: Jun 29, 2026 The RTX 5090 rental market is defined by supply constraints, not competition. A 7.4× price spread ($0.27–$2.00/hr) across providers reflects which ones have cards at all. Compare Salad, RunPod, CloudRift, and Vast.ai; break-even math against the ~$3,800 street price for ownership; and when renting beats buying for realistic workloads.
• H100 Rental Price Comparison 2026: $1.38 to $12+ an Hour for the Same Chip
  Data: Jun 29, 2026 The same NVIDIA H100 SXM rents for ~$1.38-$1.99/hr on peer-to-peer marketplaces and ~$12.29/hr on Azure — a 6-9x spread for identical silicon. This comparison maps why the gap exists and which provider tier actually matches your workload, not just which table is cheapest.
• Best Mac for Local LLM Inference (2026): The Unified-Memory Buying Guide
  Data: Jun 28, 2026 Two specs decide a Mac for local AI — unified memory (what fits) and memory bandwidth (how fast it decodes). The M-series ladder from Mac mini M4 to Mac Studio M3 Ultra, with first-party and community numbers and honest tradeoffs vs a discrete GPU.
• How to Run LLMs Locally: Which Inference Engine for Your Rig (2026)
  Data: Jun 28, 2026 A decision guide that picks the right local inference engine from your hardware, not hype. llama.cpp for CPU and portability, MLX on Apple Silicon, vLLM for CUDA serving — and why we don't recommend Ollama.
• Hardware to Run a 7B/8B Model Locally: RTX 3090, Apple M3 Max, and Budget Options
  Data: Jun 27, 2026 Benchmark-backed hardware guide for running 7B and 8B parameter models locally. Covers RTX 3090, Apple M3 Max, RTX 3060, and Apple M4 — with first-party Apple M4 benchmarks, community throughput data, VRAM requirements, and honest trade-offs.