Can my GPU run this model?
Enter the model's parameter count, quantization level, and your target context window. When the model card lists KV heads, a sliding window, or a mixture of experts, open Architecture so the cache follows that geometry. The calculator estimates required VRAM (weights + KV cache) and shows which hardware tiers comfortably fit, with links to buy.
Calculation is approximate. Actual VRAM usage varies by runtime, batch size, and system reserved memory. Results include a 15% headroom buffer for compute buffers and runtime overhead. With no architecture filled in, the KV term is a dense full-attention proxy and runs high for GQA, sliding-window, and MoE models.
Inputs
Parameter count × quant × context. Default is 200k — the window long-running agents and coding work actually use. KV cache, not weights, dominates from here up to 1M.
Architecture — GQA, sliding window, MoE
Leave this closed to keep the dense full-attention estimate. Open it when the model card lists KV heads, a sliding window, or a mixture of experts. Head dim is the config value, not hidden size divided by query heads.
Set a model and hit Run.
How the estimate is built
Weight memory is total parameters times a per-quant bytes-per-parameter factor (Q4_K_M uses 0.59). Figures are decimal GB, 109 bytes, so they add to that product. For a mixture of experts, the weight term uses the total parameter count. Active parameters do not shrink the file: every expert is resident.
The KV cache has two paths. With no layer geometry, the historical dense proxy is
(parameters / 7) × (context tokens / 1000) × 0.125 GB. It keeps existing
dense-model results stable, and it assumes full attention on every layer, so it overstates
grouped-query attention, sliding windows, and MoE depth. If you enter active parameters
and nothing else, that proxy is scaled by the active count. That stand-in is not a measured cache.
With layers, KV heads, and head dim, cache bytes are
2 × KV heads × head dim × bytes per element × the sum of cached tokens.
A full-attention layer stores the whole context. A sliding-window layer stores
min(context, window). A hybrid cycle of N local layers then M global layers
repeats through the stack. Head dim is the config value, not hidden size divided by query heads.
Bytes per element default to 2 (fp16 or bf16). The 15% headroom on the sum is where compute
buffers sit. Batch size is 1.
Meta measures Muse Glimmer’s 17 GB Q4 quant plus the vision projector at 131,072 tokens as 19.0 GiB (15.6 GiB text, 1.3 GiB projector, 2.1 GiB KV cache and compute buffers). The text config is 52 layers, 2 KV heads, head dim 128, sliding window 2048, and local×3 + global. The dense proxy reports on the order of 87 GB for a 30B Q4 at that context. The architecture path follows the GQA and hybrid cache instead. Sources: Meta’s quantization note and the Muse Glimmer model card.
Common hardware VRAM reference
Sorted by VRAM tier. Rows other than the RTX 3090 are approximate used-market estimates as of mid-2026. The RTX 3090 row is CompSniper's typical eBay sold range on 2026-10-06 (median $1,499; the sample includes some 3090 Ti). Verify before buying.
| GPU / Hardware | VRAM | BW (GB/s) | Price est. | Best for | Links |
|---|---|---|---|---|---|
| RTX 4060 Ti 8GB | 8 GB | 288 | $280–$370 used | 7B Q4 only | Amazon · eBay used |
| RTX 4060 Ti 16GB | 16 GB | 288 | $400–$480 used | 7B Q8, 13B Q4 (slow BW) | Amazon · eBay used |
| RTX 4070 Ti Super 16GB | 16 GB | 672 | $650–$800 used | 7B Q8, 13B Q4 | Amazon · eBay used |
| AMD RX 7900 XT 20GB | 20 GB | 800 | $450–$600 used | 13B Q8, 34B Q2 (ROCm) | eBay used |
| RTX 3090 24GB ⭑ | 24 GB | 936 | $1,329–$1,599 sold | 7B–34B Q4; 24GB CUDA value vs a used 4090, not vs 128GB boxes | eBay used |
| RTX 4090 24GB | 24 GB | 1008 | $1,350–$1,600 used | 7B–34B, fastest consumer | Amazon · eBay used |
| AMD RX 7900 XTX 24GB | 24 GB | 960 | $650–$850 used | 7B–34B Q4 (ROCm) | Amazon · eBay used |
| Mac Mini M4 Pro 48GB | 48 GB | 273 | ~$1,799 new | 34B Q4–Q8, silent, efficient | Amazon |
| NVIDIA A6000 48GB ⭑ | 48 GB | 768 | $2,000–$3,000 used | 70B Q4 in VRAM, best $/48GB | eBay used |
| RTX 6000 Ada 48GB | 48 GB | 960 | $4,500–$6,000 used | 70B Q4–Q6, fastest 48GB | eBay used |
| Mac Studio M4 Max 128GB | 128 GB | 546 | ~$3,999 new | 70B Q4–Q8, silent | Amazon |
| Mac Studio M4 Ultra 192GB | 192 GB | 1092 | ~$4,999 new | 70B Q8, 100B+ class, long context | Amazon |
| NVIDIA DGX Spark 128GB | 128 GB | — | appliance kit | 200k-class context on 27B–70B; not a 24 GB substitute | Spark vs dual-3090 |
| Dual DGX Spark (two-node) | 256 GB pool | — | two units | 1M-class windows that a single 128 GB box cannot hold | Dual-Spark how-to |
⭑ = top value pick at that tier. Prices verified mid-2026; VRAM prices fluctuate with GPU releases. Always check current sold listings before purchasing.
Links are affiliate links (Amazon Associates · eBay Partner Network). Commission is earned at no extra cost to you. Picks are based on benchmark data, not commission rates.
Quick answers
How much VRAM do I need for Qwen 3.8 27B?
About 17–24 GB at Q4_K_M with a short session (4k–32k). That is a used RTX 3090 decision for short prompts. Agent and coding context (128k–200k) needs 128 GB unified memory or a cluster — the 24 GB card will OOM on KV cache.
Can a 12 GB GPU run an 8B model?
Yes at Q4_K_M with a short context. A used RTX 3060 12GB is the entry pick. Do not buy 12 GB if Qwen 3.8 27B is the goal.
Why does 70B Q4 want 48 GB?
Weights are ~40 GB plus KV cache. No current one-card consumer GPU holds that. Dual used 3090s or a used A6000 are the 48 GB floor; a 32 GB 5090 is still short.
Can I run 200k or 1M context locally?
Yes on enough memory — not on a 24 GB card. Local buyers here are running long agents and coding sessions, so 200k is the default window. A 27B Q4 that fits a used 3090 at 4k will not hold 200k, let alone 1M, without 128GB+ unified memory (Mac Studio, DGX Spark), a multi-GPU pool, or a two-node kit.
Why does a 30B model at 128k sometimes look like 80 GB?
With no architecture filled in, the KV term grows with parameter count and the full context on every layer. Grouped-query attention, a sliding window, and MoE do not work that way. Meta measures Muse Glimmer’s Q4 checkpoint plus the vision projector at 131,072 tokens as 19.0 GiB. The card is 52 layers, 2 KV heads, head dim 128, and a repeating 3 local + 1 global pattern with a 2048 window. Enter those fields and the cache follows the geometry.
Get notified when new benchmarks drop
We publish hardware benchmarks and buying alerts when major models release. Weekly digest, no fluff.