Can I Run Muse Glimmer 30B Locally? 24GB Floor, DFlash, and the Agent Tax

Short answer: Meta’s K-Quant-17GB checkpoint plus the vision projector measures 19.0 GiB at a 131,072-token context (quantization doc, accessed 2026-10-06). A 24 GB GPU is that cell. The DFlash drafter is about 1.5 GiB more on Meta’s own add. A 32 GB Mac is Unsloth’s 4-bit example, not a measured 131k peak. A 128 GB unified box has room for the BF16 weights. LocalRig has not measured tok/s on any of these machines.

meta-models/Muse-Glimmer-30B is a dense causal transformer with a perception encoder (model card, August 2026, Apache-2.0). The card says ~29.6B parameters including the vision encoder. The Hub safetensors block, read 2026-10-06, counts 29,776,626,688 BF16 parameters. Those two figures are both on the record. This page does not average them. Context on the card and in the GGUF metadata is 131,072. Output is text. Input is text and image. Audio is out of scope on the card.

This is a constraint selector. Qwen 3.8 27B stays the LocalRig 24 GB dense default. Muse Glimmer is a different download, with a vision projector and an optional speculative drafter in the memory budget. The open-weight MoE in the same series is Muse Spark Open. Different parameter count, different memory class. This page stays on the 30B dense model.

What memory does each quant actually need?

Answer from published files and one vendor VRAM trace. Shorter contexts are not yet measured.

Meta’s quantization page gives one operating point: K-Quant-17GB, vision projector loaded, context 131,072.

Component	VRAM (Meta)
Text model	15.6 GiB
Vision projector	1.3 GiB
KV cache and compute buffers	2.1 GiB
Total	19.0 GiB

Meta says that fits a 24 GiB card with headroom, and to add roughly 1.5 GiB for the DFlash drafter. The 1.5 GiB line is Meta’s allowance, not a second trace that LocalRig re-ran. The 2.1 GiB line mixes KV and compute buffers, so it cannot be scaled to 4,096 or 32,768 tokens. Peaks at those windows are not yet measured.

Disk sizes from the Hub tree meta-models/Muse-Glimmer-30B-GGUF (bytes read 2026-10-06). Decimal GB is bytes / 10^9, which is how Meta’s “16.8 GB” download figure rounds.

File	Bytes	Decimal GB	Role
Muse-Glimmer-30B-KQuant-17GB-Q4_K_M.gguf	16,756,683,904	16.8 GB	Text, fixed K-quant. Meta’s 24 GB target
Muse-Glimmer-30B-KQuant-Dynamic-Q4_K_XL.gguf	19,653,960,832	19.7 GB	Text, dynamic K-quant. Card target 32 GB VRAM
mmproj-Muse-Glimmer-30B-Q4_K_M.gguf	1,400,328,928	1.4 GB	Vision projector
dflash-Muse-Glimmer-30B-Q4_K_M.gguf	1,631,208,128	1.6 GB	Optional DFlash drafter

The model card’s hardware targets, separate from that 19.0 GiB trace: K-Quant-17GB → 24 GB VRAM (1.0% average degradation on 15 benchmarks), K-Quant-Dynamic → 32 GB VRAM (0.2%), full precision → 64 GB VRAM. Those percentages are Meta’s quality claim, not a memory measurement. No 32 GB peak for the dynamic quant is published next to the 19.0 GiB row. Treat “32 GB VRAM” as the vendor target, and leave the dynamic 131k peak not yet measured.

BF16 weights are not a GGUF. Meta’s get-the-model page says the safetensors repo is around 60 GB. Hub usedStorage for meta-models/Muse-Glimmer-30B was 59,581,565,169 bytes on 2026-10-06. vLLM’s Muse page says the BF16 weights are roughly 60 GB and that serving wants 72 GB of VRAM once KV, activations, and CUDA overhead are in.

Unsloth’s how-to (accessed 2026-10-06) is a different quant family. Their recommended total memory (RAM + VRAM, or unified memory), context length not stated:

Unsloth band	Their total-memory figure	Their hardware example
2-bit UD-Q2_K_XL	12–14+ GB	RTX 4080
3-bit UD-Q3_K_XL	14–15 GB+	RTX 4090
4-bit UD-Q4_K_XL	17 GB+	Mac 32 GB
6-bit UD-Q6_K_XL	20–22 GB+	RTX 5090, Mac 48 GB
8-bit UD-Q8_K_XL	34 GB+	Mac 128 GB, DGX Spark
BF16	58 GB+	Mac 128 GB, DGX Spark

Hub file sizes for those Unsloth GGUFs the same day (bytes / 10^9): UD-Q2_K_XL 12.4 GB, UD-Q3_K_XL 13.4 GB, UD-Q4_K_XL 15.9 GB, UD-Q6_K_XL 26.3 GB, UD-Q8_K_XL 32.3 GB, Q8_0 29.6 GB. mmproj-kquant.gguf is 1.4 GB. mmproj-Muse-Glimmer-30B-BF16.gguf is 3.8 GB. dflash-kquant.gguf is 1.6 GB. A 15.9 GB file and a “17 GB+” recommendation are Unsloth’s headroom, not two different 4-bit recipes. Do not paste Unsloth’s 17 GB onto Meta’s 19.0 GiB row. They are different checkpoints.

Ollama’s tag page (accessed 2026-10-06; the page says “1 month ago,” with no absolute date) lists distribution size, 128K context, text and image input. That is not peak VRAM.

Tag	Listed size	Digest
muse-glimmer:30b / 30b-q4_K_M / latest	18 GB	de878ce33ad8
30b-q4_K_M-dflash	20 GB	da643f4663ab
30b-q8_0	31 GB	6ffafcdbd728
30b-q8_0-dflash	33 GB	51675ea1db04
30b-bf16	57 GB	318fb1b9b816
30b-bf16-dflash	59 GB	bd22c6664845
30b-mlx	19 GB	015fa21845be
30b-nvfp4-dflash	19 GB	015fa21845be (same digest)
30b-mlx-bf16	60 GB	43511b65122d
30b-mlx-bf16-dflash	65 GB	db7968c9dcb3
30b-mxfp8	33 GB	783458ca3191
30b-mxfp8-dflash	35 GB	14bd0cb8d43f
30b-nvfp4	17 GB	6a78d632287f

30b-mlx and 30b-nvfp4-dflash sharing digest 015fa21845be matches open issue ollama/ollama#17656 (jacob-bd, 2026-08-10, still open on 2026-10-06): on Ollama 0.32.7, ollama show muse-glimmer:30b-mlx reported quantization nvfp4, and the manifest layers pointed at 30b-nvfp4-dflash.

Why the VRAM calculator disagrees

The calculator in site/src/lib/vram.ts is a dense estimator. Weights = params_B × bytes_per_param. KV = (params_B / 7) × (context_tokens / 1000) × 0.125 GB. Recommended = ceil((weights + KV) × 1.15). Q4_K_M uses 0.59 bytes/param, Q8_0 uses 1.06, FP16 uses 2.00.

Run on the Hub count 29.776626688B (the safetensors total, which already includes the vision encoder). These are estimates. They assume every layer keeps a full-length KV cache.

Quant	Weights	KV @ 4,096	KV @ 32,768	KV @ 131,072	Total @ 131,072	Recommended @ 131,072
Q4_K_M	17.6 GB	2.2 GB	17.4 GB	69.7 GB	87.3 GB	101 GB
Q8_0	31.6 GB	2.2 GB	17.4 GB	69.7 GB	101.3 GB	117 GB
FP16	59.6 GB	2.2 GB	17.4 GB	69.7 GB	129.2 GB	149 GB

Meta’s measured stack at the same 131,072-token window is 19.0 GiB, not 87 GB. The card’s attention pattern is [Local, Local, Local, Global] repeating, 52 layers, sliding window 2048, 2 KV heads. Meta’s quantization page says only a quarter of the layers hold a full-length cache, which is why the KV stays small. The calculator does not know that pattern. Use it to see the dense upper bound. Buy from the 19.0 GiB row. Confirm the formula on the VRAM calculator. There is no Muse Glimmer preset in the instrument, on purpose: a 30B dropdown would print the dense number.

4,096-token and 32,768-token peaks for the real GGUF are not yet measured. A shorter -c reduces KV on llama.cpp (Meta’s llama.cpp guide). How many gigabytes that saves on this model is an empty cell.

Which machines fit?

Machine	K-Quant-17GB + vision @ 131,072	With DFlash (~1.5 GiB, vendor add)	Unsloth bands (context unstated)	BF16	Runtime to pin
16 GB GPU	No. 19.0 GiB is over the card	No	2-bit 12–14+ GB and 3-bit 14–15 GB+ are the bands under 16 GB. 4-bit is 17 GB+	No	llama.cpp b10353+ only if you accept Unsloth’s smaller band. Peak at 131k not yet measured
24 GB GPU (used 3090 / 4090)	Yes. 19.0 GiB vendor measurement	19.0 + 1.5 = 20.5 GiB on that arithmetic, still under 24	4-bit 17 GB+ clears 24 as a number. Unsloth’s example for 4-bit is a 32 GB Mac. 6-bit is 20–22 GB+; their example is a 5090 or a 48 GB Mac	No	llama.cpp b10353 (2026-08-10) or newer. Ollama library tag from v0.32.8
32 GB GPU (RTX 5090 class)	Yes	Yes on the same arithmetic	4-bit and 6-bit bands	No. Serving BF16 is the 72 GB vLLM note	llama.cpp. Meta’s speed row is this GPU class. vLLM BF16 is a different budget
32 GB Mac, unified	Weights fit on paper. OS share not yet measured	Same gap	Unsloth’s listed 4-bit example (17 GB+)	No. Tags are 57–65 GB	llama.cpp Metal, build b10353+. Ollama 30b-mlx is the advertised tag; see the open issues below
64 GB Mac, unified	Yes on capacity	Yes on capacity	4-bit and 6-bit	Card target is 64 GB VRAM. Ollama 30b-bf16 is 57 GB and 30b-mlx-bf16 is 60 GB. OS plus KV not yet measured	llama.cpp for the GGUF. Do not treat the 60 GB tag as a finished 64 GB fit
128 GB unified (Mac or mini PC)	Yes on capacity	Yes on capacity	Unsloth’s 8-bit (34 GB+) and BF16 (58 GB+) examples name a 128 GB Mac and DGX Spark	Yes on Unsloth’s 58 GB+ band and on the ~60 GB safetensors	llama.cpp. tok/s not yet measured. vLLM’s published image is NVIDIA CUDA and asks for 72 GB at BF16

Parent sizing: hardware for a 32B-class model, VRAM tiers, how much unified memory, mini PCs.

The 24 GB buy, if the job is Meta’s K-Quant-17GB with vision at the full 131k window, is a used RTX 3090. Browse used RTX 3090 24GB on eBay →. Vet the card: used RTX 3090 buying guide. A 5090 is the GPU in Meta’s speed table. It is not required to clear 19.0 GiB.

Try the same GGUF on a rented 24 GB card before buying: RunPod, Vast.ai, GPUMart. Those links are affiliate-tagged. They do not change the fit.

Which runtime, and which version?

NVIDIA, one consumer GPU: llama.cpp build b10353 or newer. Meta’s llama.cpp guide says Muse Glimmer support first shipped in release b10353 (GitHub release, published 2026-08-10). Older builds fail with unknown model architecture: 'muse-glimmer'. The guide names commit 62bf73d25, PR #26841, for the architecture, the vision projector, the ATEM tool parser, and DFlash. Pass --jinja. Name the checkpoint with --hf-file, because three GGUFs in the repo carry a Q4_K_M tag and a bare repo pull can grab the wrong one. Drop --mmproj for text-only. DFlash is -md plus --spec-type draft-dflash. Meta says a [spec] failed to measure draft model memory warning at startup still loads. Sampling on the card: temperature 1.0, top_p 0.95, top_k 64. Setup context for a 3090: llama.cpp on an RTX 3090. Picker: Ollama vs llama.cpp vs vLLM and how to run LLMs locally.

Apple Silicon GGUF: the same llama.cpp build, Metal. Meta’s guide documents -DGGML_METAL=ON. That is the path with a pinned build number.

Ollama: v0.32.7 announced Apple MLX, and v0.32.8 the same day said every platform. v0.32.7 (published 2026-08-10) limits the first note to the MLX engine on Apple Silicon, with DFlash and image input, via ollama run muse-glimmer:30b-mlx, and says NVIDIA, AMD, and other platforms were “coming days.” v0.32.8 (published 2026-08-10, later the same day) says Muse Glimmer is available on all platforms, changelog line “Add Muse Glimmer support for NVIDIA, AMD, and additional platforms,” and the run command is ollama run muse-glimmer. Follow-ups that still name the model: v0.32.9 (2026-08-11) a function-calling parser boundary fix; v0.32.10 (2026-08-12) an Ollama claim of about 7–8% faster NVFP4 MLX prefill on Muse Glimmer, with no hardware, quant file, or absolute tok/s in the note; v0.32.11 (2026-08-14) template updates and ollama launch muse. In the 20 release notes fetched 2026-10-06, v0.32.12 (2026-08-14) through v0.35.1 (2026-09-29) and v0.40.0-rc5 (2026-09-25) do not mention Muse Glimmer. None of those notes is a 3090 tok/s packet.

Two open reports, both still open when fetched 2026-10-06:

• ollama#17656, jacob-bd, 2026-08-10, Ollama 0.32.7, M3 Max 64 GB. muse-glimmer:30b-mlx showed quantization nvfp4. Reported decode about 12 tok/s, against about 88 tok/s for qwen3-coder 30.5B Q4_K_M on that same Mac. Community bug report. Not a LocalRig MLX speed.
• ollama#18269, Creence, 2026-09-06, Ollama 0.33.3, MacBook Air M4, 32 GB, digest 015fa21845be. The tag stuck in Stopping.... The reporter also saw a reboot and said they could not confirm the model caused it. Maintainer comment (jessegross, 2026-09-14): stopping usually means a runaway generation, and the memory side “should be fixed and available in the next release.” The issue was still open on 2026-10-06.

Ollama’s MLX backend is the general Mac page. It is not a Muse measurement. Until those issues close, the honest Apple path for a reproducible quant is llama.cpp Metal on the published GGUF, not the 30b-mlx tag.

Ollama importing a Hugging Face GGUF is a separate failure. ollama#18808 (rbrisita, 2026-10-05, Ollama 0.35.0, Linux, Intel CPU, NVIDIA GPU) reports ollama run hf.co/unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL producing no usable reply. A follow-up the same day shows a Modelfile rewrite that then answers. That is the HF import path. It is not a measurement of the library tag muse-glimmer:30b.

vLLM: pin v0.28.0 or newer for the release that lists the model, and read Meta’s 72 GB line before you try a 24 GB card. vLLM v0.28.0 (published 2026-08-26) release notes list Muse Glimmer as a new model via PR #51655. That PR merged 2026-08-14. The merge commit is an ancestor of the v0.28.0 tag. Current release when checked 2026-10-06 was v0.31.0 (published 2026-10-05). Meta’s vLLM page, accessed the same day, still describes #51655 as unmerged, still says pip install vllm will not serve the model, and still points at the image vllm/vllm-openai:muse-glimmer. Both statements are in the sources. The page asks for 72 GB of VRAM, CUDA 13.0, and says there is no published ROCm image. A 24 GB card is the llama.cpp cell, not this BF16 serve.

ExecuTorch is the runtime in Meta’s Mac speed table (muse-glimmer-mlx preset on the ExecuTorch page). That page does not pin an ExecuTorch semver. LocalRig will not invent one.

What did Meta measure for speed, and what is empty?

Vendor table from the model card. K-Quant-17GB plus the quantized DFlash drafter. Batch size 1, greedy. M4 Max and M5 Max via ExecuTorch. RTX 5090 via llama.cpp. Memory size, context length, driver, and OS are not stated. This is Meta’s table. It is not a LocalRig row and not a community row.

Machine (as labeled)	Baseline, no speculation	With DFlash	Speedup
NVIDIA RTX 5090	74.9 tok/s	233.4 tok/s	3.1×
Apple M4 Max	23.7 tok/s	37.8 tok/s	1.5×
Apple M5 Max	26.6 tok/s	50.2 tok/s	1.8×

Not yet measured: tok/s on a used RTX 3090, an RTX 4090, a 32 GB Mac, a 64 GB Mac, and a 128 GB mini PC. Do not copy the 5090 row onto a 3090. Do not copy the M4 Max row onto a 32 GB MacBook Air. The only Mac decode figure in this pass with a named machine is the ~12 tok/s in ollama#17656, and that report says the tag was NVFP4 on an M3 Max 64 GB, not a healthy MLX run.

Images add prompt tokens. The card caps visual tokens at 4,096 per image. The 1.3 GiB projector is residency, not a per-screenshot surcharge. A GGUF loaded without mmproj is text-only even though the base model is multimodal.

DFlash on the card: 5 draft layers, block size 16, sliding window 2048 on every draft layer, sequence length 131,072. It does not change the text the main model accepts. It changes speed, in Meta’s table, and it adds the drafter’s memory.

License and who may run it

Weights are Apache-2.0 on the model card and on the GGUF repo. A separate usage policy also applies. It says Muse Glimmer is not intended for individuals under 18, and it lists prohibited uses (illegal activity, child exploitation, weapons, malware, fraud, and others). The card’s intended use is commercial and research, including local agents, coding agents, and tool calls. Audio input and output are out of scope. Video is frames, not a native video model. Age and prohibited-use terms are policy, not a VRAM footnote. Read the policy before you ship an agent that can take actions.

Who this is NOT for

• A 16 GB card that must hold Meta’s K-Quant-17GB stack with vision at 131,072 tokens. The measured total is 19.0 GiB. Unsloth’s 2-bit and 3-bit bands are a different quality trade, and their 131k peak is not published.
• Anyone buying from the dense calculator. 101 GB recommended at Q4 and 131k is the generic estimator. Meta measured 19.0 GiB for this architecture at that window.
• Anyone who wants a LocalRig tok/s on a 3090, a 32 GB Mac, or a 128 GB mini PC. Those cells are empty.
• Anyone treating muse-glimmer:30b-mlx as a proven MLX quant. The tag shares a digest with 30b-nvfp4-dflash, and ollama#17656 and #18269 were still open on 2026-10-06.
• A 24 GB card aimed at vLLM BF16. Meta’s serve note asks for 72 GB. Quantized llama.cpp is the 24 GB path.
• Training. This page is inference fit. Unsloth’s fine-tune notebooks are a separate claim and a separate memory budget.

Methodology

• SKU: meta-models/Muse-Glimmer-30B. Card: ~29.6B including vision, Apache-2.0, August 2026, context 131,072, text+image in, text out. Hub safetensors: 29,776,626,688 BF16 parameters (API, 2026-10-06). Both figures kept.
• Fit: Meta quantization doc, one measured point (19.0 GiB at 131,072 with vision). File bytes from the Hub tree the same day. Unsloth bands from their how-to, context unstated. Ollama sizes from the tags page, distribution only.
• Estimates: calculateVram in site/src/lib/vram.ts, params = 29.776626688, marked estimate. Dense KV. Not the buy number.
• Speed: Meta model-card table, vendor, batch 1, greedy, runtime split ExecuTorch vs llama.cpp, memory config unstated. Community: ollama#17656 (~12 tok/s, reported NVFP4 path). No LocalRig row. 3090, 32 GB Mac, 64 GB Mac, and 128 GB mini PC tok/s are not yet measured.
• Runtimes pinned: llama.cpp b10353 (2026-08-10). Ollama v0.32.7 (Apple MLX announcement) and v0.32.8 (NVIDIA/AMD/all-platforms, both published 2026-08-10). Library tags re-read 2026-10-06. vLLM v0.28.0 (2026-08-26) first release note; v0.31.0 current on 2026-10-05. Meta’s vLLM page still disagrees about the unmerged PR.

Sources

• meta-models/Muse-Glimmer-30B — model card, architecture, 24/32/64 GB targets, vendor tok/s table. Accessed 2026-10-06.
• Usage policy — Apache-2.0 weights plus this policy. Accessed 2026-10-06.
• Quantization — 16.8 / 19.7 / 1.4 / 1.6 GB downloads; 19.0 GiB measured at 131,072; ~1.5 GiB drafter. Accessed 2026-10-06.
• Deploy with llama.cpp — build b10353, PR #26841. Accessed 2026-10-06.
• llama.cpp b10353 — published 2026-08-10.
• Get the model — BF16 repo around 60 GB. Accessed 2026-10-06.
• Deploy with vLLM — 72 GB serve, image vllm/vllm-openai:muse-glimmer, page still calls #51655 unmerged. Accessed 2026-10-06.
• vLLM PR #51655 — merged 2026-08-14. v0.28.0 notes list the model.
• Deploy with ExecuTorch — CUDA and MLX presets; no semver on the page. Accessed 2026-10-06.
• Muse-Glimmer-30B-GGUF — file bytes. Accessed 2026-10-06.
• unsloth/Muse-Glimmer-30B-GGUF — file bytes, repo last modified 2026-08-10. How to run, accessed 2026-10-06.
• Ollama tags and library page. Accessed 2026-10-06.
• Ollama v0.32.7 — published 2026-08-10. MLX announcement.
• Ollama v0.32.8 — published 2026-08-10. NVIDIA, AMD, and other platforms.
• Ollama v0.32.9, v0.32.10, v0.32.11 — parser fix, NVFP4 MLX prefill claim, template updates.
• ollama#17656, ollama#18269, ollama#18808.
• VRAM calculator — dense estimator this page refuses as the buy number.