Framework Desktop as a Local AI Server: Strix Halo, ROCm, and the 128GB Question

Price check, 2026-10-06. Framework’s DIY comparison listed the Ryzen AI Max+ 395 with 128GB at starting at $3,449 (pre-order). The ~$2,000–$2,500 band later in this article is the July 2026 figure and is withdrawn. Framework also opened a 192GB Ryzen AI Max PRO 495+ pre-order on 2026-09-30, with shipments starting in November and no dollar price in that post. Dated mini-PC math is on best mini PC for local LLMs.

The Framework Desktop with AMD Strix Halo is the enthusiast’s answer to the question: “What if I could repair my AI server?” It ships with up to 128GB of unified memory, runs native x86-64 Linux, lets you swap CPUs and every component, and costs nothing like a Mac for that capacity. The hype around Strix Halo has been real — Hacker News threads and r/LocalLLaMA are full of first-time builders marveling at something that actually works the way they want it to.

But unified memory is not a magic constant. It is larger than any consumer GPU VRAM, but it is also slower than GDDR6X. The real question underneath the hype is not “can I run a 70B model?” — the community says you can — but “what speed am I trading away, and is it worth it for repairability?”

This guide is for someone considering a Framework Desktop specifically for local LLM inference, and who cares enough about repairability and Linux-native serviceability to accept that tradeoff. If you want the fastest single setup for the least money, this is not that page — the answer is still a used RTX 3090 or a new RTX 4090. But if you are building a machine you want to keep running, understand, and fix yourself for the next five years, Framework changes the constraint logic.

The core constraint: unified memory scales capacity, not bandwidth

Two specs separate the Strix Halo story from every gaming GPU guide you have read.

Unified memory means the CPU and GPU share one memory pool — no copying data between system RAM and VRAM. On a PC with an RTX 4090, the system RAM lives on the motherboard (DDR5, ~100–150 GB/s), the GPU VRAM lives on the card (GDDR6X, ~936 GB/s), and anything you want to compute on the GPU has to move across PCIe. On the Framework, the APU’s CPU cores and GPU cores both read from the same LPDDR5X pool. That is why capacity scales freely: a model that would not fit on any 24GB GPU can simply load into 64GB or 128GB of unified memory.

The caveat, which every honest take on unified memory includes, is bandwidth. LPDDR5X — the memory standard the Strix Halo Max+ uses — offers roughly 200–300 GB/s depending on configuration and thermal state. That is a quarter to a third of RTX 4090 bandwidth. So you can run bigger models, but you will decode slower. This is not a secret; it is the tradeoff you are choosing when you pick unified memory.

For the full picture of how memory bandwidth drives token generation speed, the GPU guide digs into the principle that rules local inference: bandwidth, not FLOPS. The math is the same here. A 70B model at Q8_0 quantization will load into 128GB, and it will run. The decode speed will be lower than the same model on an RTX 4090. Whether that slowness matters depends on your use case — interactive single-user chat does not care about per-token latency the way a production batch workload does.

Strix Halo specs: the three tiers

Framework ships the Desktop with two APU options, each with multiple memory configurations.

• Ryzen AI Max 385 (32GB unified): The base high-end model. 32GB unified memory is enough for a 13B model at Q4_K_M with room for context, a 7B at Q8_0, or multiple smaller models. Lower power draw than the Max+, and the obvious entry point if 70B is not on your roadmap. Price point sits between a used RTX 3090 build and a new M3 Max Mac.
• Ryzen AI Max+ 395 (64GB unified): 64GB opens 32B-class models and larger at lower quantization. Community reports show this tier handles Llama 3.1 70B at Q4_K_M comfortably, and is the inflection point for “bigger model” workloads.
• Ryzen AI Max+ 395 (128GB unified): The flagship capacity tier, and the one the HN threads and r/LocalLLaMA obsess over. 128GB is the first desktop box (not a Mac) where you fit Llama 3.3 70B at Q8_0 — the high-quality quantization — and still have room for a long context window. Community-reported (not independently verified by LocalRig), early users confirm the model loads and runs.

All three tiers are equally repairable and come with the same Framework modularity — you are not buying repairability as a separate tier, you are buying it as a baseline.

The question underneath: ROCm first impressions, bandwidth reality

When the Framework Desktop launched in Q3 2025, the immediate developer enthusiasm met an honest bottleneck: ROCm and llama.cpp on Strix Halo are young. CUDA on NVIDIA cards has been optimized for almost a decade; AMD’s toolchain for consumer APUs is still being shaken out. Community reports from late 2025 through early 2026 paint a picture of something that works, but with friction:

• Performance is slower than equivalent NVIDIA on bandwidth. Community threads (r/LocalLLaMA, 2025–2026) have reported ~30–50 tok/s on Strix Halo for a 7B Q4_K_M against 80–110 tok/s on a used RTX 3090. The 2026-10-06 check does not treat either rate as a planning number: no runtime version is pinned here, and LocalRig did not re-measure them. The bandwidth gap (256 GB/s vs 936 GB/s) is the sourced claim.
• Documentation and examples lag CUDA. If you want to run Ollama or llama.cpp on Strix Halo, the path exists, but the community threads are full of “I had to patch X” or “the ROCm docs do not mention Y.” This is not a blocker for someone comfortable reading source code. It is friction for someone who expects “download and click.”
• Optimization is ongoing. Monthly ROCm releases have been landing improvements specifically for Strix Halo. If you are considering one, the version you buy in 2026 will be noticeably faster than the launch units from 2025 — and faster still in 2027. This is an edge of the platform, not a stable one.

For the full technical take, ServeTheHome and Tom’s Hardware both posted first-impressions deep dives on Strix Halo unified memory and the ROCm ecosystem — worth reading if you are serious about a purchase.

The 128GB case: Llama 3.3 70B and the bandwidth tradeoff

Here is where the 128GB tier stops being theoretical and becomes a concrete use case: running Llama 3.3 70B locally without quantizing it to oblivion.

Community report (r/LocalLLaMA, 2026, not verified by LocalRig): A Strix Halo 128GB holds Llama 3.3 70B at Q8_0 quantization entirely in unified memory, with room for a 4K-token context window. The same threads report ~15–25 tok/s. The 2026-10-06 refresh does not treat that rate as a planning number: the cite has no runtime version, and LocalRig did not re-measure it. Slower than an RTX 4090 on bandwidth, and faster than spilling weights to system RAM, is the qualitative claim that survives.

For comparison: an RTX 4090 cannot hold Llama 3.3 70B at Q8_0. Matching that file takes a larger pool — a multi-GPU build, a Mac, or this 128GB machine. A used 3090 at 24GB is further out on capacity. This page does not restate a tok/s figure for the 4090.

The honest caveat is the speed-bandwidth relationship. AMD documents 256 GB/s for this platform, against 936 GB/s for an RTX 4090 in LocalRig’s GPU catalog. If decode speed on a model a 24GB GPU already holds is the constraint, Strix Halo is the wrong machine. This page does not assign a tok/s target to that choice.

Comparison: Strix Halo vs the homelab alternatives

Where does Strix Halo sit against other local-inference paths? This is not a strict ranking — the winner depends on your constraint.

System	VRAM	~7B tok/s	Repairability	Price range	Best for
Framework Desktop Strix Halo 128GB	128 GB unified	not re-measured	Full modular repair	Starting at $3,449 DIY, 2026-10-06	70B models, serviceability, x86 Linux
RTX 4090 build	24 GB GDDR6X	not re-measured	GPU swap only	not re-priced	Maximum single-card speed
Used RTX 3090	24 GB GDDR6X	not re-measured	GPU swap only	not re-priced	Budget VRAM-per-dollar champion
Mac M3 Max 128GB	128 GB unified	not re-measured	Not user-serviceable	not re-priced	Maximum unified memory, Apple ecosystem
2× RTX 3090 (48GB)	48 GB GDDR6X	not re-measured	GPU swap only	not re-priced	32B/70B capacity, no repairability

Tok/s cells in this table were not re-measured on 2026-10-06. The Framework row’s old ~30–50 band is withdrawn as a planning number. Other rows’ prices and rates were not re-read.

Who this is for (and who it is not)

You should consider Strix Halo if:

• You want to run models larger than 24GB — 32B or 70B class — locally without a Mac.
• Repairability and modularity matter to you more than maximum speed.
• You are comfortable reading GitHub issues and occasional ROCm documentation when something is not working.
• You want true x86-64 Linux support and full control over your inference stack (not locked to Apple’s decisions).
• You can accept lower token speed than an RTX 4090 for models that only fit on Strix Halo or a Mac. This page does not quantify that gap.

You should not buy Strix Halo if:

• You need raw inference speed for a 7B or 13B model — a used RTX 3090 or new RTX 4090 is cheaper and faster.
• You are not comfortable with “first-generation” developer tooling and occasional friction in the software stack.
• You want “set it and forget it” — NVIDIA CUDA on LLMs is far more polished.
• The 128GB DIY price (starting at $3,449 on 2026-10-06) rules you out. The old $2,000–$2,500 band does not.

The bottom line: capacity with an asterisk on speed

The Framework Desktop with Strix Halo 128GB is the first x86-based, user-repairable machine that can load and run Llama 3.3 70B locally. That is real, and it is not trivial — it means local AI inference without being locked into Apple’s product lifecycle or sacrificing the ability to open your machine and fix it.

The asterisk is honest: unified memory is not GDDR6X, and you pay for that in token generation speed. AMD’s 256 GB/s against an RTX 4090’s 936 GB/s is the bandwidth gap. This page does not turn that gap into a tok/s ratio. The 4090 still cannot hold a 70B Q8 file. Slowness relative to a card that fits the model is the physics of bandwidth.

For someone building a homelab inference server that has to last five years, be debuggable when something breaks, and run models that only fit on high-capacity unified-memory systems, Strix Halo is becoming the obvious choice. For someone whose constraint is speed on 7B–13B models, or whose budget is tight, a used RTX 3090 or exploring Mac Studio or M3 Max makes more sense.

ROCm is getting better every release. The performance numbers will improve. Serviceability and modularity will not change — that is a Framework promise. If you buy one, you are betting on AMD’s toolchain improving while you use a machine you can actually repair.

---

Further reading

For related decision-making:

• Best GPU for Local LLM Inference — the VRAM and bandwidth principles that apply to Strix Halo too
• Best Mini PC for Local LLM — where Framework Desktop fits in the homelab lineup
• Strix Halo vs Mac Studio — direct comparison of the two repairable unified-memory options
• AMD 7900 XTX for Local LLM — exploring discrete AMD GPUs as an alternative path

For sizing your model before you buy:

• What Is Quantization — how to figure out if a model will fit at your desired quality
• Hardware to Run a 70B Model Locally — the full capacity math
• The Local AI Hardware Buying Framework — decision logic for every constraint

For getting started after you buy:

• How to Run LLMs Locally — Ollama, llama.cpp, and the ROCm path specifically

Framework Desktop direct (no affiliate program, for transparency): framework.com/marketplace/products/framework-laptop-13-strix-halo-ddr5-system

For upgrading or expanding the build:

• RAM and SSD on Amazon · eBay search
• Storage and accessories on Amazon · eBay search