Correction, 18 October 2026. This explainer was first published in March 2026 and has been rewritten. The earlier version built its examples on a “Qwen 3.5 30B-A3B”, which does not exist (Qwen 3 30B-A3B and Qwen 3.5 35B-A3B do), and described it as routing 2 of 64 experts (Qwen 3 30B-A3B uses 8 of 128). It put a dense 70B at about 14 tok/s on an M4 Max (the index estimates about 12; the bandwidth ceiling is about 14) and said a dense 30B model needs 64 GB (about 18 GB at 4-bit). Figures are now from the LLMCheck index, each labelled.
How a dense model works
In a dense model, every parameter takes part in every token. Qwen3.8-27B reads all 27.8B of its weights to produce each word, and so does Llama 3.3 70B with its 70B. That has two consequences on a Mac:
- Memory: all the weights must fit in unified memory. At 4-bit that is roughly 0.57 GB per billion parameters: about 16–17 GB for a 27–30B model, about 40 GB for a 70B.
- Speed: decode cannot run faster than memory bandwidth divided by the bytes read per token. A 4-bit 70B on an M4 Max (546 GB/s) has a ceiling of about 14 tok/s; the index estimates estimated~12 tok/s. On an M5 Max, tbreak’s Mac Studio review published 13.1 tok/s (4-bit, MLX).
Dense models are simple and predictable, and per gigabyte they are usually the most capable. Their speed problem is that capability grows with size, and every extra parameter is read for every token.
How a mixture-of-experts model works
A mixture-of-experts model replaces most of each layer’s feed-forward block with many small networks, the experts. A small router picks a few of them for each token; the rest sit idle for that token. Attention layers and any “shared” expert run every time. Two current examples, from their model cards:
- Qwen 3.6-35B-A3B: 35B parameters, 256 experts per MoE layer, 8 routed plus 1 shared expert per token, about 3B active parameters (model card).
- Gemma 4 26B-A4B: 25.2B parameters, 128 experts with 8 active plus 1 shared, 3.8B active (model card).
The name usually encodes both numbers: “35B-A3B” means 35B total, 3B active. The router works per token and per layer, so one sentence can pass through many different experts. Experts are not tidy subject specialists (“the biology expert”); they are learned partitions that rarely map onto human topics.
Same Mac, several times the speed
Because decode reads only the active weights, an MoE model with 3–4B active parameters decodes like a 3–4B model while holding the knowledge of a much larger one. The cleanest Mac comparison comes from Google’s own pair: one community benchmark repository ran both Gemma 4 models on the same 32 GB Mac mini M6 in LM Studio with Google’s QAT builds, and measured 48.2 tok/s for the 26B-A4B MoE against 7.5 for the dense 31B (English prose prompt, median of three runs). That is about six times faster, from the same family, on the same machine.
| Model | Design | Total / active | Index min. RAM | Mac mini M6 32 GB (170 GB/s) | M5 Max (614 GB/s) |
|---|---|---|---|---|---|
| Gemma 4 26B-A4B | MoE | 25.2B / 3.8B | 18 GB | community48 tok/s | sourced107 tok/s |
| Gemma 4 31B | dense | 30.7B | 24 GB | community8 tok/s | estimated~26 tok/s |
| Qwen 3.6-35B-A3B | MoE | 35B / 3B | 24 GB | community64 tok/s | sourced118 tok/s |
| Qwen3.8-27B | dense | 27.8B | 19 GB | community9 tok/s | estimated~29 tok/s |
| Qwen 3.5 122B-A10B | MoE | 122B / 10B | 65 GB | doesn’t fit | community43 tok/s |
| Llama 3.3 70B | dense | 70B | 44 GB | doesn’t fit | sourced13 tok/s |
The Qwen pair shows the same gap with a caveat: its two M6 figures come from different sources and runtimes (MLX for the MoE model; LM Studio and MLX runs for the dense one). The M5 Max column mixes published and estimated figures; Liquid AI’s same-setup llama.cpp table (Q4_K_M, 4K-token input) supplies the Qwen 3.6 and Gemma 4 MoE figures. A 122B MoE model with 10B active decodes several times faster on an M5 Max than a dense 70B that is smaller on disk.
What MoE does not fix
- It does not save memory. Every expert must be loaded, because any of them can be picked for the next token. Qwen 3.6-35B-A3B needs a 24 GB Mac although it reads about 3B parameters per token. Memory decides what fits; active parameters decide how fast it runs. See how much RAM you need.
- It is not smarter per gigabyte. On the index’s capability score, the dense Qwen3.8-27B scores 46 out of 50 against 38 for Qwen 3.6-35B-A3B, despite being the smaller file. When quality matters more than speed, a dense model of the same size usually wins.
- Small MoE models stop scaling with bandwidth. With only 2–4 GB of weights read per token, routing and kernel overhead become a large share of each token’s time. The index models this as a fixed overhead of about 5 ms per token, fitted to six fully specified Mac runs (methodology), so a small MoE model gains less from an M5 Ultra than a dense one does.
- Large MoE estimates are usually, not always, floors. For models above 40B total the index applies a conservative factor; measured runs landed above it for 10 of 12 models and below it for 2. Qwen3.8-Flash-Next ran at 57.0 tok/s community-measured on an M5 Ultra, against an estimate of about 85. Where a model has measured rows, its other chips are scaled from those.
Which to choose for your Mac
- 16 GB: mostly dense 8–12B models (Gemma 4 12B is the index’s pick), or OpenAI’s GPT-oss 20B, a 21B MoE with 3.6B active that fits in about 12 GB. See the best LLM for a 16 GB Mac.
- 24–64 GB: Qwen 3.6-35B-A3B for speed (the index’s #1, and its pick at 24 and 32 GB), Qwen3.8-27B when you want the strongest dense model and can wait for it. At 64 GB the index picks LFM2.5-24B-A2B, another small MoE (24B, about 2B active), which runs faster still.
- 128 GB and up: large MoE models are the only practical way to run 100B-plus models at reading speed: Qwen3.8-Flash-Next (125B, 6B active) on 128 GB, MiMo-V2.6-Flash (309B, 15B active) on 192 GB or more.
MoE and dense models in the index
Reference figures at 4-bit unless noted. The leaderboard’s reference chip is the M5 Max, or the M5 Ultra for models that need more than 128 GB.
| Model | Design | Total / active | Index min. RAM | Reference figure |
|---|---|---|---|---|
| GPT-oss 20B | MoE | 21B / 3.6B | 13 GB | sourced108 tok/s on an M5 Max |
| Gemma 4 12B | dense | 12B | 9 GB | estimated~64 tok/s on an M5 Max |
| Qwen3-Coder-Next | MoE | 80B / 3B | 48 GB | estimated~84 tok/s on an M5 Max |
| GPT-oss 120B | MoE | 117B / 5.1B | 62 GB | sourced76 tok/s on an M5 Max |
| Qwen3.8-Flash-Next | MoE | 125B / 6B | 120 GB | community46 tok/s on an M5 Max |
| DeepSeek V4 Flash | MoE | 284B / 13B | 120 GB | community39 tok/s on an M5 Max, 2-bit |
| MiMo-V2.6-Flash | MoE | 309B / 15B | 170 GB | community62 tok/s on an M5 Ultra |
| Kimi K2.6 | MoE | 1T / 32B | 429 GB | estimated~27 tok/s on an M5 Ultra, about 3-bit |
The leaderboard lists every model with its design, and each Mac’s ranking is under Best LLM by Mac.
Frequently Asked Questions
What is a mixture-of-experts (MoE) LLM?
A model whose layers contain many small expert networks, of which a router picks a few for each token. Qwen 3.6-35B-A3B, for example, has 256 experts per layer and uses 8 routed plus 1 shared expert per token, about 3B of its 35B parameters. It needs memory for every expert but reads only the active ones for each token.
Is MoE better than dense for local AI on a Mac?
Faster, not always smarter. On the same 32 GB Mac mini M6, Gemma 4 26B-A4B (MoE) was community-measured at about 48 tok/s against about 7.5 for the dense Gemma 4 31B. On the LLMCheck index's capability score, though, the dense Qwen3.8-27B scores 46 out of 50 against 38 for the MoE Qwen 3.6-35B-A3B. Choose MoE for speed, dense for quality per gigabyte.
Do MoE models need less RAM?
No. All experts must be in memory, because any of them can be chosen for the next token, so RAM depends on total parameters. Qwen 3.6-35B-A3B is about 20 GB at 4-bit and needs a 24 GB Mac, although it reads only about 3B parameters per token.
What does 'active parameters' mean?
The parameters used to produce one token: the attention layers, any shared expert and the experts the router picks. In a name like 35B-A3B, the A number is the active count. On a Mac, decode speed depends mainly on the active parameters, and memory on the total.
How fast is a dense 70B model on a Mac?
About 13 tok/s at 4-bit on an M5 Max: tbreak's Mac Studio review published 13.1 tok/s for Llama 3.3 70B in MLX. The LLMCheck index estimates about 12 on an M4 Max. The ceiling is memory bandwidth divided by roughly 40 GB of weights, about 15 tok/s on an M5 Max.
Why don't small MoE models get much faster on bigger chips?
Because they read so little per token that fixed costs, such as routing and launching GPU kernels, take a large share of each token's time. The LLMCheck index models this as about 5 ms of overhead per token, so a 3B-active model gains less from doubling bandwidth than a dense model does.