The LLMCheck index estimates Nemotron 3.5 Bolt 74B-A9B at ~31 tok/s on an M5 Pro at 4-bit, scaled from its community-measured M5 Max run with the published bandwidth model, not a measured run on this chip. It needs about 45 GB of unified memory. It is NVIDIA's 74B mixture-of-experts model (9B active), released under the OpenMDW licence.
Stock llama.cpp runs it from b11755, but NVIDIA's 24 GB GGUF falls back to the CPU on its NVFP4 layers (use a Q4_K_M file); Ollama does not support it on Apple Silicon (PR #19229) and stock mlx-lm cannot load it.
This is an LLMCheck index estimate from its published two-term bandwidth model, not a measured run; the index labels it so everywhere it appears. Own this Mac? Submit a real run →
| Metric | Value |
|---|---|
| Tokens per second | estimated~31 tok/s |
| Provenance | estimate · how estimates are made |
| Quantization | 4-bit |
| Engine | MLX |
| Figure dated | 2026-10 |
| Minimum RAM | 45 GB |
| Parameters | 74B MoE (9B active) |
| Chip bandwidth | 307 GB/s |
Each row as the index holds it, with its source. “Mac RAM” is the memory of the Mac a run used, not the model’s requirement (45 GB). The headline figure follows the reference rule: the lower-median measured plain-decode run at 5 bits or fewer (a run whose quant was not stated counts only when no stated-quant run qualifies), else the estimate; a run marked “not used to anchor estimates” is listed here but never becomes the headline.
| Speed | Quant | Engine | Mac RAM | Context | Source | Note |
|---|---|---|---|---|---|---|
| community39 tok/s | IQ2_XXS | MLX | 48 GB | 512-token coding answers, median of 5 at T 0.6 | github.com | MacBook Pro, 20-core GPU. oMLX 0.7.1rc1 (no drafter) loading NVIDIA's 24 GB checkpoint through the bundle's model file, so it does not anchor; greedy 39.8; 30.3 GB peak. Run by the Kapitsa engine's developers with their own harness.; not used to anchor estimates |
| community24.6 tok/s | IQ2_XXS + Q8_0 | llama.cpp | 64 GB | random-text prompts, greedy, -c 262144 | huggingface.co | 20-core GPU. llama-server 0.7.0-dev 620d222 (LM Studio Metal backend 2.59.0), built before PR #30699; UraionLabs' rewrite of NVIDIA's 24 GB GGUF with the 80 NVFP4 ssm tensors converted to Q8_0 (24.2 GB), so it does not anchor. NVIDIA's own file ran 12.8 tok/s on the same Mac (NVFP4 CPU fallback).; not used to anchor estimates |
| estimated31 tok/s | 4-bit | MLX | 48 GB | — | LLMCheck estimator | — |
Ollama — the index has not verified an Ollama tag for Nemotron 3.5 Bolt 74B-A9B, so it doesn’t print one (a guessed tag fails with “pull model manifest: file does not exist”). Search the Ollama library →
MLX — pick a build from Hugging Face’s MLX listings, then run it with its repo id:
Downloads: Hugging Face repo → · model card →
Prefer a GUI? LM Studio downloads GGUF and MLX builds in one click. Step by step: install Ollama on a Mac.
NVIDIA's OpenMDW 74B MoE (9B active), compressed from Super VL, with a native 2-bit '24 GB' build (15 Oct 2026). Stock llama.cpp runs it on Metal from b11755 (PR #30699, merged 20 Oct), but NVIDIA's 24 GB GGUF runs its NVFP4 layers on the CPU, so use a Q4_K_M file; Ollama does not support it on Apple Silicon (PR #19229 is open). mlx-vlm 0.7.9+ loads it: ~43 GB at 4-bit, with community runs of 58–72 tok/s (4-/5-bit) on an M5 Max. NVIDIA's ~24 GB IQ2 checkpoint ran at ~73 tok/s on a 64 GB M5 Max in oMLX (community, custom loader).
OpenMDW licence · 74B MoE · 256K context · LLMCheck capability 40/50 · see its leaderboard rank →
Nemotron 3.5 Bolt 74B-A9B needs about 45 GB of unified memory. These current Apple Silicon Macs have the headroom to run it comfortably:
As an Amazon Associate, LLMCheck earns from qualifying purchases. These affiliate links cost you nothing extra and help keep the LLMCheck index free.
See how Nemotron 3.5 Bolt 74B-A9B stacks up against other models on your specific Mac hardware.
Open Compare Tool Full Leaderboard