Nemotron 3.5 Bolt 74B-A9B runs at 64 tok/s (63.6 as recorded) on a 128 GB M5 Max, a community run reported by huggingface.co (MLX, 5-bit, 64-token greedy sample). It needs about 45 GB of unified memory. It is NVIDIA's 74B mixture-of-experts model (9B active), released under the OpenMDW licence.
Stock llama.cpp segfaults on Metal (the fix, PR #30699, is unmerged) and Ollama does not support Apple Silicon; it runs through community MLX ports (mlx-vlm, oMLX, Rapid-MLX).
This is a community run: a real run, reported by its source and linked below. The index records it as given and never adjusts it.
| Metric | Value |
|---|---|
| Tokens per second | community64 tok/s |
| Provenance | community run · huggingface.co |
| Quantization | 5-bit |
| Engine | MLX |
| Context | 64-token greedy sample |
| Figure dated | 2026-10-16 |
| Minimum RAM | 45 GB |
| Parameters | 74B MoE (9B active) |
| Chip bandwidth | 614 GB/s |
Each row as the index holds it, with its source. “Mac RAM” is the memory of the Mac a run used, not the model’s requirement (45 GB). The headline figure follows the reference rule: the lower-median measured plain-decode run at 5 bits or fewer (a run whose quant was not stated counts only when no stated-quant run qualifies), else the estimate; a run marked “not used to anchor estimates” is listed here but never becomes the headline.
| Speed | Quant | Engine | Mac RAM | Context | Source | Note |
|---|---|---|---|---|---|---|
| community71.7 tok/s | 4-bit | MLX | 128 GB | 128-token greedy sample | huggingface.co | Community MLX port: MLX-VLM e17c49a on MLX 0.32.4; 4-bit affine g64 (43.15 GB), no MTP head in the checkpoint; 46.6 GB peak. Same figure in mlx-vlm PR #2528. |
| community63.6 tok/s | 5-bit | MLX | 128 GB | 64-token greedy sample | huggingface.co | Community MLX port: MLX-VLM e17c49a on MLX 0.32.4; 5-bit affine g64, vision encoder BF16; 59.6 GB peak. |
| community37 tok/s | 8-bit | MLX | 128 GB | — | huggingface.co | Community MLX port: mlx-vlm 0.7.8 + the uploader's nemotron_h_omni port; 8-bit g64 (~81 GB); lower bound of 37–41.; not used to anchor estimates |
Ollama — the index has not verified an Ollama tag for Nemotron 3.5 Bolt 74B-A9B, so it doesn’t print one (a guessed tag fails with “pull model manifest: file does not exist”). Search the Ollama library →
MLX — pick a build from Hugging Face’s MLX listings, then run it with its repo id:
Downloads: Hugging Face repo → · model card →
The 64 tok/s figure above came from MLX at 5-bit; other runtimes and quants run at different speeds.
Prefer a GUI? LM Studio downloads GGUF and MLX builds in one click. Step by step: install Ollama on a Mac.
| Chip | Speed | Quant | Engine | Bandwidth |
|---|---|---|---|---|
| M4 Max | estimated65 tok/s | 4-bit | MLX | 546 GB/s |
| M3 Ultra | community57 tok/s | 4-bit | MLX | 819 GB/s |
| M3 Max | estimated49 tok/s | 4-bit | MLX | 400 GB/s |
| M5 Pro | estimated38 tok/s | 4-bit | MLX | 307 GB/s |
NVIDIA's OpenMDW 74B MoE (9B active), compressed from Super VL, with a native 2-bit '24 GB' build (15 Oct 2026). On a Mac it needs a community runtime today: stock llama.cpp crashes on Metal until PR #30699 merges, Ollama says Apple Silicon is unsupported, and NVIDIA's 24 GB GGUF runs its NVFP4 layers on the CPU (~6 tok/s). MLX ports for mlx-vlm from git work: ~43 GB at 4-bit, with community runs of 64–72 tok/s (5-bit / 4-bit) on an M5 Max.
OpenMDW licence · 74B MoE · 256K context · LLMCheck capability 40/50 · see its leaderboard rank →
Nemotron 3.5 Bolt 74B-A9B needs about 45 GB of unified memory. These current Apple Silicon Macs have the headroom to run it comfortably:
As an Amazon Associate, LLMCheck earns from qualifying purchases. These affiliate links cost you nothing extra and help keep the LLMCheck index free.
See how Nemotron 3.5 Bolt 74B-A9B stacks up against other models on your specific Mac hardware.
Open Compare Tool Full Leaderboard