Nemotron 3.5 Bolt on a Mac: The Metal Crash, What Works, Super VL

What NVIDIA shipped

Two open-weight models arrived on 15 October 2026, both under OpenMDW-1.1 (8/10 on the index’s license scale), per their Hugging Face cards and NVIDIA’s announcement:

• Nemotron 3.5 Super VL 120B-A12B: a Mamba-2/Transformer hybrid mixture-of-experts with 120B total and 12B active parameters, 22 of 512 routed experts per token, a C-RADIO vision encoder, text, image and video input, up to 1M tokens of context and a native multi-token-prediction head.
• Nemotron 3.5 Bolt 74B-A9B: compressed from Super VL with NVIDIA’s Star Elastic search to 74B total and 9B active, with routing cut to 9 experts per token (down from 22) and per-layer expert counts between 240 and 496. Up to 256K context, text, image and video in.

Bolt’s deployment build is the “24GB” GGUF: 22.02 GB plus a 1.73 GB vision projector. It is mixed precision, not a plain 2-bit quant. The routed and shared experts are IQ2_XXS (2.0625 bits per weight, recovered with quantization-aware distillation), the 40 Mamba layers are NVFP4, and the embeddings and output head are Q8_0. NVIDIA’s card lists Ada and Blackwell GPUs on Windows and Linux and makes no Apple Silicon claim. NVIDIA’s own 84-tokens-per-second figure was measured on an RTX Spark with a speculative drafter, not on a Mac.

On NVIDIA’s own table, Bolt beats Qwen 3.6-35B-A3B on Terminal-Bench 2.1 (50.98 vs 44.38), GPQA Diamond (86.43 vs 83.40) and BrowseComp (66.75 vs 48.74), and trails it on PinchBench (87.82 vs 88.07) and both AA-Omniscience rows (accuracy 16.90 vs 19.47; non-hallucination 61.27 vs 65.85). NVIDIA ran every column, re-scored Qwen’s MCP-Atlas from Qwen’s reported 62.8 down to 51.2, and says its fine-tuning data targeted GPQA Diamond, Terminal-Bench 2.1 and SciCode. The index therefore discounts the lead: capability 40/50 for Bolt against 38 for Qwen 3.6-35B-A3B. As of 18 October the leaderboard ranks Bolt #17 (LLMCheck Score 70) and Super VL #58 (Score 61).

Why Bolt crashes: the ggml_metal_op_mul_mat_id segfault

Upstream llama.cpp added both models in PR #30520, merged 15 October (first tagged build b11669). Bolt loads on a Mac. Then the server dies on the first request, during prompt processing, with EXC_BAD_ACCESS (SIGSEGV). The backtrace runs through ggml_metal_pipeline_max_theads_per_threadgroup ← ggml_metal_op_mul_mat_id. It is issue #30610, first reported on an M1 Max 32 GB.

The cause is one missing kernel. For batched mixture-of-experts matmuls, llama.cpp’s Metal backend builds a routing “map0” kernel per experts-per-token count, and it only instantiates 1, 2, 4, 5, 6, 8, 10, 16 and 22. Bolt asks for 9. The lookup returns a null pipeline, which is dereferenced on the next line. Single-token decode takes a different path, so the crash always lands in prompt processing. It is not the file: every quant of Bolt triggers it, from NVIDIA’s 24 GB build to a Q8_0. Super VL routes 22 experts, a count that already has a kernel, so it is unaffected.

The fix is a one-line <9> instantiation. Commenters confirmed it on an M1 Max, an M3 Ultra and an M5 Max, and it is open as PR #30699. As of 18 October the PR has no reviews and is unmerged, and the newest release, b11718, still lacks the 9-expert kernel. Until it merges and ships in a release, any llama.cpp you install from a package manager will crash on Bolt.

Ollama is a dead end for now, too. Its library page for nemotron-3.5-bolt (three 25 GB tags) says it plainly: “Apple Silicon / Metal is not currently supported.”

Why NVIDIA’s 24 GB file crawls on a Mac

Even with the patch, NVIDIA’s own file is slow on Apple Silicon. Its 80 Mamba projection tensors (ssm_in/ssm_out) are NVFP4, and Metal has no NVFP4 matrix multiply, so llama.cpp runs those layers on the CPU for every token. The community numbers in the issue thread show the cost, all with the unmerged patch:

• M3 Ultra 96 GB, llama-bench tg128: NVIDIA’s file 6.40 tok/s; the same file with only the NVFP4 tensors rewritten to Q8_0 (22.0 → 24.2 GB) 37.74 tok/s; a community Q4_K_M made from the BF16 weights (45 GiB) 41.61 tok/s. All three are community-measured, posted 16 October.
• M5 Max: about 11–12 tok/s community-reported with only the 9-expert fix (the file used is not stated).
• M5 Pro 64 GB: UraionLabs publishes that Q8_0 rewrite as a 24.21 GB “Metal GGUF” and reports decode rising from 12.8 to 24.6 tok/s and 2K-token prefill from 181 to 676 tok/s, community-measured with llama-server.

The lesson for Mac owners: the “24GB” build is the smallest file but not the fastest. On the same M3 Ultra, a 45 GiB Q4_K_M decoded faster than the 24 GB rewrite. If you have the memory, a conventional 4-bit build is the better Mac file.

What works today

Three routes run Bolt on a Mac now. All of them are community or third-party software, which is why the index carries a runtime caveat on every Bolt figure. Stock mlx-lm (0.32.2) cannot load it yet; support is PR #1982, still open.

1. mlx-vlm from git: text, image and video (64 GB+)

Support for Bolt and Super VL merged into mlx-vlm on 16 October (PR #2528), after the latest PyPI release (0.7.8, 13 October), so install from git. The nativ-community MLX conversions pin the tested commit and need MLX 0.32.4. The 4-bit build is 43.15 GB on disk and peaked at 46.6 GB in its card’s checks.

pip install 'git+https://github.com/Blaizzy/mlx-vlm.git@e17c49a5061b4399df3f5bda206ea2a63d45486a'
python -m mlx_vlm generate \
  --model nativ-community/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-4bit \
  --prompt "Write a short Python function that returns the first n Fibonacci numbers." \
  --thinking-mode disabled --max-tokens 512 --temperature 1.0 --top-p 0.95

Add --image photo.jpg or --video clip.mp4 --fps 2 for vision. A separate 8-bit port (divinetribe, about 81 GB, mlx-vlm 0.7.8 plus a bundled install script) needs a 128 GB Mac.

2. Rapid-MLX 0.16.2 or later: text only (48 GB+)

Rapid-MLX shipped Bolt text support on 16 October, with a 3-bit build (31.7 GB, recommended for 48 GB+ Macs) and a 4-bit build (43.3 GB, 64 GB+). Image, video and the drafter are not supported. Install from PyPI: Homebrew still carries 0.15.7, which predates Bolt.

pip install -U rapid-mlx
rapid-mlx serve rapid-mlx/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-MLX-3bit

3. llama.cpp with the PR #30699 patch (36 GB+)

If you want GGUF and an OpenAI-compatible server with vision, build the PR branch and use a file without NVFP4 tensors: the UraionLabs rewrite on a small Mac, or a Q4_K-class build from the BF16 weights (Unsloth’s UD-Q4_K_XL is 50.2 GB) if you have 64 GB or more.

git clone https://github.com/ggml-org/llama.cpp && cd llama.cpp
git fetch origin pull/30699/head:bolt-metal && git checkout bolt-metal
cmake -B build && cmake --build build --config Release -j
hf download UraionLabs/Nemotron-3.5-Bolt-74B-A9B-24GB-Metal-GGUF --local-dir bolt-metal
./build/bin/llama-server -m bolt-metal/Nemotron-3.5-Bolt-74B-A9B-24GB-Metal-IQ2_XXS.gguf \
  --mmproj bolt-metal/mmproj-Nemotron-3.5-Bolt-74B-A9B-24GB-BF16.gguf \
  -ngl 99 -fa on -c 65536 -np 1 --jinja

UraionLabs also reports its file loading in LM Studio’s bundled llama.cpp engine (Metal backend 2.59.0). That engine is built from the same upstream commit that crashes in issue #30610, so LLMCheck cannot confirm which LM Studio builds carry the fix. Treat LM Studio as worth a try, not as a supported path.

Bolt speeds on a Mac so far

These are the runs in the LLMCheck dataset. All are community figures from ported or patched runtimes. The three M5 Max runs are short samples or a quoted range, with the prompt length unstated, so treat them as indicative rather than as a decode benchmark.

All five are community runs (hollow circle); each figure links to its source. Runs on unmerged code (the 8-bit bundled port, the patched llama.cpp) are listed but do not anchor the index’s estimates.

The Rapid-MLX run comes from the runtime’s own performance note: its 4-bit build at about 56.5 tok/s on a 256 GB M3 Ultra, a community-measured figure. Both M3 Ultra runs sit well below what bandwidth scaling from the M5 Max predicts, so the index scales Bolt’s Ultra-chip estimates from the Rapid-MLX run instead of from the M5 Max.

For the chips people actually buy at 48–128 GB, the index figures below update with every build. At publication the M5 Max figure is the 5-bit community run, and the rest are 4-bit MLX estimates anchored on the M5 Max runs, which is why the M4 Max estimate can sit slightly above it.

How much memory Bolt really needs

“24GB” describes NVIDIA’s target, a 24 GB NVIDIA GPU at batch size 1. It does not describe a Mac. The 24 GB build plus its vision projector is 23.75 GB before context, macOS and your other apps. On Apple Silicon it also needs the patched llama.cpp and runs part of every token on the CPU. The issue reporter loaded it on a 32 GB M1 Max, and UraionLabs calls 36 GB the practical floor for its rewrite and 48 GB comfortable.

The index lists Bolt at a 45 GB minimum, the 4-bit MLX build plus its usual 2 GB allowance, which puts it in the 48 GB tier. In practice:

• 24 GB: no.
• 32 GB: only NVIDIA’s file on the patched llama.cpp, which a 32 GB M1 Max ran in the issue thread. It is slow, because of the CPU fallback, and leaves almost no headroom.
• 36–48 GB: the patched llama.cpp with the 24.2 GB rewrite, or Rapid-MLX’s 3-bit build (text only) on 48 GB, such as the 48 GB Mac mini M5 Pro.
• 64 GB: the 4-bit MLX builds, with vision through mlx-vlm. This is where Bolt stops being a squeeze; see the 64 GB Mac mini M5 Pro page.
• 128 GB: 8-bit, or 4-bit with room for long contexts. Only 8 of Bolt’s 88 layers are attention, so even a 128K-token cache is about 1 GB.

Super VL: the clean option for 96–128 GB

Super VL is the model to reach for if you want NVIDIA’s new family on a Mac without patching anything. Its 22-expert routing has a Metal kernel, so the crash does not apply, and three runtimes support it, two of them in normal releases:

• llama.cpp b11669 or later (Homebrew’s llama.cpp 0.7.0 already includes it). The lmstudio-community Q4_K_M is 70.10 GB, ggml-org’s Q4_0 is 71.80 GB, and Unsloth’s UD-IQ3_XXS is 59.76 GB; add the 1.73 GB mmproj for images. No llama.cpp Mac speed has been published yet.
• Ollama 0.40.5 or later: ollama run nemotron-3.5-super. The default tag is 80–90 GB, and MLX-format tags are listed (nvfp4 80 GB, mxfp8 130 GB). Unlike Bolt’s page, it carries no Apple Silicon warning.
• mlx-vlm from git (the same PR #2528), with text, image, video and the MTP drafter.

The one Mac figure so far comes from the mlx-vlm PR: 30.6 tok/s plain decode on a 128 GB M5 Max at NVFP4 (75.0 GB of weights, 88.1 GB peak, 160 generated tokens), community-measured. With the native MTP head the same PR reports 57.1 tok/s. That is speculative decoding, a different workload, so the index records the 30.6.

At publication the M5 Max figure is the NVFP4 mlx-vlm run, and the others are estimates from the index’s bandwidth model. No Super VL run on those chips has been published yet, and the Bolt runs above are a reminder that Ultra chips rarely deliver their full bandwidth advantage. The index lists a 72 GB minimum, so a 96 GB Mac holds the 70 GB Q4_K_M with short contexts (you may need to raise macOS’s GPU memory limit). 128 GB is the comfortable tier, and the one you need for Ollama’s 80–90 GB tags or NVFP4 in MLX.

On quality, NVIDIA reports SWE-bench Verified 71.10 and SWE-Bench Pro 51.78 from its own NeMo Gym harness. Vendor scaffolds typically run about 15–20 points above Scale’s standardized SEAL harness on SWE-Bench Pro. Artificial Analysis independently puts it at 26 on its Intelligence Index. The index scores capability conservatively at 42/50.

Which one should you run?

• 24–32 GB Mac: neither. Qwen 3.6-35B-A3B is the index’s pick at this tier (#2, Score 81), community-measured at community64 tok/s on a 32 GB Mac mini M6, and it runs in every standard runtime.
• 48–64 GB Mac: Bolt is worth trying if its agentic and browsing scores matter to you and a community runtime does not bother you. If you just want the best model that works out of the box, Qwen 3.6-35B-A3B is still faster and simpler (estimated~92 tok/s estimated on an M5 Pro, against estimated~38 tok/s estimated for Bolt). See the 64 GB tier.
• 96–128 GB Mac: Super VL if you need image and video input or very long context with a clean runtime. It is not the speed pick: GPT-oss 120B ranks higher at this tier (#45, Score 64) at sourced76 tok/s published for an M5 Max (a press review, quant unstated) against Super VL’s 30.6 community-measured, and Qwen3.8-Flash-Next is the index’s 128 GB pick.

Don’t buy a Mac for Bolt yet. Its best Mac numbers come from a port that is two days old, and the two fixes that would make NVIDIA’s own file work well on Apple Silicon have not shipped.

What to watch

Frequently Asked Questions