Recursive Flint on a Mac: Speeds, RAM and Setup for the 40B-A2.5B Coding Model

What Flint is

From Recursive’s model card:

• Parameters: 39.8B in total and about 2.5B active per token, counting the output head (about 2.2B without it). “40B-A2.5B” is the rounded name.
• Layout: 49 layers, 2 dense and 47 mixture-of-experts; 256 routed experts with 6 chosen per token, plus one shared expert.
• Attention: 35 sliding-window layers (513 tokens) and 14 full-attention layers, with 4 key/value heads. That keeps the memory cost of long contexts low.
• Context: 262,144 tokens, shared between prompt and output. Text in, text out.
• Behaviour: it always reasons before answering, with no switch to turn that off, and it makes OpenAI-compatible tool calls. Recursive recommends temperature 1.0, top-p 0.95, top-k 20 and an output budget of at least 32,768 tokens for hard problems.
• Training: from scratch, with a stated knowledge cutoff of June 2026. Licence: Apache 2.0 for the weights; Recursive’s hosted API has separate terms.

Benchmarks, with their harnesses

All of these are Recursive’s own runs at its recommended sampling settings, averaged over at least five attempts. The comparison columns come from the same chart; scores Recursive did not measure itself are the developers’ own.

Recursive’s benchmark chart (Hugging Face card). † = measured by Recursive; other comparison scores are the developers’ own.

Three cautions. First, the SWE-bench Pro v2 column is Recursive’s internal run, on its own harness: it is not the public SWE-bench Pro, and it is not comparable with Scale’s standardized SEAL leaderboard, where vendor-scaffold scores typically sit 15–20 points higher than SEAL’s. Second, Recursive counts Terminal-Bench solutions found by web retrieval as failures, and says other developers may not. Third, the NVFP4 column shows Recursive’s NVFP4 checkpoint within 1.2 points of BF16 on every test; Recursive publishes no scores for the INT4, Q4_1 or Q4_K_M builds most Macs run. The LLMCheck index has no Arena, MMLU or HumanEval figure for Flint, so its capability score of 37/50 is estimated from these vendor coding benchmarks alone.

Every measured Mac figure, by runtime

These are all the plain-decode Flint runs in the index. “Counts” means the run can anchor the index’s estimates for other Macs; the others are shown for completeness.

Recursive Flint: every measured plain-decode run in the LLMCheck index (data version 2026-10-24), with speculative decoding and MTP off. Decode only; prefill excluded.

The three stock llama.cpp runs come from one harness, run by the developers of the Kapitsa engine on 22 October: the median of five 512-token coding answers at Flint’s recommended temperature, with Recursive’s Q4_1 file. The M5 Pro ran 1.16× the M4 Pro, close to their 1.12× bandwidth ratio. The 36 GB M3 Pro ran at 32K context, because 140K ran out of memory.

The same harness timed rec, Recursive’s engine, on the same two Macs: 126.6 tok/s on the M4 Pro and 144.9 on the M5 Pro, 1.6× llama.cpp on both (community; M5 Pro write-up). Kapitsa’s own engine tied rec. Neither engine is in the table, for the reason in the next section. On the 36 GB M3 Pro, rec refused to start: it requires 48 GB.

Speed falls with context, as with any model. On the M5 Pro, llama.cpp b11840 decoded at 90.7 tok/s after a 1K prompt and 30.8 after a 128K prompt (community, Kapitsa’s context sweep, 24 October). Long-prompt figures are a different workload, so the index keeps them in prose.

Why custom-engine and pre-merge runs don’t count

An estimate for your Mac should come from software you can install today. So the index displays, but does not build estimates on:

• Vendor and custom engines. rec is Recursive’s own engine, which supports only the M4 Pro, M5 Pro and M5 Max with 48 GB or more. Recursive’s chart puts it at 131.2 tok/s on an M4 Pro (published), the source of its “more than 125 tokens per second on an M4 Pro MacBook Pro” claim. Kapitsa’s runs above show the speed is real. It is not a general runtime, though, and it says nothing about other Macs.
• Pre-merge code. Recursive’s llama.cpp comparison point (88.0 tok/s on the same chart) states no build and was charted before Flint support merged into llama.cpp (PR #30928, 21 October; first release b11803). The 104.5 tok/s M3 Ultra run used mlx-lm from PR #1998, which merged on 24 October but is not yet in a release (the latest is v0.32.2).
• Unexplained builds and cut-down chips. The 110.2 run used an oMLX build tagged “+omni” at about 6 bits. The 131 run was on the binned M5 Max.

One more caveat: llama.cpp’s first Flint support scales expert activations with a workaround that its author says costs 10–30% of throughput until a follow-up change. Stock llama.cpp figures are likely to rise.

How the M5 Max figure is built

The leaderboard scores every model on one reference chip, and for Flint that is the M5 Max. The index holds no eligible run on the full 40-core M5 Max (614 GB/s) at 5 bits or fewer. The fastest relevant run is on the binned part, the 32-core-GPU M5 Max (36 GB; 460 GB/s): 131 tok/s in Rapid-MLX 0.16.8 with Recursive’s INT4 checkpoint, greedy decoding, about 1K of context and speculative decoding off (community; published by Rapid-MLX’s developers, release notes). A binned run never sets a reference figure.

Left to its usual method, the index scales a chip from the model’s nearest-bandwidth measured run. For the M5 Max that is the M3 Ultra (819 GB/s; 90.6 tok/s at 4-bit and 51.3 at BF16 in mlx-vlm 0.7.10), which gives about 77 tok/s. That would have put the full M5 Max well below the cut-down one. The M3 generation is simply slower per GB/s on this model than the M5: its M3 Ultra runs (90–105 tok/s) are no faster than the M5 Pro’s (81–115) on well under half the bandwidth.

So from this update the index applies a floor. An estimate may not fall below a measured plain-decode run of the same model on a slower configuration of the same chip generation, a binned part of the same chip included, at equal or more bits (methodology). The full M5 Max is therefore estimated at estimated~131 tok/s. That is the binned run’s figure carried up as an estimate, marked with a hollow diamond, and it is a floor: a chip with a third more bandwidth should be at least that fast.

The Score follows: capability 37 + speed 25 (the cap) + accessibility 10 (a 20 GB minimum) + licence 8 (Apache 2.0) = 80, #2 behind Qwen 3.6-35B-A3B. Without the floor, about 77 tok/s would have earned 19 speed points and a Score of 74. One stock run on a full M5 Max at 5 bits or fewer would replace the estimate. The mlx-community 3-bit build’s card reports 76–119 tok/s on an M5 Max in mlx-vlm, slowing as the output grows; that is a range, not a decode run, so it is not a row.

How much RAM Flint needs

The weights are the easy part. The context is what decides between 24, 32 and 48 GB, because Flint thinks before every answer, often for 10–30K tokens according to the mlx-community card.

Flint builds by download size, checked on Hugging Face and ollama.com on 24 October 2026. The last column is LLMCheck’s guidance from file size against memory, not a measurement.

• 24 GB: a 3-bit build of about 18 GB, with a short context of around 32K tokens. macOS lets the GPU use only about 16 GB of a 24 GB Mac by default, so the mlx-community card raises the limit to 20 GB with sudo sysctl iogpu.wired_limit_mb=20480 (it resets at restart). The card also says the build has not yet been tested on a real 24 GB Mac. Its next-token choices match Recursive’s INT4 86% of the time.
• 32–36 GB: the 23 GB 4-bit builds, with a modest context. Measured peaks: 22.2 GB for oMLX’s oQ4e at 1K, about 24 GB for Rapid-MLX’s INT4 on the 36 GB M5 Max, and 26.6–26.9 GB for llama.cpp with the Q4_1 file at 4K–16K prompts.
• 48 GB: what Recursive asks for. Its Q4_1 GGUF card says the file requires at least 48 GB, and its example runs the full 262,144-token window with an F16 cache; rec requires 48 GB, and Rapid-MLX recommends it. By LLMCheck’s arithmetic from the card’s attention layout, an F16 cache costs about 28 KB per token in the 14 full-attention layers, roughly 7.5 GB at the full window, on top of the weights.

The index lists a 20 GB minimum, from the 3-bit build plus headroom. Flint can go lower, but not usefully: on a 16 GB M1 Pro, llama.cpp ran Unsloth’s 11.86 GB UD-IQ2_XXS build (1- to 3-bit experts) at 42.4 tok/s with an 8K context while the Mac swapped during every run (community, Kapitsa write-up). That shows Flint can be squeezed onto 16 GB, not that it fits.

Setup: Ollama, llama.cpp, LM Studio, MLX or rec

Ollama (easiest)

Ollama added Flint in v0.40.8 on 21 October, and its release notes say Flint is supported on macOS; v0.40.11 is current. The default tag is 23 GB:

ollama run flint                    # = flint:40b, 23 GB
ollama run flint:40b-a2b-q4_1       # Q4_1, 26 GB

The library also lists 40b-a2b-qat-int4 and an MLX 40b-a2b-gguf-nvfp4 tag (both 23 GB) and an 80 GB MLX BF16 tag (checked on ollama.com on 24 October). Ollama sets its default context by GPU memory (4K below 24 GiB, 32K at 24–48 GiB, 256K at 48 GiB and above), and Flint’s reasoning alone can fill 4K, so set it explicitly with OLLAMA_CONTEXT_LENGTH as memory allows.

llama.cpp (b11803 or later)

The ggml-org card’s one-line start:

llama serve -hf ggml-org/Flint-GGUF

The repo ships Q4_K_M (23.4 GB), Q8_0 and BF16; add :Q8_0 to the name to choose a file.

Recursive’s own Q4_1 file runs with llama-server -m flint-int4-q4_1.gguf -ngl all -c 262144 -np 1 --cache-ram 0 on a 48 GB Mac; on 32 GB, lower -c, which the card allows for reducing cache memory. Pass Recursive’s sampling settings. Issue #31000, in which tool-call markup stays inside the reasoning with the NVFP4 file, was still open on 24 October.

LM Studio

Search “flint” for the lmstudio-community GGUFs (Q4_K_M 24.17 GB, Q6_K, Q8_0), or load one of the Unsloth files for a 24 GB Mac.

MLX

mlx-vlm 0.7.10 (21 October) added native Flint support, and it is the MLX path today: pip install -U mlx-vlm, then point mlx_vlm.generate or mlx_vlm.server at a Flint MLX build, such as mlx-community/flint-mixed-3-4bit on 24–32 GB. mlx-lm support merged on 24 October and will arrive in its next release. oMLX 0.7.2 runs the oQ4e build measured above. The mlx-community card reports that in mlx-vlm 0.7.10 Flint sometimes uses wrong tool-argument names (41 of 85 calls failed in its OpenCode test, though every task still passed).

rec (48 GB M4 Pro, M5 Pro or M5 Max)

Recursive’s engine is the fastest way to run Flint on the Macs it supports, about 1.6× llama.cpp in Kapitsa’s runs. It needs at least 48 GB and about 30 GB of disk, and it is built for one session at a time. rec launch pi or rec launch opencode starts a coding harness, and rec serve exposes OpenAI- and Anthropic-compatible APIs. The installer is in the rec README.

Flint vs Qwen 3.6-35B-A3B vs KAT-Coder-V2.5

Speeds are the LLMCheck index’s per-chip figures, refilled on every build; the shape beside each says whether it is measured or estimated. SWE-bench scores are each vendor’s own run on its own harness.

Where the two leaders were run side by side, Flint was faster: in the same Kapitsa harness on 48 GB MacBook Pro M5 Pros, stock llama.cpp gave 92.4 tok/s on an M5 Pro with Q4_1 against 81.96 for Qwen 3.6-35B-A3B with Q4_K_M (community). Flint’s vendor SWE-bench Verified score (74.2) is level with Qwen’s (73.4), on a different harness. KAT-Coder-V2.5 has no M5 Pro run yet. Qwen 3.6-35B-A3B keeps #1 on a published M5 Max figure rather than an estimate, adds image and video input, and has six months of community use behind it. Flint is a week old, and its scores are its maker’s.

• 24 GB: both need a 3-bit or small 4-bit build. Flint’s ~18 GB 3-bit leaves more room than Qwen’s 23 GB Ollama tag; the index’s 24 GB tier pick is still Qwen 3.6-35B-A3B.
• 32–36 GB: try Flint for agentic coding next to Qwen 3.6-35B-A3B. On the 36 GB Mac Studio M5 Max, Flint is the index’s headline pick (that Mac’s page).
• 48 GB on an M4 Pro, M5 Pro or M5 Max: Flint with rec and the full context window is the fastest way to run it, about 1.6× llama.cpp in community runs.
• KAT-Coder-V2.5, Kwaipilot’s fine-tune of Qwen 3.6-35B-A3B, remains the choice if you work through Claude Code or a similar harness, where Kwaipilot ran its scores.

Which Mac to buy for Flint

• Trying it on a Mac you own: any 24 GB Apple Silicon Mac should run the 3-bit build with a short context after raising the GPU memory limit (the build is untested on a 24 GB Mac so far), such as the Mac mini M6 24 GB.
• Best value for Flint: a 48 GB M5 Pro, the Mac mini or a MacBook Pro. It runs the 4-bit model with its full context, it is one of rec’s three supported chips, and the index shows community92 tok/s on the M5 Pro.
• Fastest single Mac: a full 40-core M5 Max (48 GB MacBook Pro, or 64 GB and up), estimated at estimated~131 tok/s. The binned 32-core-GPU M5 Max (36 GB) is measured at community131 tok/s, but it is short of rec’s 48 GB.

Frequently Asked Questions