22 November 2026 — Google Analytics added

No ranking or data change. Every page now loads Google Tag Manager, which runs Google Analytics alongside Cloudflare’s cookieless Web Analytics. Google Analytics sets cookies; the privacy policy lists what it receives and how to opt out, and the footer now names both services. The embeddable checker at /embed/checker/ loads no Google script.

19 November 2026, second update — “Can I run it?”, the macOS GPU memory limit, a context calculator, Markdown versions

No ranking change. Data version 2026-11-18 is unchanged: the leaderboard, the Scores and the RAM-tier picks are as in the update below.

The macOS GPU memory limit. macOS lets the GPU use only part of a Mac’s unified memory: from two-thirds on an 8 GB Mac to 87% at 256 GB on macOS 26, and less on macOS 15 and earlier. Apple publishes no formula, so the index reads the default from startup logs people have posted (rule and sources; at 512 GB the reported figure cannot be told apart from a raised limit, so the 256 GB band’s 87% is used). Every Best-by-Mac page, model page and the new “Can I run it?” pages now say whether a model fits the default limit, needs it raised (with the sudo sysctl iogpu.wired_limit_mb command, which lasts until the next restart), or is over even a raise that leaves macOS 4–16 GB, in which case runtimes move part of it to the CPU, shorten the context, run slowly or stop out of memory. A published run counts as evidence: one on a Mac with no more memory, or a published peak memory inside the raised limit, overrides the formula. A Mac page’s headline pick must fit within that safe raise, as a model with a runtime caveat cannot be the pick: the 24 GB Mac mini M4 Pro now leads with Qwen 3.6-35B-A3B instead of KAT-Coder-V2.5, whose published runs peaked above 20 GB. “Agent-ready at 64K” now means the model and a 64K context fit that safe raise, not just the Mac’s memory, and a speed measured on a bigger Mac is shown for a smaller one only when its build fits there. The RAM-tier picks do not change. Step 1 of the methodology, which said the GPU gets about three-quarters of memory on every Mac, is corrected.

How far off the estimates are. Every estimated speed now comes with the formula’s typical error, measured against the published and community runs in the index: about ±9% for dense models, ±15% for small mixture-of-experts models and ±34% for large ones, where the formula usually undershoots (details).

“Can I run it?” One page per model with a Mac runtime (122): a verdict for each of the 54 Mac configurations, the memory each quantization needs, every measured run with its source, the longest context that fits, and setup. The hub picks a model and lists the Macs that run it.

Tools. A context memory calculator shows what a 32K, 64K or 128K context costs on top of the weights for 117 models, with the runtime settings that control it. The Mac checker can now be embedded on other sites: free, no account, the same picks as each Best-by-Mac page.

Markdown versions for AI assistants. The leaderboard, key numbers, methodology, every Best-by-Mac page and every “Can I run it?” page have a Markdown twin under /md/, linked from the page and from llms.txt, which also gains a dated “Key facts” section. Those pages carry a “Cite this page” box with a ready-made citation (CC BY 4.0).

New post: Black Friday 2026: which Mac to buy for local LLMs, with the best Mac per budget from the index’s own picks, what Amazon stocks and what is build-to-order from Apple.

19 November 2026 — a site update: the checker matches each Mac’s page, context memory, Mac-vs-Mac pages, honest Amazon labels

No ranking change. The top of the leaderboard is as on 16 November: MiMo-V2.6-Flash and Qwen 3.6-35B-A3B tie at Score 81 (MiMo-V2.6-Flash listed first on the name tie-break), then GLM-4.7-Flash and Recursive Flint at 80. Data version 2026-11-18: 130 models, 620 speed figures (73 published, 247 individual runs, 300 estimated) for 122 models on 18 Apple chips, and 54 Mac configurations.

The homepage checker shows the same picks as each Mac’s page. It now reads the ranked list every Best-by-Mac page prints (data/picks.json): the same models, order, speeds and provenance marks, and the nearest page with no more memory or bandwidth when your exact Mac has none. Its old size-sorted pick and separate “Mac score” are gone. Nine Macs the picker could not select before can now be chosen, a cross-site scripting hole in the checker is closed, and an Intel Mac gets its own answer instead of an upgrade path written for unified memory.

Context memory, from each model’s config.json. For 119 of the 130 models the index now reads the attention layout from the model’s published config.json and computes what a long context costs in KV cache (methodology). The leaderboard gains a RAM @64K column and an “agent-ready at 64K” filter, each Best-by-Mac page a 64K column, and each model page a memory-by-context table. These are arithmetic on the published architecture, labelled estimated.

Speed at depth, prefill, first token and speculative decoding, from the same session as each measured row only: decode at longer prompt depths on 165 rows, prompt-processing speed on 193, time to first token on 151 measured rows, and 16 speculative-decoding (MTP, DFlash, draft-model) figures. They appear on model pages, /benchmarks and in the CSV, and are never used for a ranking, a Score, an estimate or a tier pick (why).

Leaderboard filters by RAM tier, licence, use case, measured figures only, Ollama tag and agent-ready at 64K, on both the table and the phone cards, with the state kept in the address. Every model links its real Hugging Face card: all 130 hf links now point at the releasing lab’s official model card.

New pages. Twenty Mac-vs-Mac pages (M5 Pro vs M5 Max, M4 Max vs M5 Max, M3 Ultra vs M5 Ultra and the other pairs people search for), each with a verdict per use case, and a two-Mac mode in the compare tool. Three guides: a local coding agent on a Mac (Claude Code, OpenCode and Codex CLI on Ollama, LM Studio or oMLX), the best used Mac for local LLMs and clustering Macs over Thunderbolt 5. One post: Mac vs NVIDIA DGX Spark vs AMD Strix Halo, whose non-Mac figures are quoted from their sources and never enter the index. The chip comparison was cleaned up.

Honest Amazon labels. Amazon US stocks Apple’s standard configurations only. The high-memory ones (48–128 GB M5 Max, 48–64 GB M5 Pro, every M5 Ultra) now say “build-to-order from Apple” and link Apple’s store, with no affiliate link; configurations whose Amazon stock is not verified (the 24 and 32 GB MacBook Air M5, the M6 MacBook Pro and iMac) get a label without a memory size and a reminder to check the listing’s memory; and Macs Apple no longer sells new link Amazon Renewed listings, labelled as such. The build now fails on a regression.

Faster and easier to read. Each font family is now one variable file instead of nine identical per-weight copies, about 236 KB less on a first visit. The dark theme’s memory-tier colours were darkened so their labels meet WCAG AA contrast (at least 4.5:1; the smallest had fallen to about 1.3:1).

Contributions. The first run submitted through the dataset’s GitHub form is in: Jeremy Sonander’s run of Qwen3.8-27B (MLX 8-bit, LM Studio) on a 64 GB MacBook Pro M1 Max, 11.38 tok/s, community-measured. The contribute page was rewritten around plain decode with speculative decoding off, and a helper script runs llama-bench or mlx_lm.benchmark and pre-fills the submission form. A weekly release watcher now lists new model repos from the labs the index follows, runtime releases and Apple announcements for review; it never adds a model itself.

16 November 2026 — a tie at #1, GLM-4.7-Flash corrected, fourteen models, 126 new measurements

Catalog 130 models, 619 speed figures (73 published, 246 individual runs, 300 estimated) for 122 models on 18 Apple chips, and 54 Mac configurations; data version 2026-11-14. The research cutoff is 14 November; two runtime changes merged just after it are included and dated below. The month in one read: the November edition.

A tie at #1. MiMo-V2.6-Flash and Qwen 3.6-35B-A3B both score 81. Ties break by name, so MiMo-V2.6-Flash is listed first, and every page that names the #1 says it is a tie. MiMo-V2.6-Flash’s M5 Ultra figure is now the lower median of 15 eligible community sessions on 256 GB M5 Ultras (eleven on the 80-core GPU, four on the 64-core; oMLX at 4-bit and MXFP4, MTP off; 61.0–89.4 tok/s): 76.8, shown as 77 tok/s community-measured, against 62 from a single session on 24 October. Most of those sessions were posted to the oMLX community board between 24 September and 26 October and were missed by the October updates. MiMo-V2.6-Flash needs a Mac Studio with 192 GB or more. Qwen 3.6-35B-A3B needs only 24 GB and is the best pick on a Mac with 32 GB or less; its M5 Max figure is now the lower median of eight runs, 130 tok/s community-measured (was 118 published). Counting only the 80-core sessions would give MiMo-V2.6-Flash 78 tok/s and Score 82, #1 outright; the index has no grounds to drop the 64-core sessions (correction 1, below). GLM-4.7-Flash moves from #61 (Score 61) to #3 (Score 80) on a correction, and Recursive Flint is #4 (Score 80) on its first measured full-chip M5 Max figure. The tier picks shared by Best by RAM, the homepage, llms.txt and Key Numbers, with the 24 and 32 GB tiers now listed separately: 8 GB MiniCPM5-2B (was Nanbeige4.2-3B), 16 GB Gemma 4 12B, 24 GB Recursive Flint (was Qwen 3.6-35B-A3B), 32 GB Qwen 3.6-35B-A3B, 64 GB LFM2.5-24B-A2B, 128 GB Qwen 3.6-35B-A3B (was Qwen3.8-Flash-Next) and 192–512 GB MiMo-V2.6-Flash.

Rules written down (methodology). (1) One row per measurement session. A session is one sitting by one person or publisher: one Mac, one runtime build and one checkpoint, however many runs it logs. It enters the dataset once, at its shortest context (the median where that context was repeated), and the lower median is taken over sessions, so one prolific submitter cannot set a model’s figure; a single 21-run MiMo-V2.6-Flash series on 26 September is one row. The dataset was already built this way; the rule is written down because the #1 turns on it. (2) Binned means a configuration with less memory bandwidth than its full chip, such as the 36 GB M5 Max (460 GB/s) or the 16 GB M6 (153). Fewer GPU cores alone does not make a part binned: the 64-core M5 Ultra and the 60-core M3 Ultra have their chip’s full bandwidth, and their runs count like any other. (3) Fine-tunes and derivatives (checkpoints built on a catalog model, or with an identical architecture) are listed only when four conditions all hold: an accountable publisher with a licence file in the repository; a third-party evaluation, or the publisher’s own table against its base with the harness stated, showing a gain worth at least 2 capability points; a downloadable Mac build on a stock runtime; and more than a safety-removal, persona or style change. A lab’s updated checkpoint of its own model replaces its entry instead, and a lab’s official distillation into a different architecture counts as a new model. Six entries already in the catalog are derivatives under this definition (KAT-Coder-V2.5, Nemotron 3.5 Bolt E4B, Hermes 4 70B, Bonsai 27B, Ternary Bonsai 2 27B and Ternary Bonsai 2 9B); they stay listed and will be checked against the four conditions in the December update. (4) The new Mistral Ecosystem Open-Weight License is worth 5 of 10 licence points: an MIT-style grant, except that a company running a Model-as-a-Service business needs a separate commercial licence, which puts it with the other licences that need a separate agreement for API hosting.

Corrections. (1) The 64-core M5 Ultra with 256 GB is sold. Apple’s store offers the Mac Studio with a 30-core-CPU, 64-core-GPU M5 Ultra and 256 GB at $9,499 (checked 14 and 15 November), with the same 1.2 TB/s as the 80-core part. The 24 October entry below, the post on Qwen 3.6-35B-A3B’s return to #1 and the dataset note on MiMo-V2.6-Flash’s 76.4 tok/s run said Apple does not sell this configuration, and the run was set aside on that basis; it now counts. (2) GLM-4.7-Flash had been listed as a dense 31B model since January. Zhipu AI’s card calls it a 30B-A3B mixture-of-experts model and its configuration (glm4_moe_lite) agrees, so the index had costed every token as a read of all 31B weights (~26 tok/s estimated on an M5 Max). A community oMLX run of 46.6 tok/s on a 32 GB Mac mini M6 (oQ4e, MTP off) exposed the error; costed with about 3B active it estimates at 107 tok/s on an M5 Max, and its Score rises from 61 to 80. Its capability score (35) is unchanged; the rank comes from speed, an 18 GB minimum and the MIT licence. (3) Mac prices predated Apple’s June 2026 increase: the index listed the 24 GB MacBook Pro M5 Pro at $1,999 (Apple: $2,499) and the 128 GB MacBook Pro M5 Max at $3,899 (Apple: $6,699), and several build-to-order Mac Studio and Mac mini prices were approximations (the 256 GB M5 Ultra at $7,099; Apple: $9,499). Every current configuration now carries Apple’s list price, checked on 14 November. (4) The 512 GB M5 Ultra Mac Studio was described as not yet orderable. It has been orderable since 28 October at $22,299 (36-core CPU and 80-core GPU only, 1 TB), and the pages that said otherwise carry dated updates. (5) The Mac Advisor offered Macs Apple no longer sells new (the Mac mini M4 and M4 Pro, the MacBook Air M4, and the Mac Studio with M4 Max or M3 Ultra) as new purchases, and its budget slider stopped at $8,000, below the 256 GB ($9,499) and 512 GB ($22,299) M5 Ultra at Apple’s prices, so it could recommend neither. Those Macs are now labelled previous generation, priced used and linked to renewed listings first; “new” now means a Mac Apple sells today; the slider runs to $22,500. (6) Privacy wording. The footer said “No trackers, no third-party requests” and the privacy page that pages load no third-party scripts, but Cloudflare injects its cookieless Web Analytics beacon into every page at the edge. Both now say so: no ad trackers, and one third-party script, Cloudflare’s cookieless analytics. (7) The LM Studio guide said LM Studio runs on Intel Macs, in its text and its structured data. It needs an Apple Silicon Mac on macOS 14 or newer, and the guide now sends Intel users to the Intel Mac guide. (8) The /benchmarks title, shortened on 18 October to “Tokens per Second”, again carries the live figure count (“619 tok/s Figures, M1–M6”).

Fourteen models added, each traced to its Hugging Face card: six released on or after 24 October, six older releases the October entry had queued as “seen, not yet evaluated”, and two catalog gaps. For 8–32 GB Macs: MiniCPM5-2B (OpenBMB, Apache 2.0, 2.5B; Q4_K_M is 1.6 GB), the new 8 GB pick at about 39 tok/s estimated on an 8 GB MacBook Air M1, Score 69; K2-Horizon-3.7B (MBZUAI’s Institute of Foundation Models, Apache 2.0, 1 September, training data and code published; a catalog gap), Score 70 (#19), whose 3.2 GB file runs at about half MiniCPM5-2B’s speed on an M1, so it is not the 8 GB pick; LFM3-1.7B (Liquid AI, LFM Open, about 1 GB at 4-bit), 116.7 tok/s community-measured on a base M5, Score 63; Ling-3.0-tiny (Ant Group, MIT, a 7.9B MoE with 1.3B active), which fits an 8 GB Mac and ran at 174 tok/s community-measured on an M5 Max, its capability held at 15 because its only scores are vendor-quoted, Score 65; and Nanbeige4.3-33B (Nanbeige, Apache 2.0, dense 33.1B; Q4_K_M 20.8 GB, for 24 GB Macs), capability 46, level with Qwen3.8-27B, and 25 tok/s community-measured on an M5 Max, Score 70.

For 48 GB and up: Ripple-65B Preview (Ricursive Intelligence, which is not related to Recursive, the maker of Flint; Apache 2.0, dense 64.8B), for 48–64 GB Macs at ~13 tok/s estimated on an M5 Max; its MLX runs use a bundled model file and do not anchor; Score 59. Ling-3.0-flash-VL (Ant Group, MIT, a 124B vision MoE with 5.5B active), whose 54 GB IQ3_XXS build fits a 64 GB Mac (llama.cpp only, no MLX yet), Score 70. Step-3.7-Flash (StepFun, Apache 2.0, 198B MoE with 11B active), for 96–128 GB Macs, 48 tok/s community-measured on an M5 Max at 2-bit; it supersedes Step-3.5-Flash, which stays listed; Score 64. dots3-note preview (Xiaohongshu (RedNote), Apache 2.0, 280B MoE with 16B active), 117.5 GB at IQ3_XXS, so a 128 GB Mac; Score 62. Inkling 2 Small (Thinking Machines, Apache 2.0, 594B MoE with 17B active, weights 12 November), 226 GB at UD-IQ3_XXS, tight on a 256 GB Mac Studio, listed with a runtime caveat; Score 68. Nemotron 3 Ultra 550B-A55B (NVIDIA, OpenMDW; released 3 June and missed until now), 223 GB at UD-IQ3_XXS, also tight on 256 GB; Score 56. Mistral Large 4 (Mistral, Mistral Ecosystem Open-Weight, about 1.05T with 49B active, weights 29 October): Q2_K is 381 GB, so only the 512 GB Mac Studio holds it; 15 tok/s published on a 512 GB M3 Ultra at Q2_K and about 21 estimated on an M5 Ultra; Score 58 (details). Hy4 preview (Tencent, Apache 2.0, 770B MoE with 49B active), 289.5 GB at about 3 bits, so 512 GB, with a runtime caveat because its llama.cpp support has been validated only on CUDA; Score 63. Kimi K3.1 Pro (Moonshot AI, a custom licence with Kimi K3’s clauses, 2.8T MoE with 104B active, weights 2 November) is server-class: even its 1-bit GGUF exceeds 512 GB, so it has no Mac figure; Score 58. Kimi K3 stays listed, marked superseded.

Model updates and runtime status. Recursive Flint now has qualifying runs on a full 40-core M5 Max, among them its first on stock llama.cpp: 129.6 tok/s with b11871 and Recursive’s Q4_1 GGUF on a MacBook Pro 16-inch (community, 26 October). With four oMLX runs on full M5 Max chips (77.4–132.5), its leaderboard figure is the lower median of the five, 123 tok/s community-measured, replacing the ~131 estimate that the binned 36 GB run had held up; the Score stays 80. The binned run now sits above the full chip’s figure, and both are shown, as measurements that disagree. DeepSeek V4.1 Flash loses its runtime caveat: upstream llama.cpp supports it from b11870 (PR #31086, merged 25 October), though no plain-decode Metal llama.cpp figure has been published. MiniMax-M3.1-Flash loses its caveat after full llama.cpp support for its sparse-attention layers landed in b12420 (PR #32301, merged 15 November); mlx-vlm 0.7.11 adds vision. Raon-1 mini preview loses its caveat: it runs in oMLX 0.7.3 and mlx-vlm 0.7.12 without remote code, and community runs of 114.4 and 129.1 tok/s on a 40-core M4 Max raise its M5 Max estimate from 99 to 123 tok/s (Score 77, #10). Ling-3.1-flash now points at inclusionAI’s 27 October “1019” checkpoint (Artificial Analysis Intelligence Index 41 → 44), which has public MLX builds, so its “no public MLX build” note is gone. In the other direction, Inkling and Inkling-Small gain a runtime caveat (their llama.cpp support is the unmerged PR #25731, stock mlx-lm cannot load them, and their MLX builds bundle their own loader), so neither can be a Best-by-Mac headline pick. Inkling 2 Small keeps its caveat until a released runtime has a Mac speed: llama.cpp merged its support late on 14 November (first in b12406), stock mlx-lm cannot load it (#2061), and oMLX runs it only from its main branch. Kimi K3 and Step-3.5-Flash are marked as superseded by Kimi K3.1 Pro and Step-3.7-Flash.

126 measured runs entered (26 published, 100 individual runs), so measured figures rise from 193 to 319 of 619 and cover 61 of the 130 models (was 48); for 35 models the leaderboard figure itself is a measurement. Beyond MiMo-V2.6-Flash and Flint, the ones that moved the index: the Czech site Vše Smart published two reviews with their setups stated (21 rows), among them GPT-oss 120B at 110 tok/s in llama.cpp and 94 in mlx-lm on a 128 GB M5 Max (MXFP4) and Qwen3-235B-A22B at 46 on a 256 GB M5 Ultra (MLX, 4-bit), all published. With a 92.0 tok/s oMLX run, GPT-oss 120B’s M5 Max figure is the lower median of four runs, 92 tok/s community-measured, and its Score rises from 64 to 68. The Mac rental firm Bohemian Silicon measured Qwen 3.5 397B-A17B at 61.6 tok/s on a 256 GB M5 Ultra (community; mlx-lm 4-bit, 224 GB peak) against an estimate of 36; Score 58 → 65. Hugging Face’s Mistral Large 4 launch post gives 15 tok/s on a 512 GB M3 Ultra (published, Q2_K, llama.cpp). Community runs on 32 GB M6 Macs add Qwen 3.6-35B-A3B (58.7 tok/s), Nemotron 3.5 Lightning (60.3), GPT-oss 20B (65.6), Gemma 4 26B-A4B (49.4) and GLM-4.7-Flash (46.6). The first M1 Pro row (LFM3-1.7B, 83.8 tok/s community-measured) takes the dataset to 18 chips. Still missing: a fully specified generation figure for the M6 MacBook Pro or iMac, and any plain-decode run on a 512 GB M5 Ultra.

New Macs, and per-Mac pages from 42 to 54. Apple announced the 14-inch MacBook Pro with M6 (from $1,999; 153 GB/s at 16 GB and 170 at 24–32 GB, the Mac mini M6’s chip, so its figures carry over) and the iMac with M6 (from $1,499; Apple says only “up to 170 GB/s”, and the index assumes 153 at 16 GB, as Apple specifies for its other M6 Macs) on 27 and 28 October; both shipped on 6 November. The MacBook Air M5 ($1,299, 153 GB/s), on sale since March, had been missing. Also new: the 48 GB MacBook Pro M5 Pro ($3,099), the 48 GB Mac Studio M5 Max ($3,099, the cheapest Mac with 614 GB/s) and the 48 GB Mac mini M4 Pro (used). The PROLED MacBook Pro uses the same M5 Pro and M5 Max chips and bandwidth, so it needs no configurations of its own, and there is still no M6 Pro, Max or Ultra. The Mac mini M4 and M4 Pro, the MacBook Air M4, and the Mac Studio with M4 Max or a 256 GB M3 Ultra are no longer sold new and are labelled previous generation; the 512 GB M3 Ultra is used-only, since Apple dropped that option around March. The 512 GB M5 Ultra is $22,299 (80-core GPU only); the 256 GB model is $9,499 with the 64-core GPU or $10,799 with the 80-core, both 1.2 TB/s. What the extra memory buys: Mistral Large 4 on a 512 GB Mac Studio.

What the new runs say about the estimator (methodology). Large Qwen MoEs ran well above their estimates on M5 Ultras: Qwen 3.5 397B-A17B at 1.7× (61.6 against 36) and Qwen3-235B-A22B at 1.6× (46 against 29). Across the 18 large MoEs that now have qualifying runs, measured speed lands at 0.26–0.88 of the unscaled formula (median 0.57). The 0.42 large-MoE factor is kept as a conservative default, but it is not a guaranteed floor: three models measured below it (DeepSeek V4.1 Flash, Qwen3.8-Flash-Next and Mistral Large 4). It now sets the figure only for large MoEs nobody has measured; a model with its own runs is scaled from them, which lifts Qwen3-235B-A22B’s M5 Max estimate from 16 to 26 tok/s. No constant was refit, and a separate factor for Qwen-family MoEs would need a held-out test first. The estimator can now also cost Q4_K_S and oMLX’s oQ5e, oQ6e and oQ8e builds.

Considered and not added. Fine-tunes and derivatives, none of which meets all four conditions: Arete-27B (Nous Research; a supervised fine-tune of Qwen3.8-27B with no standard-benchmark table against its base), Sage-A1.5 (built on Qwen 3.6-35B-A3B; no comparison the index can score, and its only Mac run used multi-token prediction), Thinklite-27B (an efficiency tune of Qwen3.8-27B that matches its base’s accuracy rather than raising it), Clover-35B-A3B-Preview (a diffusion model built on Qwen 3.6-35B-A3B, with vendor-run charts only), Nex-N2.6 (built on GLM-5.3-Flash; no licence file), Hydrogen-1.0 (a gated science assistant built on GLM-5.3; no evaluation against its base), Atria Dawn (built on GLM-5.2; no licence file and no Mac build of the final weights), Ornith-1.5 9B, 35B-A3B and 122B (the licence file the card links returns 404, and every evaluation is vendor-run), Intern-S2-397B (Qwen 3.5 397B-A17B’s configuration with an extended vocabulary; benchmarks published only as an image) and Ling-3.0-flash-Fin (finance continued training of Ling-3.0 Flash; no licence file). webAI Air-26B-A4B is a quantized build of Gemma 4 26B-A4B under a non-commercial licence, not a new model. Server-only or no practical Mac path: LongCat-2.5 (1.6T, MIT; published for NPU deployment and 1.34 TB in its smallest format), LongCat-2.0 (superseded and server-class), ERNIE-5.5-Flash-Base (a base checkpoint only, under a licence that excludes the EU, the UK and South Korea) and Kimi K3.1 Highspeed (API only). Size class and niche: MiniCPM5-1B and Spark-X2.6-1.7B (the index lists their larger siblings), Raon-1 nano (KRAFTON’s fork runtimes only; its upstream llama.cpp pull request opened on 13 November), SenseNova-Loop-9B-Preview (CUDA only), and draft, translation, image, OCR and world models. Unverifiable: Qwen 4 still has no weights (checked 14 November: Qwen/Qwen4-27B is not public, and the community “Qwen4-27B” repositories are empty placeholders), and nothing named GLM-5.5 or GLM-6 is on Hugging Face. October’s “seen, not yet evaluated” list, resolved: Hy4-preview, MiniCPM5-2B, dots3-note-prev, Step-3.7-Flash, Ling-3.0-tiny and Ling-3.0-flash-VL are added; MiniCPM5-1B, Intern-S2-397B, Ling-3.0-flash-Fin and LongCat-2.0 are not, for the reasons above; Mistral Large 4’s weights arrived on 29 October and it is listed. Seen, not yet evaluated: K2-Horizon-3.7B’s siblings, K2-Horizon-7B and K2-Horizon-MoVA-36B-A4B.

New and refreshed pages. Two posts: the November edition and Mistral Large 4 on a 512 GB Mac Studio. Twelve new Best-by-Mac pages (three each for the M6 MacBook Pro, the M6 iMac and the M5 MacBook Air, plus the three 48 GB configurations) and model pages for the new models and the newly measured model-and-chip pairs (251 live, was 209). /hardware (Apple prices, the M6 MacBook Pro and iMac, the PROLED model, the 512 GB Studio), the chip comparison, /methodology and more than twenty posts carry dated 14 November updates, among them the M5 Ultra, Qwen 3.6-35B-A3B #1, DeepSeek V4.1 Flash, MiniMax-M3.1-Flash, Muse Spark Open, Nemotron 3.5 Bolt and Recursive Flint posts, the Qwen 4 tracker, the October edition, the estimate-accuracy check and the 16 GB, 24–32 GB, coding and vision buying guides. /software and the Ollama, MLX and troubleshooting guides name the current releases: Ollama 0.40.16, LM Studio 0.4.28, Jan 0.8.8, oMLX 0.7.3 and llama.cpp 1.0.0 (b12359, 13 November), which Homebrew now ships, fixes for Nemotron 3.5 Bolt, MiniMax-M3.1-Flash and DeepSeek V4.1 Flash included.

Broken URLs fixed. Root-level requests for /benchmarks.json and /benchmarks.csv (96 404s from 15 October to 13 November) now redirect to the files under /data/, using two of Cloudflare’s 100 redirect rules (94 in use); a misspelt chip-comparison URL that crawlers kept requesting now forwards to the comparison through a static page; and a JavaScript template on /benchmarks no longer exposes an unfilled model-page link that crawlers followed.

Guards added to the build: when two or more models share the top Score, every sentence that names the #1 must say it is a tie and name the tie-break, and no sentence may claim a tie the ranking does not have; Ripple-65B Preview may never be attributed to Recursive, Flint’s maker.

24 October 2026 — Qwen 3.6-35B-A3B is #1 again, Recursive Flint and four more models, 92 new measurements

Catalog 116 models, 480 speed figures (47 published, 146 individual runs, 287 estimated) for 109 models on 17 Apple chips; data version 2026-10-24. Five models added, each traced to its Hugging Face card: Recursive Flint (Recursive, Apache 2.0, 39.8B MoE with about 2.5B active, 21 October — details; its capability score is estimated from the vendor’s coding benchmarks only), Raon-1 mini preview (KRAFTON, Apache 2.0, 28.8B MoE; listed with a runtime caveat, since it needs KRAFTON’s llama.cpp branch or remote-code MLX), Nemotron 3.5 Bolt E4B (NVIDIA, OpenMDW, released 23 October as announced — details), MiniMax-M3.1-Flash (MiniMax Community License, 428B MoE with 23B active, weights 23 October — details) and Spark-X2.6-4B (SparkLLM, Apache 2.0, 4.1B; its card’s vendor-run scores have not been reproduced). DeepSeek V4 Pro now points to DeepSeek’s official DeepSeek-V4-Pro-0813 weights.

92 measured runs entered (10 published, 82 individual runs), so measured figures rise from 101 to 193 of 480. The ones that moved the index: MiMo-V2.6-Flash has its first plain-decode M5 Ultra run, 61.7 tok/s community-measured on an 80-core, 256 GB Mac Studio (oMLX, 4-bit, MTP off), against the ~79 the index had scaled from M3 Ultra runs. A second run of 76.4 is displayed but does not count: its page reports a 64-core GPU with 256 GB, a configuration Apple does not sell. LFM2.5-24B-A2B ran at 113.5 and 122.0 tok/s on an M5 Pro (community), which makes it the pick for 64 GB Macs. The index’s first press figures for the M5 Pro come from Frandroid (Qwen 3.6-35B-A3B at 68.7 tok/s published, llama.cpp, 4-bit; with six plain-decode community runs from the Kapitsa engine’s developers, 75.7–95.6, its M5 Pro figure is the lower median of seven, 79.48 tok/s in Ollama, community-measured), and the Mac rental firm Bohemian Silicon posted six M5 Ultra runs in mlx-lm 0.32 (GPT-oss 20B at 184 tok/s and Qwen 3.5 35B-A3B at 162, community). Qwen3.8-Flash-Next has seven plain-decode M5 Max runs (28.5–69.7, community; six count, lower median 46.4); Recursive Flint has community llama.cpp runs of 92.4 tok/s on an M5 Pro and 79.6 on an M4 Pro (Q4_1); MiniMax-M3.1-Flash ran at 34.5 on an M5 Ultra and 25.3 on an M3 Ultra (oMLX, 3-bit, MTP off). DeepSeek V4.1 Flash ran at 38.7 tok/s on a 512 GB M3 Ultra with oMLX 0.7.2’s fused kernels (community, at a 16K-token prompt, so it is shown but does not anchor; MTP off), about twice September’s 20.23 at 4K. Its M5 Ultra estimate still follows the September run (~29 tok/s, Score 70).

A new #1. Under the reference-row rule the measured M5 Ultra run replaces MiMo-V2.6-Flash’s estimate: it falls from Score 82 to 78 (#7), and Qwen 3.6-35B-A3B is #1 (Score 81; 118 tok/s published on an M5 Max). It needs only 24 GB, so the #1 is also the best pick for a Mac with 32 GB or less; MiMo-V2.6-Flash remains the pick for 192–512 GB Mac Studios. Recursive Flint enters at #2 (Score 80) on an estimated ~131 tok/s on the M5 Max. The tier picks shared by Best by RAM, the homepage, llms.txt and Key Numbers are now: 8 GB Nanbeige4.2-3B, 16 GB Gemma 4 12B, 24 and 32 GB Qwen 3.6-35B-A3B, 64 GB LFM2.5-24B-A2B (was Qwen 3.6-35B-A3B), 128 GB Qwen3.8-Flash-Next and 192–512 GB MiMo-V2.6-Flash. Why the #1 moved and what would move it back: Qwen 3.6-35B-A3B is #1 again.

Estimator rules (methodology). (1) Coherence across a chip family: an estimate is never below a measured run of the same model on a slower configuration of the same chip family at equal or more bits, binned parts included, and never above a faster configuration’s figure; across families only the cap applies, and only when the faster chip ran a build of the same or fewer bytes per weight. Without it, scaling Recursive Flint from its nearest anchor, an M3 Ultra run, put the full M5 Max at 77–95 tok/s, while the binned 36 GB M5 Max measured 131; the Best-by-Mac pages showed the cut-down chip faster than the full one. Per-Mac figures now rise, or stay level, with bandwidth within a chip family, and validation checks it. (2) Natively quantized models: a GPT-oss run whose source did not state a quant is costed and counted as MXFP4, the format OpenAI ships; otherwise the first run labelled MXFP4 would have hidden the model’s other runs and lifted its 16 GB Mac mini M6 estimate from 41 to 115 tok/s. (3) The estimator can now cost Q4_1, FP16, IQ2_XXS and oQ2e builds; a row whose source states its bits per weight is costed from that.

Runtime status. llama.cpp merged the Metal fix for Nemotron 3.5 Bolt 74B on 20 October (PR #30699, release b11755), so stock builds no longer crash on it; NVIDIA’s own 24 GB GGUF still runs its NVFP4 layers on the CPU, Ollama still does not support the 74B on Apple Silicon, and its caveat is rewritten to say so. MiniMax-M3.1-Flash’s garbled llama.cpp output (issue #31063) is fixed from release b11861 (24 October); Homebrew’s llama.cpp 0.7.0 (b11669) predates that fix and the Bolt one.

Corrections. (1) Three pages said DeepSeek V4 Pro’s GA weights were unpublished; DeepSeek published them as DeepSeek-V4-Pro-0813 under MIT on 13 August. The pages carry dated corrections, and the August edition an editor’s note. (2) More than a dozen pages still called MiMo-V2.6-Flash #1 at Score 82 or about 79 tok/s, some in their FAQ structured data; they now carry the new figures. (3) Three troubleshooting guides showed speed tables with no source, including a CPU-versus-GPU table for models that do not fit the Mac it named; the tables now read their figures from the index, each labelled measured or estimated, and claims the index does not make are no longer attributed to it. (4) The methodology page said no cap applies across chip families; one does, as described above. (5) Ling-3.1-flash was described as having no published Mac measurement; one run exists (19.5 tok/s on a 512 GB M3 Ultra, on a private MLX conversion), and it still does not set the reference. (6) Key Numbers re-dated its Qwen 4 and Llama 5 answers on every rebuild; they now carry the date the claim was last checked.

Considered and not added. Ornith-1.5 35B and 122B (the licence file is missing and the evaluations are vendor-run); Sage-A1.5 (a fine-tune, pending a policy on fine-tunes, and its only Mac run used multi-token prediction); Atria Dawn (no Mac build yet); Tess-4.5-27B (scores tie its base model); Smaug-Nemotron-3.5 (no Mac files); Ling-3.0-flash-Sante (a domain fine-tune without transcribable scores). A MiMo-V2.6-Pro run on a 512 GB Mac and a Recursive Flint run on a 16 GB M1 Pro are quoted in posts but not entered: both are below the model’s minimum memory. Seen but not yet evaluated, queued for primary-source checks: Hy4-preview, MiniCPM5-2B and 1B, dots3-note-prev, Intern-S2-397B, Step-3.7-Flash, Ling-3.0-tiny, Ling-3.0-flash-Fin and -VL, and LongCat-2.0. Qwen 4 still has no weights (checked 24 October), and Mistral Large 4’s weights are due about 31 October.

New and refreshed pages. Four posts: Qwen 3.6-35B-A3B is #1 again, Recursive Flint on a Mac, Nemotron 3.5 Bolt E4B on a Mac and MiniMax-M3.1-Flash on a Mac Studio. The October edition, the Qwen 4 tracker, the Bolt, Muse Spark Open, DeepSeek V4.1 Flash and M5 Ultra posts, the estimate-accuracy check, the 16 GB, 24–32 GB, coding and vision buying guides, /hardware, /software and the Ollama and MLX guides gain dated updates (Ollama 0.40.11 is current; mlx-vlm 0.7.10; oMLX 0.7.2, with 0.7.3.dev1 out as a development release).

Guards added to the build: per-Mac figures may not fall as bandwidth rises within a chip family; a natively quantized model must keep all its measured anchors; a ternary or sub-2-bit model may not gain a row with a bandwidth-bound quant label; warnings for measured rows whose quant the estimator cannot cost, and for models whose reference is a run of unstated quant.

18 October 2026 — Muse Spark Open, twenty models, 79 new measurements, a new #1

Catalog 111 models, 383 speed figures (37 published, 64 individual runs, 282 estimated) for 104 models on 17 Apple chips; data version 2026-10-17. Twenty models added, each traced to its Hugging Face card or the lab’s own page: Muse Spark Open (Meta, 816B; weights 15 October; fits one 512 GB Mac Studio at 2–3-bit — details), Beam 1.0, Nemotron 3.5 Bolt 74B-A9B and Super VL 120B-A12B, LFM2.5-24B-A2B, North Mini Code 1.0, MiniCPM-V 4.7 35B-A3B, Mistral Medium 3.5, Ternary Bonsai 2 27B and 9B, Kolibri-1, Ling-3.1-flash, MiniMax-M3, MiniMax-M2.7, Granite 4.2 30B and 8B, Mellum2.1-12B-A2.5B-Thinking, MiMo-V2.6-Distill-Qwen-9B, Kimi K2.7 Code and Step 5 Preview (no Mac runtime yet). New licences are priced: Muse Model License, StepFun Model License and Mistral Modified MIT 5 points, MiniMax Community 4, MiniMax Non-Commercial 2. Repointed: DeepSeek V4.1 Flash now tracks DeepSeek’s 17 October “1017” checkpoint (DeepSWE v1.1 74.2 → 77.0, Mini-SWE-V2 harness); it runs on a 512 GB Mac Studio through oMLX’s 432 GB oQ4e build, so its minimum memory is 434 GB (was 315). MiMo-V2.6-Flash and MiMo-V2.6-Pro point to Xiaomi’s 27 September MOPD checkpoints, which supersede the RL release; the card scores they still quote are labelled as RL-checkpoint figures. Step 5 Preview and Solar Open 2 are listed without a speed: neither has an upstream llama.cpp or MLX runtime.

79 measured runs entered (28 published, 51 individual runs); two of them replace Maple Preview’s untraceable figures, so measured figures rise from 24 to 101 of 383. The ones that moved the index: KAT-Coder-V2.5 decodes at 89.3 tok/s community-measured on an M5 Max (oMLX, OptiQ 4-bit, MTP off), against the index’s estimate of 117, and its Score falls from 82 to 79. Qwen3.8-Flash-Next has its first plain-decode figures: 47.0 and 43.8 tok/s community-measured on an M5 Max (MTPLX 4-bit, MTP off; estimate 49) and 57.0 on an M5 Ultra (community, llama.cpp IQ4_NL; estimate 84). The first press figures for the Mac mini M6 come from tbreak’s review: 62.0 tok/s published for Qwen 3 30B-A3B and 53.4 for Qwen 3 4B (MLX, quant not stated); tbreak’s Mac Studio M5 Max review adds five more published rows. Apple ML Research published single-Mac Studio figures for both Ultras (8K-token prompt, 128 tokens generated): the M5 Ultra runs Qwen3.8-27B in BF16 at 19.7 tok/s against 12.5 on the M3 Ultra (1.58×) and Mistral Medium 3.5 in FP8 at 7.7 against 4.95 (1.56×), in line with MacStories’ published 48 against 31 (oQ4e, 1.55×). MiMo-V2.6-Flash ran at about 59 tok/s on a 256 GB M3 Ultra in two community 4-bit MLX runs, roughly twice the index’s estimate of 30.

A new #1. The leaderboard now shows a model’s lower-median measured run at 5 bits or fewer on its reference chip when one exists, and the estimate otherwise (the reference row). A run whose quant the source did not state counts only when no stated-quant run qualifies, and runs on unmerged or custom-loader code, SSD-streamed runs and runs on binned chips never set the reference. Re-anchored on two community runs of about 59 tok/s on a 256 GB M3 Ultra, MiMo-V2.6-Flash is #1 (Score 82; ~79 tok/s estimated on an M5 Ultra), but it needs a Mac Studio with 192 GB or more; at 32 GB or less the best pick is Qwen 3.6-35B-A3B (#2, Score 81). KAT-Coder-V2.5 is #3 on a community-measured 89.3 tok/s on an M5 Max. Best-by-Mac pages name a headline pick only if it reaches 15 tok/s on that Mac and has a standard Mac runtime (Nemotron 3.5 Bolt, DeepSeek V4.1 Flash, Beam 1.0, Ling-3.1-flash and Maple Preview, which run only through community ports, forks or untested Metal kernels, are listed and ranked but never headline), and the new Best by RAM page, the homepage, llms.txt and the new Key Numbers page share the same tier picks. How the estimates held up against the new runs: the accuracy check.

Amazon links were broken in browsers. From about 15 August to 17 October, a click on any Amazon link in a real browser landed on the 404 page instead of Amazon: browser navigations were answered by the static-asset layer before they reached the /go/az redirect Worker, while plain command-line requests still got the redirect, which hid the fault. /go/* now runs the Worker first, and every deploy checks that a browser navigation gets the redirect.

New and refreshed pages. Posts on M5 Ultra vs M3 Ultra vs M5 Max, the best local LLM for 16 GB and 24–32 GB Macs and vision models by RAM; a guide to Ollama’s MLX runner; two glossary terms (decision model, ternary quantization); an Atom feed for the blog. Rewritten or refreshed: the Qwen 4 pages, the coding, RAM and MoE explainers, M5 Max vs M4 Max (which now absorbs the Mac Studio M4 Max vs MacBook Pro M5 Max comparison), the Intel Mac, MacBook Air, Ollama and MLX guides, /software, and the October edition, which gains a dated update box.

Corrections. (1) Alibaba’s 22 September release says only that Qwen 4 is in training; the Max, Plus, Flash and open-weight 27B tiers several pages listed are not in it and are removed or struck through, and there is no Qwen 4.1. (2) Ollama 0.40, which runs supported models on MLX by default on Apple Silicon, was released on 25 September, not 4 October. (3) DeepSeek V4.1 Flash had been described as having no Mac runtime; oMLX has supported it since 11 September (20.23 tok/s community-measured on a 512 GB M3 Ultra). (4) Gemma 4 E2B and E4B have a 128K context, not 256K. (5) Kimi K2.6 and K2.5 minimum memory now comes from their smallest 3-bit GGUFs (429 and 417 GB). (6) Model pages no longer print invented ollama run tags; each shows a tag checked on ollama.com or a search link. (7) First-person testing claims were removed from nine pages; the May and June editions carry dated corrections, and their structured data no longer repeats the retracted answers. LLMCheck runs no tests of its own. (8) Maple Preview 20B-A1B’s 281 tok/s M5 Pro figure could not be traced to any DeepGrove source and is replaced by the figures in DeepGrove’s own mlx-lm fork: 387 tok/s on an M5 Pro and 169 on a Mac mini M4 with the exact output head (published). Maple runs only through DeepGrove’s forks, so it no longer headlines the 16 GB M4 Mac pages. (9) Runs on code from an unmerged pull request or a bundled custom loader (Kolibri-1’s only run, a custom MLX port of Qwen3.8-Flash-Next, an 8-bit Nemotron 3.5 Bolt port) are displayed but no longer anchor estimates, as the methodology says; pre-release builds of a released runtime count as released. Kolibri-1’s reference estimate moves from ~69 to ~75 tok/s (Score 66 → 68, #34 → #24). (10) Nemotron 3.5 Bolt gains a fully specified M3 Ultra run (56.5 tok/s community-measured in Rapid-MLX 0.16.2), which now anchors its Ultra-chip estimates. (11) Qwen3.8-Flash-Next is in Ollama’s library (qwen3.8-flash-next:125b-mlx, 105 GB), and Inkling-Small’s 80.2% SWE-bench Verified is no longer called the open-weight record: Beam 1.0’s card reports 81.6%. (12) Bonsai 27B’s three 1-bit estimates (30, 6 and 5 tok/s on the M5 Max, M4 and M2) are retracted: 1-bit decoding is limited by compute rather than memory bandwidth, so the bandwidth model cannot estimate it. The index now quotes only Prism ML’s published 66.4 (M5 Max), 44.2 (M5 Pro) and 26.0 (M4 Pro), and Bonsai 27B rises to #10. (13) Model pages said a model “needs about N GB” using the memory of the Mac in its speed row rather than the model’s own requirement, which was wrong on 106 of 109 pages; they now use the catalog minimum. (14) The open-data downloads were stale: benchmarks.csv still held the 23 September data (258 rows, no provenance column, 18 M4 Ultra rows), and the /data/v1/ files dated 11 July still listed models removed in the August audit (Llama 5, Phi-5, Qwen 4 and others). Both are now rebuilt from benchmarks.json on every update: 383 rows, each with its provenance.

Considered and not added. Hy-Memory-30B-A3B (Tencent: a memory-writing specialist with no general benchmarks); MAI-Gym-9B (Alibaba Tongyi: a GUI-agent checkpoint whose framework is unreleased); LAR-27B (Amazon: an unreplicated 84.2% SWE-bench Verified with an undisclosed harness, and no quantized builds); Granite Switch 4.1 (IBM: a repackaging of Granite 4.1); Comet 60B-A12B (Multiverse Computing: no Mac build); NVIDIA Nemotron-Labs Dory 1.0 4B (its llama.cpp support is a draft pull request, untested on Metal); MiniCPM-V 4.7 1B and Ternary Bonsai 2 2B and 4B (small; covered in posts only). Qwen 4 and Mistral Large 4 have no released weights, so neither is in the catalog. Runs that did not state their setup fully were left out, among them a Kimi K2.7 Code figure from an app that does not report its decode mode.

Guards added to the build: every measured row must sit under its own bandwidth ceiling; a dated monthly edition may not carry build-filled speeds; the sitemap must list exactly the live canonical pages, and a page’s lastmod moves only when its content changes; every catalog licence must be priced identically on the leaderboard and in the compare tool; every robots.txt group must repeat every Disallow; every “#1” sentence must also name the best pick at 32 GB or less when the #1 needs more; _redirects warns above 95 rules.

5 October 2026 — MoE refit, the M6 measured, and corrections to the hardware data

The first Mac mini M6 measurements, and a refit. A community benchmark of a 32 GB Mac mini M6 (raw data published) and an M5 Max run of Qwen 3.6-35B-A3B showed small mixture-of-experts models decoding two to four times faster than the index estimated; dense models matched within a token per second. Small MoEs (40B total or less) now use the two-term model with a fitted ~5 ms per-token overhead (six runs, three chips, mean error ~12%); large MoEs keep the conservative 0.42 factor; a model’s own measurement anchors its other chips. KAT-Coder-V2.5 (82) and Qwen 3.6-35B-A3B (81) now lead the leaderboard. Estimated rows no longer carry a time-to-first-token, which the estimator never modelled. Details on /methodology and in the October edition.

Catalog 91 models, 244 speed figures (11 published, 13 individual runs, 220 estimated; data version 2026-10-05). Added, verified on their Hugging Face cards: GPT-oss 20B (OpenAI, Apache 2.0, 21B-A3.6B) and Gemma 4 12B (Google, Apache 2.0). GPT-oss 120B had been costed as a dense model; it is a 117B MoE with 5.1B active, and its figures are corrected. MiniMax M2.5 gains estimates on the Ultras it actually fits.

Corrections. (1) Apple’s M5 bandwidths: the MacBook Pro configurations carried the M5 Pro at 273 GB/s and the M5 Max at 600 (Apple: 307 and 614), and those values fed the estimator. The efficiency constant was refit (0.801 → 0.783), so M5 Max estimates are unchanged, M5 Pro estimates rose up to 12% and other chips moved about 2%; binned configurations (36 GB M5 Max at 460 GB/s, 16 GB M6 at 153) now use their own figures. Sixteen pages quoting the old figures, and several quoting the base M3 at 200 GB/s (Apple: 100), are corrected with dated notes. (2) There is no M4 Ultra. Apple never made one, but the index listed a “Mac Studio M4 Ultra” with 18 estimated rows, two Best-by-Mac pages, Advisor configurations and Amazon links. All removed; its URLs redirect to the M5 Ultra, and about twenty articles that cited it carry corrections. (3) The Mac Advisor scaled every model as if the index’s reference speeds were M3 Max figures, overstating M5 Max and Ultra configurations by 1.5–2×; it now reads the same per-Mac figures as the Best-by-Mac pages, with provenance. (4) From 23 September the leaderboard ranked MiMo-V2.6-Flash #1 on an M5 Ultra estimate shown in its M5 Max column; models too big for any M5 Max are now ranked on the M5 Ultra and tagged. (5) Seven articles attributed figures to tests LLMCheck never ran, and an M5 comparison presented an impossible table as its own Ollama measurements; LLMCheck runs no tests of its own, and the wording and tables are corrected. (6) /hardware, the homepage picks, and two M5 guides now read their speeds from the index at build time instead of hand-copying them.

Guards added to the build: every chip must be one Apple made; M5 bandwidths must match Apple’s; no page may sell or describe an M4 Ultra; no estimated row may carry a TTFT; no MoE estimate may beat its active-weight bandwidth ceiling; estimates must rise with bandwidth; the compare tool must match the catalog; the Associates tag may appear only in the redirect Worker.

23 September 2026 — the first measurements, four models, an estimator fix

The Macs shipped and the first reviews are in. BGR measured Qwen3.8-27B at ~55 tok/s on an M5 Ultra and ~40 on an M3 Ultra (estimates: 57 and 39) and Qwen 3.5 122B-A10B at ~80 and ~60. Those four figures enter the dataset as published measurements; the M5 Ultra estimate for Qwen3.8-27B is replaced. MacStories’ figures used multi-token prediction on long prompts and stay out of the plain-decode dataset. Write-up: how the estimates held up.

Catalog 89 models, 258 speed figures (9 published, 5 individual runs, 244 estimated; data version 2026-09-23). Added, each verified on its Hugging Face card: Qwen 3.5 122B-A10B (Apache 2.0, February 2026, missed until now), MiMo-V2.6-Flash (Xiaomi, 309B-A15B, MIT, omnimodal; 192 GB Mac Studio), MiMo-V2.6-Pro (1.02T-A42B, MIT, server-class) and DeepSeek V4.1 Flash (552B encoder-decoder, MIT — no Mac runtime yet). Details. Muse Spark: still no weights on Meta’s Hugging Face organization.

Integrity fixes. Provenance marks now mean one thing everywhere (filled square = published figure, vendor or press; hollow circle = one person’s run). Best-by-Mac pages mark every speed — estimates read from the dataset had rendered as bare numbers — and fill a chip without its own row from the model’s nearest row, fixing cases like an M2 Ultra outrunning an M5 Ultra. Nine rows labelled Q8_0 carried Q4 speeds above their own bandwidth ceiling; the estimator now costs each row at its labelled quantization and the leaderboard reference stays at 4-bit. Present-tense “we benchmark” wording removed from 44 generated pages and a handful of posts; the March M5 Max guide’s results table, presented as our own tests, now carries an editor’s note and the index’s estimates. /software adds oMLX.

7 September 2026 — September edition, four new models, MoE speeds recalibrated

Catalog 85 models, 248 speed figures (data version 2026-09-07). Added, each verified on its Hugging Face card: GLM-5.3 (Zhipu, 753B-A40B, weights 25 Aug under the custom GLM-5.3 License — MIT-like below a US$10B MaaS gate; #8), GLM-5.3-Flash (320B-A18B, MIT, the “Ox Alpha” model, 26 Aug; #10), Qwen3.8-Flash-Next (125B-A6B Qwen4 preview, Qwen Community 1.0 — coding-assistant and MaaS products need a separate license; its 4-bit MLX is 112 GB, a 128 GB-tier model; #13) and DeepSeek V4 Flash Vision-Exp (305B-A13B, MIT, experimental, weights 31 Aug; #3). Qwen3.8-27B is the new #1 (Score 71), as August’s edition predicted; its entry now carries the card’s SWE-Bench Pro 61.7 (vendor scaffold) and Terminal-Bench 2.1 73.0, plus the M5 Max row its description had quoted without one. Qwen3.8-Flash-Next is now the top pick on every 128 GB-and-up Best-by-Mac page.

MoE speed estimates recalibrated. Sparse models now use active-parameter bytes scaled by a factor of 0.42, fitted to the index’s one community-measured MoE row (DeepSeek V4 Flash, M5 Max, 39 tok/s); the existing MoE estimates already sat in the 0.24–0.53 band. One published figure changed: DeepSeek V4 Flash on M5 Ultra, ~80 → ~47 tok/s; the two 25 August Studio posts carry a dated note. Documented on /methodology. Three dense models that the compare tool mislabelled as MoE are corrected. September edition, plus model pages for GLM-5.3-Flash and Qwen3.8-Flash-Next.

Site structure, shipped 27 Aug – 7 Sep. /software now server-renders its 13 app cards (they were built client-side into empty containers, invisible to crawlers). /models/ consolidated: 92 pages render, the rest 301 or meta-refresh to the Best-by-Mac page for their chip. The glossary is 37 indexable term pages instead of one. Stale post-launch titles fixed sitewide; a nav overlap between 760 and 980 px fixed; /benchmarks no longer describes its estimated rows as measured runs. validate.py gained guards for each of these.

25 August 2026 — the M6 / M5 Ultra wave

Apple refreshed the desktops (newsroom, 25 Aug; ships 22 Sep): Mac mini with M6 (2nm, 153–170 GB/s, from $899) or M5 Pro (307 GB/s, from $1,699); Mac Studio with M5 Max (460–614 GB/s, from $2,499) or the quad-die M5 Ultra (1.2 TB/s, 96–512 GB, from $5,499). The index gained 12 Mac configs (44 total in the Advisor), 13 Best-by-Mac pages (44 total), 25 estimated benchmark rows for M6 and M5 Ultra (227 total, every one provenance-marked), and 25 new model×chip pages. Every new-machine speed is estimated from Apple’s published bandwidth via the two-term model — dense by formula, MoE scaled from each model’s existing reference row — and will be replaced by sourced figures as reviews land after 22 September. Launch coverage: the full analysis, the $899 M6 mini, the M5 Ultra.

15 August 2026 — speed model refitted, Qwen3.8-27B, click-tracked affiliate links

The speed model was refitted, and 134 estimates changed. An audit against our own published formula found that 63 estimated figures were physically impossible — they implied a model reading its weights faster than the memory bus can deliver them. Llama 3.1 8B was listed at 75 tok/s on an M4 whose bandwidth caps it at 26. The cause was a single efficiency multiplier, which cannot fit both a 30B model (genuinely bandwidth-bound) and a 3B one (dominated by fixed per-token cost). The estimator is now a two-term formula whose constants are solved from the two vendor-published Apple Silicon figures in the dataset, then validated against a third it had never seen: predicted 24.7 tok/s vs 24 measured, +3%. Full derivation on the methodology page. No measured figure was altered — sourced and community rows are never recomputed, and two that had been overwritten were restored. The benchmarks table also now carries a provenance mark on every speed cell; it previously shipped 202 numbers with none, which contradicted our own rule.

New entry. Qwen3.8-27B (Alibaba, Apache 2.0) went public on 14 August and enters the index at Score 71, narrowly ahead of Qwen 3.6-27B (70) — both are estimated at ~29–30 tok/s under the refitted model, so the capability score is what separates them. It is a 27.8B dense model with a native vision encoder and a 262K context window; at 4-bit it needs about 19 GB, so it fits a 24 GB Mac. It benchmarks above Qwen 3.6-27B on every published coding row. Nobody has published an Apple Silicon tok/s figure for it, so the speed cell is estimated from the bandwidth model (the 16 GB 4-bit build against 600 GB/s), never presented as a measurement. Catalog is now 81 verified models. GLM-5.3 was announced on 14 August but no weights exist; per the provenance rule it is not listed.

Affiliate links are now measurable. All 460 Amazon links moved from direct amazon.com hrefs to a first-party redirect at /go/az on our own Cloudflare Worker, which counts the click and then forwards. No third-party script, no cookie, no change to the privacy position. The Associates tag now lives in one place — the Worker — so it cannot be dropped by an edit, and the destination is rebuilt server-side from an allowlist, so the link cannot be pointed anywhere else. Every affiliate button now names Amazon in its own label.

Fewer dead ends. Benchmarks, Software, the leaderboard and both programmatic hubs previously had no route onward to the pages that answer “which Mac, then?” — each now ends with an explicit next step. robots.txt was also corrected: its per-crawler groups were Allow-only, which per RFC 9309 meant the disallow rules in the wildcard group were invisible to Googlebot, Bingbot and every AI crawler. Each group now carries them.

August 2026 — catalog audit & grand update

The correction. A full audit cross-checked every catalog entry against official sources (Hugging Face orgs, lab blogs, license files). Entries that could not be verified were removed: the “Qwen 4 / 4.1” family, “Llama 5” family, “Phi-5” family, “Gemma 4.5”, “DeepSeek R2/R3”, “Mistral Medium 4” & “Voyage” line, “Command R+ 2”, and “Grok 4 Open”. Misnamed entries were reconciled to their real counterparts (“GLM 5.2 Air” → GLM-4.5-Air; “Mistral Voyage 24B” → Mistral Small 3.2 24B; “DeepSeek R2” slot → DeepSeek V3.2-Speciale); licenses were corrected (Kimi K3 & K2.x are custom/Modified MIT, not plain MIT); affected pages carry correction notes and removed URLs redirect. Their benchmark rows were removed from the open dataset.

The update. Catalog now 80 verified models. Added the real July–August wave: Muse Glimmer 30B (Meta’s Apache 2.0 return), DeepSeek V4 Flash (runs on 128GB Macs), Inkling & Inkling-Small (Thinking Machines), Kimi K3 (weights, corrected license), Qwen3.8-2.4T-A95B, Qwen 3.6-27B (new verified Mac #1), Laguna S/XS 2.1, Nemotron 3.5 Lightning, KAT-Coder-V2.5, Ling-3.0 Flash, Hunyuan Hy3, Solar Open 2, Apertus 1.5, GLM-4.7-Flash, and the on-device wave (Bonsai 27B, Maple Preview, LFM2.5-2.6B, Nanbeige4.2-3B). Benchmarks: 202 rows (now labeled estimated / vendor-sourced / community per row) incl. new M1 Ultra, M2 Max and M3 Ultra coverage; harness annotations added to SWE-Bench Pro claims. Published the August State of Open-Source Local LLMs report and reissued July as a corrected edition.

July 2026

Expanded the leaderboard to 79 models, including several entries later removed by the August audit (see above). Launched Best LLM by Mac pages (ranked per chip + RAM). Published the July State of Open-Source Local LLMs report. Privacy hardening: removed all third-party analytics and fonts; added privacy & terms pages. Truth & provenance layer: every figure now labeled estimated/sourced/community with published estimation math; launched the Mac Advisor, Cite & Contribute pages and the open llmcheck-data repo.

June 2026

Published the June State of Open-Source Local LLMs report. Added new tok/s benchmark data across M4 and M5 chips and expanded the guides & troubleshooting hub.

May 2026

Published the May State of Open-Source Local LLMs report. Grand catalog refresh across model families and RAM tiers.

April 2026

Published the first monthly State of Open-Source Local LLMs report and expanded the Apple Silicon benchmark database.

See the current rankings

Every model in the catalog, ranked by speed, capability, RAM and license.

→ Open the Leaderboard