Home/Tools/Context memory calculator

Tool · context memory

How much memory does a 32K, 64K or 128K context need on a Mac?

A long context adds a KV cache on top of the weights. According to the LLMCheck index, Qwen 3.6-35B-A3B needs 24 GB at a short prompt and 26 GB at 64K tokens (16-bit cache); Qwen 3 30B-A3B, full attention in every layer, needs 18 and 25 GB. Pick a model to see which Macs fit.

Weights + runtime
estimated24.0 GB
the catalog’s minimum RAM: weights, about 2 GB of runtime and the first 4,096 tokens
KV cache at 64K
estimated1.3 GB
16-bit cache, 65,536 tokens; 1.3 GB beyond the first 4,096
Total needed
estimated25.3 GB
The smallest Mac that holds it by default has 36 GB (32 GB after raising the GPU limit).

total = 24.00 + KV(65,536) − KV(4,096) = 24.00 + 1.34 − 0.08 = 25.26 GB

Qwen 3.6-35B-A3B: hybrid: full attention on 10 of 40 layers; Gated DeltaNet layers keep a constant state · config.json · its Mac figures

Which Macs fit Qwen 3.6-35B-A3B at 64K

MemoryVerdictMacs with that memory
8 GB✕ too small17.3 GB shortOllama default: 4Kused: MacBook Air M1
16 GB✕ too small9.3 GB shortOllama default: 4KMac mini M6, MacBook Pro M6, iMac M6, MacBook Air M5 · used: Mac mini M4, MacBook Air M4, MacBook Air M1, MacBook Air M2, MacBook Air M3, Mac mini M2 Pro
24 GB✕ too small1.3 GB shortOllama default: 4KMac mini M5 Pro, MacBook Pro M5 Pro, Mac mini M6, MacBook Pro M6, iMac M6, MacBook Air M5 · used: Mac mini M4, Mac mini M4 Pro, MacBook Air M4, MacBook Air M2, MacBook Air M3
32 GBM5 and M6↑ fits after raising the GPU limitsudo sysctl iogpu.wired_limit_mb=28672Ollama default: 32KMac mini M6, MacBook Pro M6, iMac M6, MacBook Air M5
32 GBM1–M4↑ fits after raising the GPU limitsudo sysctl iogpu.wired_limit_mb=28672Ollama default: 4Kused: Mac mini M2 Pro, Mac Studio M2 Max
36 GB✓ fitswithin the default GPU limitOllama default: 32KMac Studio M5 Max
48 GB✓ fitswithin the default GPU limitOllama default: 32KMac mini M5 Pro, MacBook Pro M5 Pro, Mac Studio M5 Max · used: Mac mini M4 Pro
64 GB✓ fitswithin the default GPU limitOllama default: 256KMac mini M5 Pro, Mac Studio M5 Max, MacBook Pro M5 Pro, MacBook Pro M5 Max · used: Mac mini M4 Pro, Mac Studio M4 Max, MacBook Pro M3 Max, Mac Studio M1 Ultra, Mac Studio M2 Max
96 GB✓ fitswithin the default GPU limitOllama default: 256KMac Studio M5 Ultra
128 GB✓ fitswithin the default GPU limitOllama default: 256KMac Studio M5 Max, MacBook Pro M5 Max · used: Mac Studio M4 Max, MacBook Pro M3 Max, Mac Studio M2 Ultra, Mac Studio M1 Ultra
192 GB✓ fitswithin the default GPU limitOllama default: 256Kused: Mac Studio M2 Ultra
256 GB✓ fitswithin the default GPU limitOllama default: 256KMac Studio M5 Ultra · used: Mac Studio M3 Ultra
512 GB✓ fitswithin the default GPU limitOllama default: 256KMac Studio M5 Ultra · used: Mac Studio M3 Ultra

“Fits” means within macOS’s default GPU memory limit: the macOS 26 figure on M5 and M6 Macs, the lower macOS 15 figure on M1–M4 Macs, which may still run it. “Fits after raising the GPU limit” means within the safe raised limit the command sets, which leaves macOS at least 4 GB and lasts until the next restart (sudo sysctl iogpu.wired_limit_mb=0 restores the default). Over the limit, the runtime moves part of it to the CPU or shortens the context (llama.cpp, Ollama), runs it slowly (MLX), or stops with an out-of-memory error. Runtimes also need a compute buffer that grows with context, so a fit within about 1 GB is optimistic. The Ollama default is the context Ollama picks with no setting, from the default GPU limit: 4,096 tokens below 24 GiB, 32,768 from 24 GiB and 262,144 from 48 GiB, never more than the model’s window.

Settings for a 65,536-token context

Ollama
# Ollama app: the context-length slider in Settings; or, for the server (Terminal)
OLLAMA_CONTEXT_LENGTH=65536 ollama serve
# the Mac app also reads launchctl: set it, then quit and reopen Ollama
launchctl setenv OLLAMA_CONTEXT_LENGTH 65536
# one chat session
ollama run qwen3.6:35b-a3b
/set parameter num_ctx 65536

Without a setting, Ollama picks the context from the GPU memory it sees: 4,096 tokens below 24 GiB, 32,768 from 24 GiB and 262,144 from 48 GiB, never more than the model’s window. On a Mac that is the default GPU limit; the table above gives each Mac’s. Memory grows with OLLAMA_NUM_PARALLEL × context. ollama ps shows the CONTEXT and the RUNNER: its MLX engine never reads OLLAMA_KV_CACHE_TYPE (it keeps the cache at the model’s own precision and grows it as the chat grows).

llama.cpp
llama-server -m model.gguf -c 65536

-c reserves the cache for all 65,536 tokens when the model loads. Left at its default (0) it asks for the full 262,144-token window, and llama.cpp’s --fit (on by default) shrinks it toward 4,096 if that does not fit the GPU memory limit, so set -c to the context you need. 8-bit cache: add -fa on -ctk q8_0 -ctv q8_0.

LM Studio
lms load <model> --context-length 65536

Or set Context Length in the model’s load settings. K and V cache quantization sit in the same load settings; for GGUF models the V cache needs Flash Attention, and LM Studio has offered KV-cache quantization for MLX models since 0.3.9.

mlx-lm
mlx_lm.generate --model <mlx-model> --prompt "..."

mlx-lm needs no context setting: it grows the cache as the prompt grows instead of reserving it. --max-kv-size turns on a rotating cache only for models without their own cache layout; mlx-lm ignores it for models that define one. Leave it out to keep the whole context. --kv-bits 8 gives an 8-bit cache, except on sliding-window layers: mlx-lm cannot quantize its rotating cache yet, so Gemma and GPT-oss stop with an error.

estimated Every memory figure on this page is an estimate from the model’s own config.json, never a measurement (how the index computes context memory).

How the calculator works

A model needs its weights in memory plus a key and a value vector for every token in the context, in every attention layer: the KV cache. The catalog’s minimum RAM already covers the weights, about 2 GB of runtime and the first 4,096 tokens of cache, so a longer context adds only the cache beyond those tokens:

total = base + KV(context) − KV(min(context, 4,096))
KV(n) = (bytesPerToken × n + slidingBytesPerToken × min(n, window)) × bits ÷ 16 ÷ 109

base is the catalog minimum RAM for the build the index lists, or, for another quant, parameters × the GB per billion that quant costs (Q4_K_M 0.57, Q8_0 1.06, F16 2) + 2 GB. A mixture-of-experts model keeps every expert in memory, so its total parameter count is what counts. Ternary and sub-2-bit packs, and “E” models whose parameter count is an effective figure, offer only the catalog build.

bytesPerToken comes from each model’s Hugging Face config.json: 2 (key and value) × KV heads × head size × 2 bytes, summed over the full-attention layers. Sliding-window layers stop growing at their window; linear-attention and Mamba layers add nothing per token; latent-attention models (DeepSeek, Kimi) cache one compressed vector per layer. An 8-bit cache halves the KV term. The verdicts compare the total with macOS’s default GPU memory limit (the macOS 26 figure on M5 and M6 Macs, the lower macOS 15 figure on M1–M4 Macs) and with the limit safely raised.

Worked examples

Three models chosen by rule, not by hand, one per attention design: the index’s picks for 16 GB and 32 GB Macs and the highest-ranked full-attention model above 16 GB with an official Ollama tag, each at its catalog build. They are recomputed on every build.

Gemma 4 12B 12B · minimum 9 GB · 256K window

The index’s pick for 16 GB Macs (by RAM). Its cache costs 16 KB per token over its full-attention layers, plus 320 KB per token in sliding-window layers that keep at most 1,024 tokens (16-bit): 8 full layers (1 kv heads x 512) + 40 sliding layers (8 x 256, window 1024); attention_k_eq_v: K and V counted separately (a runtime that stores the shared tensor once needs half). Its Mac speeds →

ContextKV cache, 16-bitTotal, 16-bitTotal, 8-bit KVSmallest Mac (default limit)
4Kestimated0.4 GBestimated9.0 GBestimated9.0 GB16 GB
8Kestimated0.5 GBestimated9.1 GBestimated9.0 GB16 GB
32Kestimated0.9 GBestimated9.5 GBestimated9.2 GB16 GB
64Kestimated1.4 GBestimated10.0 GBestimated9.5 GB16 GB
128Kestimated2.5 GBestimated11.1 GBestimated10.0 GB16 GB
256Kestimated4.6 GBestimated13.2 GBestimated11.1 GB24 GB

Qwen 3.6-35B-A3B 35B MoE · minimum 24 GB · 256K window

The index’s pick for 32 GB Macs (by RAM). Its cache costs 20 KB per token over its full-attention layers (16-bit): hybrid: full attention on 10 of 40 layers; Gated DeltaNet layers keep a constant state. Its Mac speeds →

ContextKV cache, 16-bitTotal, 16-bitTotal, 8-bit KVSmallest Mac (default limit)
4Kestimated0.1 GBestimated24.0 GBestimated24.0 GB32 GB
8Kestimated0.2 GBestimated24.1 GBestimated24.0 GB32 GB
32Kestimated0.7 GBestimated24.6 GBestimated24.3 GB32 GB
64Kestimated1.3 GBestimated25.3 GBestimated24.6 GB36 GB (32 GB raised)
128Kestimated2.7 GBestimated26.6 GBestimated25.3 GB36 GB (32 GB raised)
256Kestimated5.4 GBestimated29.3 GBestimated26.6 GB48 GB (36 GB raised)

Qwen 3 30B-A3B 30B MoE · minimum 18 GB · 128K window

The highest-ranked model above 16 GB on the leaderboard that keeps full attention in every layer and has an official Ollama tag. Its cache costs 96 KB per token over its full-attention layers (16-bit): GQA on all 48 layers (4 kv heads x 128). Its Mac speeds →

ContextKV cache, 16-bitTotal, 16-bitTotal, 8-bit KVSmallest Mac (default limit)
4Kestimated0.4 GBestimated18.0 GBestimated18.0 GB32 GB (24 GB raised)
8Kestimated0.8 GBestimated18.4 GBestimated18.2 GB32 GB (24 GB raised)
32Kestimated3.2 GBestimated20.8 GBestimated19.4 GB32 GB
64Kestimated6.4 GBestimated24.0 GBestimated21.0 GB32 GB
128Kestimated12.9 GBestimated30.5 GBestimated24.2 GB48 GB (36 GB raised)

The settings, verified

Each flag was checked against the project’s own documentation or source on 19 November 2026:

FAQ

How much memory does a 64K context need on a Mac?

It depends on the model’s attention layout far more than its size. According to the LLMCheck index, Qwen 3.6-35B-A3B needs about 26 GB at 64K tokens with a 16-bit KV cache (24 GB at a short prompt), while Qwen 3 30B-A3B, with full attention in every layer, needs about 25 GB (18 GB short). The calculator does the same sum for 117 catalog models.

Does an 8-bit KV cache halve the memory?

It halves the cache, not the weights. At 128K, Qwen 3.6-35B-A3B drops from about 27 GB to 26 GB and Qwen 3 30B-A3B from about 31 GB to 25 GB. In llama.cpp use -ctk q8_0 -ctv q8_0 (the V cache needs Flash Attention), in Ollama OLLAMA_KV_CACHE_TYPE=q8_0 (GGUF models on its llama.cpp engine only), in mlx-lm --kv-bits 8 (not yet for sliding-window models such as Gemma and GPT-oss).

Why can a model that runs at a short prompt struggle at 128K?

Because the KV cache for the whole context sits in memory beside the weights: llama.cpp reserves it for the -c tokens when it loads the model, and Ollama for its context setting. When weights plus cache pass the GPU memory limit, the runtime moves part of it to the CPU or shortens the context (llama.cpp, Ollama), runs it slowly (MLX), or stops with an out-of-memory error. Set the context you need, not the largest the model supports.

How do I set the context length in Ollama, LM Studio, llama.cpp and mlx-lm?

Ollama: the context-length slider in its app’s Settings, OLLAMA_CONTEXT_LENGTH for the server (launchctl setenv for the Mac app) or /set parameter num_ctx in a chat; with no setting it picks from the GPU memory it sees, on a Mac the default GPU limit: 4,096 tokens below 24 GiB, 32,768 from 24 GiB and 262,144 from 48 GiB, never more than the model’s window. LM Studio: Context Length in the load settings, or lms load --context-length. llama.cpp: -c (its default, 0, asks for the full window and --fit shrinks it toward 4,096 if it does not fit). mlx-lm grows the cache as needed and takes no context setting.

What is the macOS GPU memory limit?

By default macOS lets the GPU use only part of unified memory. On macOS 26, which every M5 and M6 Mac runs, that is about 11.8 GB of 16 GB, 25.0 GB of 32 GB, 107.5 GB of 128 GB; macOS 15 allowed less (10.7, 21.3, 96.0), so the calculator uses the lower figure for M1–M4 Macs. Over the limit, the runtime moves part of it to the CPU or shortens the context (llama.cpp, Ollama), runs it slowly (MLX), or stops with an out-of-memory error. The command the calculator prints raises the limit, leaving macOS at least 4 GB, until the next restart.

Are these memory figures measured?

No. They are estimates: the KV cache is computed from each model’s own config.json (layers, KV heads, head size, sliding windows) and the weights from the bytes each quant costs. LLMCheck runs no tests of its own. Runtimes add a compute buffer that grows with context, so treat a fit within about 1 GB as optimistic.

Why do some models need so little memory for a long context?

Three designs keep the cache small: sliding-window layers hold only their last window of tokens (1,024 in Gemma 4 12B, 128 in GPT-oss 20B), hybrids with linear-attention or Mamba layers keep a fixed-size state in those layers (the Qwen3-Next design), and multi-head latent attention caches one compressed vector per layer (DeepSeek, Kimi).

Related

Cite this page

LLMCheck. (2026). Context Length Memory Calculator for Mac LLMs. llmcheck.net. https://llmcheck.net/tools/context-calculator/ (updated ). CC BY 4.0.

How to cite LLMCheck →Data: CC BY 4.0