AI & LLM Glossary for Mac Users
According to LLMCheck, terms like quantization, tokens per second, unified memory and mixture of experts decide which local AI model suits your Mac. This glossary defines 39 AI and LLM terms, from active parameters to VRAM, with Apple Silicon context and figures from the LLMCheck index that name their chip and provenance.
39 key terms explained with Apple Silicon context and figures from the LLMCheck index, each labelled measured or estimated.
A_
Active Parameters Architecture
In a mixture-of-experts model, the active parameters are the subset of weights used for each token. Qwen 3 30B-A3B holds 30B parameters but activates only 3B per token, so according to the LLMCheck index it runs at 62 tok/s published on a Mac mini M6 — small-model speed with 30B-class reasoning quality.
Apple Silicon Hardware
Apple's custom ARM-based chip family (M1 through M6) featuring Unified Memory architecture. LLMCheck defines Apple Silicon as the most efficient consumer hardware for local AI inference because CPU, GPU, and Neural Engine share the same memory pool — eliminating the CPU-to-GPU data transfer bottleneck that limits NVIDIA GPU setups for large models.
Attention Mechanism Architecture
The core component of transformer models that allows the AI to weigh the relevance of different parts of the input when generating each output token. Self-attention enables models to understand context across long passages. Longer context windows require more attention computation, which is why prompt processing speed matters on Apple Silicon.
C_
Context Window Model Spec
The maximum number of tokens a model can read and write in one conversation or prompt. Windows range from 4K tokens in older models to 10M tokens (Llama 4 Scout). Most Mac workflows need 8K–32K, and every extra token of context costs memory for the KV cache, so long contexts need headroom beyond the model file.
D_
Decision Model Concept
A model that answers a fixed question with a typed result, such as a choice, a probability or a score, in one forward pass instead of generating text. Liquid AI's d1-3B takes 30 ms per decision on an M5 Pro (vendor-reported). Ollama serves decision models through /v1/systemone and, since v0.40.4, ollama run --ask; LM Studio 0.4.26 added /v1/decisions.
Dense Model Architecture
A model architecture where every parameter is used for every generated token, so a dense model decodes slower than a mixture-of-experts model of similar size. According to the LLMCheck index, the dense Llama 3.3 70B needs 44 GB and runs at 13 tok/s published on an M5 Max, while the 35B-A3B MoE Qwen 3.6-35B-A3B runs at 118 tok/s published.
E_
Embedding Concept
A numerical representation of text as a vector (list of numbers) that captures semantic meaning. Embeddings enable similarity search, RAG systems, and semantic understanding. On Mac, embedding models like nomic-embed-text run locally via Ollama and are much smaller than generative LLMs — typically requiring only 1–2 GB RAM.
F_
Fine-Tuning Training
Further training a pre-trained model on your own data to improve it at a particular task or domain. On a Mac, practical fine-tuning means small models and parameter-efficient methods such as LoRA, using mlx-lm or the Hugging Face libraries. Full fine-tuning of large models still needs data-centre GPUs, which can be rented by the hour.
G_
Gemma 4 Model Family
Google DeepMind's Apache 2.0 open-weight family, released 2 April 2026, with function calling and text and image input (audio on E2B, E4B and 12B). Its five variants: E2B and E4B (128K context), and the 12B, 26B-A4B mixture-of-experts (3.8B active) and 31B (256K context). According to the LLMCheck index, Gemma 4 26B-A4B scores 78, ranking #6 of 116.
GGUF Format
GGUF is the single-file format for quantized models used by llama.cpp, Ollama and LM Studio. It replaced GGML in 2023 and stores the weights at a chosen quantization level (Q4_K_M, Q8_0 and others) together with the tokenizer and metadata. LLMCheck defines GGUF as the format Mac users should look for when downloading models for those apps.
GPU Cores Hardware
The parallel processors in an Apple Silicon GPU, which run LLM inference through Metal. More cores mean more compute for prompt processing: the M5 Max has up to 40 GPU cores and the M4 Pro up to 20. For generation speed, memory bandwidth is the stronger predictor, because decoding is limited by how fast weights stream from memory.
H_
Hallucination Concept
When an AI model generates plausible-sounding but factually incorrect information. All LLMs hallucinate to some degree. Larger, more capable models generally hallucinate less often, but no model is hallucination-free. RAG (Retrieval-Augmented Generation) reduces hallucination by grounding responses in retrieved source documents, and it works fully offline on a Mac.
I_
Inference Concept
The process of running a trained AI model to generate text, code, or predictions. LLMCheck defines local inference as running this process entirely on your Mac's hardware without any server communication. Inference speed is measured in tokens per second and depends primarily on memory bandwidth and model size.
L_
llama.cpp Software
An open-source C/C++ engine for running LLMs on consumer hardware, with Metal GPU support on Apple Silicon and the GGUF file format. LM Studio and many Mac apps build on it; since version 0.40, Ollama falls back to it for models its MLX runtime does not support. In one published M4 Max comparison, MLX decoded about 20% faster.
LLM (Large Language Model) Concept
An AI model trained on text to understand and generate language. According to the LLMCheck index, the top three local LLMs are Qwen 3.6-35B-A3B (Score 81), Recursive Flint (80) and KAT-Coder-V2.5 (79); on a Mac with 32 GB or less, the best pick is Qwen 3.6-35B-A3B (Score 81). Local LLMs run entirely on-device.
LLMCheck Score Metric
LLMCheck's 0–100 composite score for ranking local LLMs on a Mac: up to 50 points for capability (from published benchmarks), 25 for speed (tok/s on an M5 Max, or an M5 Ultra for models that need more than 128 GB), 15 for accessibility (minimum RAM) and 10 for licence openness. Higher means better overall suitability.
LM Studio Software
A free desktop app for running local LLMs on a Mac, with a chat interface, one-click model downloads and a local server. It runs GGUF models through llama.cpp and MLX models through Apple's framework, and needs no Terminal or coding knowledge. LLMCheck recommends it as the easiest starting point for beginners. Download it from lmstudio.ai.
LoRA (Low-Rank Adaptation) Training
A parameter-efficient fine-tuning method that trains small low-rank adapter matrices instead of changing the full model. Adapters are typically 10–100 MB, against a base model of several gigabytes, so they are quick to train and easy to swap. On Apple Silicon, mlx-lm supports LoRA training, which makes customizing a small model practical on a Mac.
M_
Memory Bandwidth Hardware
The rate at which data moves between memory and the processor, in GB/s, and the main predictor of generation speed on a Mac: every token reads the model's active weights from memory. Apple's figures: M5 Ultra 1,228 GB/s, M3 Ultra 819, M5 Max 614 GB/s, M4 Max 546, M5 Pro 307, M4 Pro 273, M6 170. Higher bandwidth means faster tok/s.
Metal Software
Apple's GPU programming framework that enables LLM inference on Apple Silicon GPUs. All major local AI apps (Ollama, LM Studio, llama.cpp, MLX) use Metal for GPU-accelerated inference. Metal's direct access to Unified Memory makes Apple Silicon uniquely efficient — there's no CPU-to-GPU memory copy overhead that slows down NVIDIA CUDA setups for oversized models.
Mixture of Experts (MoE) Architecture
A model architecture that splits its parameters into expert subnetworks and routes each token to only a few of them, so it needs a large model's memory but decodes like a small one. Gemma 4 26B-A4B activates 3.8B of 26B parameters per token and, according to the LLMCheck index, runs at 48 tok/s community-measured on a Mac mini M6.
MLX Software
Apple's open-source machine-learning framework for Apple Silicon. In a published comparison on an M4 Max, Qwen 3 30B-A3B decoded at 109.7 tok/s with MLX against 89.9 with llama.cpp, about 20% faster. Since version 0.40, Ollama runs the models MLX supports on MLX by default; to use it directly, install mlx-lm with pip install mlx-lm or Homebrew.
N_
Neural Engine Hardware
A dedicated block in Apple Silicon for machine-learning operations. Local LLM apps such as Ollama, LM Studio and MLX generate text on the GPU through Metal rather than on the Neural Engine. With the M5 generation Apple also put a Neural Accelerator in every GPU core, which speeds up prompt processing, the phase where M5 chips gain most for LLMs.
O_
Ollama Software
A free, open-source tool that runs local LLMs from the command line and serves them through an OpenAI-compatible API on localhost:11434. Since version 0.40, on Apple Silicon it runs models its MLX runtime supports on MLX by default, with llama.cpp as the fallback. Install it with Homebrew or the Mac app, then run a model, for example: ollama run gemma4:e4b.
P_
Per-Layer Embeddings (PLE) Architecture
A technique in Google's Gemma 4 E2B and E4B models that gives each decoder layer its own small embedding for every token. It lets E2B run with 2.3B effective parameters (5.1B including embeddings) and E4B with 4.5B (8B), which is what the E in their names means. PLE adds capability per active parameter; it is not a mixture-of-experts design.
Parameter Count Model Spec
The number of trainable weights in a neural network, typically measured in billions (B). Larger models generally have higher capability but require more RAM and run slower. According to LLMCheck's leaderboard, the sweet spot for Mac users in 2026 is 8–35B parameters — large enough for strong reasoning, small enough to run at usable speeds on 16–32 GB Macs.
Prompt Processing (Prefill) Performance
The phase in which a model reads your whole prompt before writing the first token, also called prefill. It is limited by GPU compute rather than memory bandwidth, so it is where newer chips gain most: Apple measured Qwen3.8-27B reading an 8,192-token prompt in 4.1 seconds on an M5 Ultra, against 18.2 seconds on an M3 Ultra.
Q_
Quantization Optimization
Storing a model's weights at lower precision than the 16 bits it was trained in. The LLMCheck index costs Q4_K_M at about 0.57 GB per billion parameters, so a 70B model drops from about 140 GB at 16-bit to about 40 GB at Q4_K_M. Fewer bits mean faster generation and a small loss of quality.
R_
RAG (Retrieval-Augmented Generation) Technique
Retrieval-augmented generation: before answering, the system retrieves passages from your documents and adds them to the prompt, so the model answers from your sources rather than from memory alone. It reduces hallucination on private data. On a Mac, both the embedding model that finds the passages and the LLM that answers can run locally, for example through Ollama.
S_
System Prompt Concept
A hidden instruction given to an LLM before the user's message that defines the model's behavior, persona, or constraints. System prompts consume part of the context window — typically 100–2,000 tokens. When running local LLMs on Mac, you have full control over system prompts with no restrictions, unlike cloud AI services that enforce their own system prompts.
T_
Temperature Parameter
A sampling parameter that controls randomness in model output. Temperature ranges from 0.0 (deterministic, always picks the most likely token) to 2.0 (highly random). For coding and factual tasks, LLMCheck recommends temperature 0.1–0.3. For creative writing and brainstorming, 0.7–1.0 produces more varied output.
Ternary Quantization Optimization
Storing each weight as −1, 0 or +1 with a shared scale, instead of 4 to 16 bits. PrismML's Ternary Bonsai 2 27B fits a 27B model in a 7.2 GB file and runs at 47 tok/s published on an M5 Max. Ternary decoding is compute-bound, so the LLMCheck index does not extrapolate it to other chips.
Time to First Token (TTFT) Performance
The delay between sending a prompt and seeing the first generated token. It grows with prompt length and depends on prompt-processing speed, where newer chips gain most: Apple measured 18.2 seconds on an M3 Ultra and 4.1 seconds on an M5 Ultra for Qwen3.8-27B with an 8,192-token prompt. Short prompts start far sooner.
Token Concept
The basic unit of text an LLM reads and writes. One English token is about 0.75 words or 4 characters, and the word "LLMCheck" might split into 3 tokens. Token counts set context-window limits and generation speed: at 40 tokens per second, a model writes roughly 30 words per second of text.
Tokens Per Second (tok/s) Metric
The primary speed metric for local LLM inference: how many tokens a model generates per second. According to the LLMCheck index on an M5 Max: Phi-4 Mini ~168 tok/s estimated, Qwen 3 30B-A3B 131 tok/s published, Llama 4 Scout ~19 tok/s estimated, Llama 3.3 70B 13 tok/s published. Above about 30 tok/s, output feels like real-time conversation.
Top-p / Top-k Sampling Parameter
Sampling strategies that control which tokens the model considers when generating output. Top-p (nucleus sampling) considers the smallest set of tokens whose cumulative probability exceeds p (e.g., 0.9). Top-k considers only the k most likely tokens. LLMCheck recommends top-p 0.9, top-k 40 as a balanced starting point for most local LLM tasks.
Transformer Architecture
The neural-network architecture behind modern LLMs, introduced in the 2017 paper "Attention Is All You Need". Self-attention lets every token weigh every other token. Many current models are hybrids that keep some attention layers: Qwen 3.6 interleaves Gated DeltaNet linear-attention layers with standard attention, and NVIDIA's Nemotron 3.5 adds Mamba layers, which makes long contexts cheaper.
U_
Unified Memory Hardware
Apple Silicon's shared memory pool, which the CPU, GPU and Neural Engine all address directly. It is why a Mac can load models far larger than a consumer graphics card can: discrete GPUs carry 8–32 GB of VRAM, while a Mac Studio offers up to 512 GB. The trade-off is lower bandwidth than the HBM memory on data-centre GPUs.
V_
VRAM (Video RAM) Hardware
Memory on a discrete graphics card, where model weights must sit for fast GPU inference. Consumer NVIDIA cards carry 8–32 GB, which caps the size of model they can run without splitting it across cards. Apple Silicon has no separate VRAM: unified memory gives the GPU the whole pool, up to 512 GB on a Mac Studio.
Frequently Asked Questions
What is a local LLM?
A local LLM is a large language model that runs entirely on your own hardware, such as a Mac with Apple Silicon, without sending any data to external servers. According to the LLMCheck index, Macs with 8 GB to 512 GB of unified memory run everything from 3B-parameter models to 1T-parameter mixture-of-experts models; for a Mac with 32 GB or less the best pick is Qwen 3.6-35B-A3B.
What does tokens per second mean for AI?
Tokens per second (tok/s) measures how fast an AI model generates text. One token is roughly 0.75 words. According to the LLMCheck index, 30+ tok/s feels like real-time conversation, 10–30 tok/s is comfortable for most tasks, and below 10 tok/s can feel slow for interactive use. The fastest figure in the LLMCheck index is Maple Preview 20B-A1B at 387 tok/s on an M5 Pro (published, Ternary); its ternary weights decode through compute-bound kernels, so among bandwidth-bound models the fastest measured reference figure is LFM2.5-2.6B at 220 tok/s published on an M5 Max (Q4_K_M).
What is quantization in AI models?
Quantization stores a model's weights at lower precision than the 16 bits it was trained in, typically 4 or 8 bits. The LLMCheck index costs Q4_K_M at about 0.57 GB per billion parameters, so a 70B model drops from about 140 GB at 16-bit to about 40 GB and fits a 64 GB Mac. The quality loss at 4-bit is usually small.
What is Unified Memory and why does it matter for AI?
Unified Memory is Apple Silicon's shared memory architecture where CPU, GPU, and Neural Engine all access the same memory pool. Unlike NVIDIA GPUs where model weights must be copied between system RAM and VRAM, Apple Silicon eliminates this transfer bottleneck. According to the LLMCheck index, this makes Macs the most efficient consumer hardware for running large AI models that exceed typical GPU VRAM limits.
What is MoE (Mixture of Experts) in AI?
Mixture of Experts (MoE) is a model architecture in which only a fraction of the parameters work on each token. Qwen 3 30B-A3B activates 3B of its 30B parameters, so tbreak's M5 Max review published 130.6 tok/s for it against 24.9 for the dense Qwen 3 32B. That is why the best-ranked models for a 24–32 GB Mac are mixture-of-experts models, even though the most capable one that fits, Qwen3.8-27B, is dense.
What is GGUF format?
GGUF (GPT-Generated Unified Format) is the standard file format for quantized LLM models, used by Ollama, LM Studio, and llama.cpp. It replaced the older GGML format in 2023 and supports metadata embedding, multiple quantization levels, and vocabulary storage in a single file. When downloading models for Mac, GGUF is the format to look for.
What is memory bandwidth and why does it affect AI speed?
Memory bandwidth measures how fast data moves between memory and the processor, in GB/s. Generating each token reads the model's active weights from memory, so bandwidth sets the speed limit: the M5 Max (614 GB/s) has about three times the bandwidth of an M2 Pro (200 GB/s). It is the single most important spec for local AI speed on a Mac.
Find the Right Model for Your Mac
Use LLMCheck's leaderboard to compare all 116 models by speed, capability, RAM, and license.
View Leaderboard →