How to Install MLX on Mac: mlx-lm Setup & First Model (2026)

What is MLX?

MLX is an array framework from Apple’s machine learning research team, built for Apple silicon. What matters for local LLMs:

• Unified memory. Arrays live in memory the CPU and GPU share, so MLX runs an operation on either device (currently the CPU and the GPU) without copying data between them.
• Metal GPU kernels, including kernels for quantized low-bit weights, and lazy evaluation that only computes what is needed.
• M5 GPU Neural Accelerators. On M5-family chips, MLX uses the Neural Accelerators inside the GPU when the Mac runs macOS 26.2 or later. Apple measured time to first token up to about 4× faster than an M4; generation, which is bound by memory bandwidth, improved about 1.2×.

mlx-lm sits on top: commands to generate, chat, convert and quantize models, fine-tune with LoRA, serve a local API and benchmark, all reading models straight from Hugging Face.

Prerequisites

• An Apple Silicon Mac, any M1, M2, M3, M4, M5 or M6. MLX does not run on Intel Macs.
• macOS 14 or newer, the floor Homebrew’s mlx formula sets; macOS 26.2 or newer to use the M5 Neural Accelerators.
• Python 3.11 or newer for mlx-lm 0.32.x (its pyproject.toml requires >= 3.11). Check with python3 --version; if it is older, brew install python gives you a current one, or skip Python entirely with the Homebrew install below.

Install MLX

With pip (this also installs MLX itself, 0.32.2 or newer):

pip install -U mlx-lm

Or with Homebrew, which packages mlx-lm 0.32.2 with its own Python:

brew install mlx-lm

mlx-lm’s README also lists conda install -c conda-forge mlx-lm. Check the install:

mlx_lm.generate -h
python3 -c "import mlx.core as mx; print(mx.__version__)"   # pip installs only

On 24 October 2026 the current releases are MLX 0.32.4 (14 October) and mlx-lm 0.32.2 (16 October).

Download a model

The quickest start is a ready-made MLX build from the mlx-community organization on Hugging Face; mlx-lm downloads it the first time you use it. A 4-bit Qwen 3.5 9B is about 6 GB:

mlx_lm.generate --model mlx-community/Qwen3.5-9B-4bit \
  --prompt "Explain the difference between TCP and UDP in simple terms"

On an 8 GB Mac use mlx-community/Qwen3.5-4B-4bit instead. Models are cached in your Hugging Face cache folder, so the second run starts at once.

To build your own MLX copy of a model that has none, convert and quantize it. This is the example from mlx-lm’s README; -q quantizes to 4-bit and the result is saved in ./mlx_model:

mlx_lm.convert --model mistralai/Mistral-7B-Instruct-v0.3 -q

Run inference

mlx_lm.generate prints the answer, then a Prompt line and a Generation line, each in tokens per second. What to expect for Qwen 3.5 9B at 4-bit, according to the LLMCheck index:

Using the Python API

Current mlx-lm takes sampling settings through a sampler, built with make_sampler; the old temp= argument to generate() is gone:

from mlx_lm import load, generate
from mlx_lm.sample_utils import make_sampler

model, tokenizer = load("mlx-community/Qwen3.5-9B-4bit")

messages = [{"role": "user", "content": "Write a Python function that checks for palindromes."}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)

text = generate(
    model,
    tokenizer,
    prompt=prompt,
    max_tokens=512,
    sampler=make_sampler(temp=0.7, top_p=0.9),
    verbose=True,
)

For streaming output, stream_generate takes the same arguments and yields the text as it is produced. The same API is the base for your own tools: chatbots, code assistants or document pipelines that never leave the Mac.

Interactive chat and a clean benchmark

# Multi-turn chat in the terminal
mlx_lm.chat --model mlx-community/Qwen3.5-9B-4bit

# Plain-decode speed: 512 prompt tokens, 128 generated, 5 trials
mlx_lm.benchmark --model mlx-community/Qwen3.5-9B-4bit -p 512 -g 128 -n 5

mlx_lm.benchmark runs no draft model, so its generation figure is plain decode, the same kind of number the LLMCheck index records. Apps that turn on multi-token prediction or DFlash report higher, prompt-dependent figures.

MLX vs Ollama vs LM Studio vs llama.cpp

Since Ollama 0.40 the question is less “which engine is fastest” than “how much control do you want”:

On speed, the cleanest same-Mac comparison the LLMCheck index draws on (arXiv 2601.19139, one M4 Max, 4-bit) found mlx-lm sourced107.4 tok/s against llama.cpp sourced89.9 tok/s on Qwen3-30B-A3B, a 19% lead; 4% on Qwen3-8B (sourced79.9 tok/s vs sourced76.9 tok/s); and a 14% deficit on Gemma 3 4B (sourced105.4 tok/s vs sourced123.2 tok/s). Decode speed is set mostly by memory bandwidth, so a faster Mac moves the number far more than a different engine. The Ollama on MLX guide shows how to check which engine Ollama picked.

• mlx-lm, for control. Pick it when you write Python, want exact settings, fine-tune with LoRA (see the fine-tuning guide) or need a trustworthy plain-decode benchmark.
• Ollama, for convenience. One install, one command per model, a local API that most apps understand, and MLX underneath for popular models. See the Ollama setup guide.
• llama.cpp, for reach. GGUF covers more architectures and quantization types than any other format, and new models often land there first.

Compare models on your hardware in the benchmarks, or browse the software directory for MLX front ends such as oMLX and LM Studio.