What is Quantization?

A compression technique that reduces model precision from 16-bit to lower bit-widths. LLMCheck defines quantization as the most important optimization for running large models on Mac. Q4_K_M (4.5 bits/param) reduces model size by ~75% with only 2–3% quality loss. Q8_0 (8 bits) preserves near-original quality at ~50% size reduction. A 70B model shrinks from ~140 GB (F16) to ~40 GB (Q4_K_M), fitting in 64 GB Unified Memory.