What is Transformer?

The neural network architecture underlying all modern LLMs, introduced in the 2017 paper "Attention Is All You Need." Transformers use self-attention mechanisms to process input tokens in parallel rather than sequentially. Every model on LLMCheck's leaderboard — from Phi-4 Mini (3.8B) to Kimi K2.5 (1T+) — is based on the transformer architecture.