Kimi K3
2 articles
Inside Kimi K3's Two Architectural Pillars: KDA and AttnRes, Explained From Zero
Beginner-friendly, analogy-driven walkthrough: KDA turns attention's 'open-book exam' into 'one page of smart notes' — 75% less KV-cache memory, 6.3x faster decoding at 1M tokens. AttnRes turns the residual stream's 'running ledger' into 'a notebook with an index' — 25% better training efficiency at under 2% overhead. By the end you'll know exactly how 2.8T parameters stay standing.
Kimi K3 Is Here: 2.8 Trillion Parameters, Fully Open Source — This Time It's Different
Moonshot AI releases Kimi K3: 2.8 trillion parameters, the world's first open-source 3T-class model, 1M-token context, and native multimodality. A deep dive into the KDA and AttnRes architecture innovations, its #4 global ranking on Artificial Analysis, pricing that matches Claude Sonnet 5, and the four long-term ways an open frontier model reshapes the industry.