transformers
3 posts tagged transformers.
Paper digest: Mixture-of-Experts
Mixture-of-Experts lets AI models grow much larger without costing much more to run. How it works, from Google's early papers to Mistral's Mixtral.
Paper digest: FlashAttention
FlashAttention made AI models faster and able to read longer text without changing their answers. How it works, and why it affects what AI costs you.
Attention Is All You Need, in plain English
The 2017 Google paper that introduced the Transformer, the design behind ChatGPT and most AI tools today. What it showed, how it works and its limits.