TRANSFORMER EXPLAINER: Interactive Learning of Text-Generative Models
Abstract
Transformers have revolutionized machine learning, yet their inner workings remain opaque to many. We present TRANSFORMER EXPLAINER, an interactive visualization tool designed for non-experts to learn about Transformers through the GPT-2 model. Our tool helps users understand complex Transformer concepts by integrating a model overview and smooth transitions across abstraction levels of math operations and model structures. It runs a live GPT-2 model locally in the user’s browser, empowering users to experiment with their own input and observe in real-time how the internal components and parameters of the Transformer work together to predict the next tokens. 125,000 users have used our open-source tool at https://poloclub.github.io/ transformer-explainer/.
Cite
Text
Cho et al. "TRANSFORMER EXPLAINER: Interactive Learning of Text-Generative Models." AAAI Conference on Artificial Intelligence, 2025. doi:10.1609/AAAI.V39I28.35347Markdown
[Cho et al. "TRANSFORMER EXPLAINER: Interactive Learning of Text-Generative Models." AAAI Conference on Artificial Intelligence, 2025.](https://mlanthology.org/aaai/2025/cho2025aaai-transformer/) doi:10.1609/AAAI.V39I28.35347BibTeX
@inproceedings{cho2025aaai-transformer,
title = {{TRANSFORMER EXPLAINER: Interactive Learning of Text-Generative Models}},
author = {Cho, Aeree and Kim, Grace C. and Karpekov, Alexander and Helbling, Alec and Wang, Zijie J. and Lee, Seongmin and Hoover, Benjamin and Chau, Duen Horng (Polo)},
booktitle = {AAAI Conference on Artificial Intelligence},
year = {2025},
pages = {29625-29627},
doi = {10.1609/AAAI.V39I28.35347},
url = {https://mlanthology.org/aaai/2025/cho2025aaai-transformer/}
}