The Illustrated Transformer

Alammar, Jay

The Illustrated Transformer

Notable Works 2018

/misc/2018/alammar2018misc-illustrated/

Abstract

The Transformer was proposed in the paper Attention is All You Need. A TensorFlow implementation of it is available as a part of the Tensor2Tensor package. Harvard’s NLP group created a guide annotating the paper with PyTorch implementation. In this post, we will attempt to oversimplify things a bit and introduce the concepts one by one to hopefully make it easier to understand to people without in-depth knowledge of the subject matter.

Notable Works Semantic Scholar

Cite

Text

Alammar. "The Illustrated Transformer." Blog post, 2018.

Markdown

[Alammar. "The Illustrated Transformer." Blog post, 2018.](https://mlanthology.org/misc/2018/alammar2018misc-illustrated/)

BibTeX

@misc{alammar2018misc-illustrated,
  title     = {{The Illustrated Transformer}},
  author    = {Alammar, Jay},
  howpublished = {Blog post},
  year      = {2018},
  url       = {https://mlanthology.org/misc/2018/alammar2018misc-illustrated/}
}