Output Probabilities ↑ Softmax Seq2seq Linear Add Norm output sequence Feed Forward Add Norm Add Norm Multi-Head Encoder Decoder Feed Attention Forward Nx Nx Add Norm Add Norm Masked Input sequence Multi-Head Multi-Head Attention Attention Positional Positional Encoding ⊕ Encoding Input Output Embedding Embedding <EOS> Inputs Outputs Sequence to Sequence Learning with (shifted right) Neural Networks Transformer https://arxiv.org/abs/1409.3215 https://arxiv.org/abs/1706.03762 Seq2seq Encoder Decoder Input sequence output sequence https://arxiv.org/abs/1706.03762 Sequence to Sequence Learning with Neural Networks https://arxiv.org/abs/1409.3215 Transformer 15