Self-attention qi =Waai Q ki=Wkai Wk a a2aat K vi Wvai d 21Self-attention 𝒗 𝟏 𝒌 𝟏 𝒒 𝟏 𝒗 𝟐 𝒌 𝟐 𝒒 𝟐 𝒗 𝟑 𝒌 𝟑 𝒒 𝟑 𝒗 𝟒 𝒌 𝟒 𝒒 𝟒 𝒂 𝟒 𝒂 𝟑 𝒂 𝟐 𝒂 𝟏 𝒒 𝒊 = 𝑊𝑞𝒂 𝒊 𝒌 𝒊 = 𝑊𝑘𝒂 𝒊 𝒗 𝒊 = 𝑊𝑣𝒂 𝒊 𝒒 𝟏𝒒 𝟐𝒒 𝟑𝒒 𝟒 = 𝑊𝑞 𝒂 𝟏𝒂 𝟐𝒂 𝟑𝒂 𝟒 = 𝑊𝑘 = 𝑊𝑣 𝒂 𝟏𝒂 𝟐𝒂 𝟑𝒂 𝟒 𝒂 𝟏𝒂 𝟐𝒂 𝟑𝒂 𝟒 𝒗 𝟏 𝒗 𝟑𝒗 𝟒 𝒗 𝟐 𝒌 𝟏 𝒌 𝟑𝒌 𝒌 𝟐 𝟒 I I I 𝑄 𝐾 𝑉 21