Self-attention a1,2=q1.k2 1,3=q1.k3 1,4=q1.k4 attention score 01,2 01,3 01,4 q query k2 key k3 a Q2 Q3 q1 Waal k2 Wka2 k3 =Wka3 k4=Wka4 16 𝒒 𝟏 𝒌 𝟐 Self-attention 𝒂 𝟒 𝒂 𝟑 𝒂 𝟐 𝒂 𝟏 𝒒 𝟏 = 𝑊𝑞𝒂 𝟏 𝒌 𝟑 𝒌 𝟒 𝒌 𝟐 = 𝑊𝑘𝒂 𝟐 𝒌 𝟑 = 𝑊𝑘𝒂 𝟑 𝒌 𝟒 = 𝑊𝑘𝒂 𝟒 𝛼1,2 𝛼1,3 𝛼1,4 𝛼1,2 = 𝒒 𝟏 ∙ 𝒌 𝟐 query key attention score 𝛼1,3 = 𝒒 𝟏 ∙ 𝒌 𝟑 𝛼1,4 = 𝒒 𝟏 ∙ 𝒌 𝟒 16