「xi1 xi Xi- m x= 0 norm x Layer Norm norm norm https://arxiv.org/ abs/1607.06450 residual a+b x1 mean m FC standard deviation o Self-attention Self-attention + norm norm … norm 𝑥1 𝑥2 ⋮ 𝑥𝐾 𝑥1 ′ 𝑥2 ′ ⋮ 𝑥𝐾 ′ mean 𝑚 https://arxiv.org/ abs/1607.06450 Layer Norm 𝑥𝑖 ′ = 𝑥𝑖 − 𝑚 𝜎 standard deviation 𝜎 + 𝒃 𝒂 𝒂 + 𝒃 FC residual 19