Local Attention Truncated Attention Set to 0 Similar with CNN Calculate attention key weight Local Attention / Truncated Attention Calculate attention weight Set to 0 …… Similar with CNN key query