叉熵 交叉熵 S=<so> <s> It was the greatest buy of all </s> ·Hs(p1)=HP2)=H(p3)=,原因是: 所有的 unigrams除了p1(the),p(buy),andp(o都是 0 所有 bigram的概率都是0 所有 trigram的概率都是0. 我们希望使每个概率都是非零的交叉熵 • 交叉熵 – S = <s0> <s> It was the greatest buy of all </s> • HS (p1 ) = HS (p2 ) = HS (p3 ) =∞，原因是： – 所有的unigrams除了p1 (the), p1 (buy), and p1 (of) 都是 0 – 所有bigram的概率都是 0. – 所有trigram的概率都是 0. • 我们希望使每个概率都是非零的