F(elm)s∑[ΣQ,(WR.w,ml,2 (3.27) To specialize (3.27)to a memoryless channel,we choose 0)=Π0(x) (3.28) That is.we are considering an ensemble in which each letter of each codeword is selected independently with the probability mass function (a).Then for a DMC,(3.27)becomes P.(ems∑∑Π∑Qx,WPy.lx,) -iE∑ex,hP0.I1x (3.29) Recognizing that x andyi are now dummy variables of summation,we can rewrite (3.29)as 2) m=1,2 (3.30) This is an upper bound on the average error probability over an ensemble of codes with two codewords of length N. To emphasize the exponential dependence on the code length N,we rewrite (3.30)as We are free to choose (x)so that we obtain the tightest upper bound Pa(elm)s2-NR where (3.31) Due to monotonicity of the log function,an equivalent expression for Ro is Ro=-l0g2 ΣewPo (3.32) The quantity that we have denoted Ro is usually called the cut-offrate of the DMC. For symmetric channels,an equiprobable distribution on the channel input alphabet, (x)=1/K,xe.Ax,achieves the extremum.In this case, Ro=log2 K-log2 可 In particular,for the BSC, 3-9 2 (| ) () ( | ) P em Q P B NN ⎡ ⎤ ≤ ⎢ ⎥ ⎣ ⎦ ∑ ∑ y x x y x , m=1, 2 (3.27) To specialize (3.27) to a memoryless channel, we choose 1 () ( ) N N n n Q Qx = x = ∏ (3.28) That is, we are considering an ensemble in which each letter of each codeword is selected independently with the probability mass function Q(ak). Then for a DMC, (3.27) becomes 1 2 1 ( | ) . ( ) ( | ) N n N B n nn y y nx P em Qx Py x = ⎡ ⎤ ≤ ⎢ ⎥ ⎣ ⎦ ∑ ∑ ∏∑ 2 1 () ( | ) n n N n nn ny x Qx Py x = ⎡ ⎤ = ⎢ ⎥ ⎣ ⎦ ∏∑ ∑ (3.29) Recognizing that xn and yn are now dummy variables of summation, we can rewrite (3.29) as 2 (| ) () ( | ) N B y x P em Qx Py x ⎧ ⎫ ⎪ ⎪ ⎡ ⎤ ≤ ⎨ ⎬ ⎢ ⎥ ⎪ ⎪ ⎣ ⎦ ⎩ ⎭ ∑ ∑ , m = 1, 2 (3.30) This is an upper bound on the average error probability over an ensemble of codes with two codewords of length N. To emphasize the exponential dependence on the code length N, we rewrite (3.30) as 2 2 log ( ) ( | ) (| ) 2 y x N Q x P yx P em B ⎧ ⎫ ⎡ ⎤ ⎪ ⎪ − −⎨ ⎢ ⎥ ⎬ ⎪ ⎢⎣ ⎥⎦ ⎪ ⎩ ⎭ ∑ ∑ ≤ We are free to choose Q(x) so that we obtain the tightest upper bound 0 (| ) 2 NR P em B − ≤ where 2 0 2 max log ( ) ( | ) Q y x R Qx Py x ⎧ ⎫ ⎪ ⎪ ⎡ ⎤ = −⎨ ⎬ ⎢ ⎥ ⎪ ⎪ ⎣ ⎦ ⎩ ⎭ ∑ ∑ (3.31) Due to monotonicity of the log function, an equivalent expression for R0 is 2 0 2 log min ( ) ( | ) Q y x R Qx Py x ⎧ ⎫ ⎪ ⎪ ⎡ ⎤ = − ⎨ ⎬ ⎢ ⎥ ⎪ ⎪ ⎣ ⎦ ⎩ ⎭ ∑ ∑ (3.32) The quantity that we have denoted R0 is usually called the cut-off rate of the DMC. For symmetric channels, an equiprobable distribution on the channel input alphabet, Q(x)=1/K, ∀x∈AX, achieves the extremum. In this case, 2 02 2 1 log log ( | ) y x R K Py x K ⎧ ⎫ ⎪ ⎡ ⎤ ⎪ = − ⎨ ⎢ ⎥ ⎬ ⎪⎩ ⎭ ⎣ ⎦ ⎪ ∑ ∑ In particular, for the BSC