2.3.PROPERTIES OF THE ENTROPY FUNCTION 13 凸区域：若对于区域D中任意两点ā和3，ā∈D,3∈D,有 Aa+(1-A)3eD,0≤A≤1 则称D是凸区域。 凸函数：若定义在凸区域D上的函数f(x)满足 f(Aā+(1-a)≤f(a)+(1-)f( ∀a,3eD,0≤≤1 则称函数f(x)为凸-∩函数 m2h作衙e Theorem 2.3.2 (Jensen's inequality).If f is a conver-U function and X is a random variable,then E[f(x1≥f(E[X) Proof.For a two mass point distribution,the inequality becomes Pif(1)+p2f(x2)2 f(pi1+p2z2) which follows directly from the definition of convex function em is true for distributions withk-1mass points.Then for=1.2.we have ∑f)=mf)+1-)∑kf) ≥(+a-n是ka i=1 =f∑P) (2.10) =1 ◇ Property 2.3.2.H(p)is the conver function of p. IT-inequality:For a positive real number r, logr≤(r-1)loge (2.11) with equality if and only if (iff)r =1. Proof.Let f(r)=Inr-(r-1).Then we have fe)=-1adf"()=-<0 We can see that f(r)is convex-in the interval.Moreover,the maximize value is zero,which is achieved with r 1.Therefore, lnr≤r-1 with equality iffr=1.Equation(2.11)follows immediately2.3. PROPERTIES OF THE ENTROPY FUNCTION 13 凸区域：若对于区域D中任意两点α¯ 和β¯, ¯α ∈ D, β¯ ∈ D, 有 λα¯ + (1 − λ)β¯ ∈ D, 0 ≤ ∀λ ≤ 1 则称D是凸区域。 凸函数：若定义在凸区域D上的函数f(x)满足 f(λα¯ + (1 − α)β¯) ≤ λf(¯α) + (1 − λ)f(β¯), ∀α, ¯ β¯ ∈ D, 0 ≤ λ ≤ 1. 则称函数f(x)为凸- ∩ 函数. Theorem 2.3.1. If the function f has a second derivative which is non-negative (resp. positive) everywhere (f ′′(x) ≥ 0), then the function is convex-∪ (resp. strictly convex). Theorem 2.3.2 (Jensen’s inequality). If f is a convex-∪ function and X is a random variable, then E[f(x)] ≥ f(E[X]) Proof. For a two mass point distribution, the inequality becomes p1f(x1) + p2f(x2) ≥ f(p1x1 + p2x2) which follows directly from the definition of convex function. Suppose that the theorem is true for distributions with k − 1 mass points. Then writing pi ′ = pi 1−pk for i = 1, 2, . . . , k − 1, we have ∑ k i=1 pif(xi) = pkf(xk) + (1 − pk) ∑ k−1 i=1 p ′ i f(xi) ≥ pkf(xk) + (1 − pk)f( ∑ k−1 i=1 p ′ ixi) ≥ f ( pkxk + (1 − pk) ∑ k−1 i=1 p ′ ixi ) = f( ∑ k i=1 pixi) (2.10) Property 2.3.2. H(p) is the convex-∩ function of p. IT-inequality: For a positive real number r, log r ≤ (r − 1) log e (2.11) with equality if and only if (iff) r = 1. Proof. Let f(r) = ln r − (r − 1). Then we have f ′ (r) = 1 r − 1 and f ′′(r) = − 1 r 2 < 0. We can see that f(r) is convex-∩ in the interval r > 0. Moreover, the maximize value is zero, which is achieved with r = 1. Therefore, ln r ≤ r − 1 with equality iff r = 1. Equation (2.11) follows immediately