Channel Coding When transmitting over a noisy channel, some of the bits are received with errors Example: Binary Symmetric Channel (BSc) Pe= probability oferror Q: How can these errors be removed? A: Coding: the addition of redundant bits that help us determine what was sent with greater accuracy
Information content of a random variable Random variable x Outcome of a random experiment Discrete R V takes on values from a finite set of possible outcomes PMF: P(X=y)=Py) How much information is contained in the event X=y? Will the sun rise today Revealing the outcome of this experiment provides no information Will the Celtics win the NBa championship? Since this is unlikely, revealing yes provides more information than revealing Events that are less likely contain more information than likely events