Entropy and Code Length
Entropy and Code Length
- Given R.V. X with H(X)
- Samples from X can be encoded in H(X) bits.
- Example:
- Biased coin comes up heads 1 time in 1024
- How long is a message describing 1 flip?
- How long is a message describing 1024 flips?
- Just encode the position of the heads....
H(x) = 11.4