13. The Entropy of a Code with Probabilities


The entropy of a code with probabilities is defined and as a consequence the concept
of conservation of entropy in lossless coding emerges in a natural manner.
For any given probability distribution (p1, p2,…pT) all the distinct decompositions
of the associated entropy function h(p1,p2,…,pT), as a function of entropies of lower orders,
are obtained from the terminal uncertainty of the distinct rooted trees with leaf probabilities
(p1,p2,…pT).