The entropy at order m is given by

where in the second equation
may be understood as the probability of any observed m bit state having b one bits - all such states have the same probabilty. If the distributions of equations 4.6 or 4.7 are taken for the distribution of the bit strings it is immediately obvious that because effectively the bits are treated by these distributions as independent (a large unobserved portion is assumed) that the entropy will simply be the entropy for a single bit times the number of observed bits. Thus, letting q be the probability that a single bit is one we have for the large string case
![]()
which is trivially verified for the binomial form
. For the Poisson form, note that effectively we have assumed that
while m q is constant. The argument is slightly trickier and the limit of m must be taken carefully, but the result is again that the entropy per bit is -q log(q) - (1-q) log(1-q).
Clearly, if the entropy is to show the effects of the finite unobserved portion then we must compute with the non-limiting case form of the distribution, equation 4.3.