nextuppreviouscontents
Next:High order entropies of Up:Constraint induced correlationsPrevious:The bit string system

Distribution of the observed states

We may easily compute the distribution of the observed states from the assumption that all microstates of the system with the same total number of bits B are equally probable.

The total number of states (where there are B one bits among the N bits) is
equation1250
Now suppose an m bit state having b one bits is observed. Fix the locations of the one bits in the observed state. How many microstates correspond to this observed state? To find this, just distribute the remaining N-b one bits among the remaining N-m locations. Then each distinct N-m bit pattern is a microstate corresponding to the observed state. The number of observed states with b 1-bits in fixed locations of the m observed bits is thus
equation1254
and therefore the probability of any observed state tex2html_wrap_inline13747 having b 1-bits and m-b 0-bits is given by
equation1257
Note that the probability of seeing some observed state of length m having b one bits is tex2html_wrap_inline13761, so that
equation1262
Note that tex2html_wrap_inline13763 is the hypergeometric distribution, the distribution giving the probability that b balls of m balls are labeled one, where the m balls are chosen randomly without replacement from an urn holding N balls of which B are labeled one [43]. This allows us another approach to deriving the distribution tex2html_wrap_inline13775. First note that drawing m balls from an urn randomly without replacement is equivalent to randomly lining up all N balls, and taking the first m of them. Then note that for each arrangement of the undrawn balls there are C(m,b) distinct orderings of the m drawn balls of which b are labeled one. Each of these orderings is equally probable by the assumption of equal probability microstates, so the probability of any one must be tex2html_wrap_inline13789. Thus tex2html_wrap_inline13791, as already derived in equation 4.3.

It is useful to express the hypergeometric distribution in the large N limit when q := B/N is fixed. Doing this shows that tex2html_wrap_inline13775 becomes the binomial probability distribution for a fixed sequence involving b ones and m-b zeros, and that the hypergeometric distribution becomes the binomial distribution for bit sequences of length m having b one bits,
eqnarray1273
Now, if we consider tex2html_wrap_inline13807 and tex2html_wrap_inline13809 then the factor tex2html_wrap_inline13811 may be written tex2html_wrap_inline13813, and in turn this may be approximated as exp(-m q). Similarly, tex2html_wrap_inline13817. Finally, in this limit we have with tex2html_wrap_inline13819 being the expected number of ones in m locations at rate q that the hypergeometric distribution becomes the Poisson distribution
equation1284
Thus, the hypergeometric, binomial, and Poisson distributions are simply related - the binomial distribution is the hypergeometric distribution for an urn with a large number of balls and a fixed fraction of one balls, and the Poisson distribution occurs when additionally the fraction of one balls in the urn is small. In the bit string system these cases correspond to having a large unobserved portion of the bit string (binomial case) and to having a small number of ones relative to the number of locations of the bit string (Poisson case).


nextuppreviouscontents
Next:High order entropies of Up:Constraint induced correlationsPrevious:The bit string system
David Wolf

Tue Mar 25 08:11:49 CST 1997