The total number of states (where there are B one bits among
the N bits) is
Now suppose an m bit state having b one bits is observed.
Fix the locations of the one bits in the observed state. How many microstates
correspond to this observed state? To find this, just distribute the remaining
N-b one bits among the remaining N-m locations.
Then each distinct N-m bit pattern is a microstate corresponding
to the observed state. The number of observed states with b 1-bits
in fixed locations of the m observed bits is thus
and therefore the probability of any observed state
having b 1-bits and m-b 0-bits is given by
Note that the probability of seeing some observed state of length
m having b one bits is
,
so that
Note that
is the hypergeometric distribution, the distribution giving the probability
that b balls of m balls are labeled one, where the m
balls are chosen randomly without replacement from an urn holding N
balls of which B are labeled one [43].
This allows us another approach to deriving the distribution
.
First note that drawing m balls from an urn randomly without replacement
is equivalent to randomly lining up all N balls, and taking the
first m of them. Then note that for each arrangement of the undrawn
balls there are C(m,b) distinct orderings of the m
drawn balls of which b are labeled one. Each of these orderings
is equally probable by the assumption of equal probability microstates,
so the probability of any one must be
.
Thus
,
as already derived in equation 4.3.
It is useful to express the hypergeometric distribution in the large
N limit when q := B/N is fixed. Doing this
shows that
becomes the binomial probability distribution for a fixed sequence involving
b ones and m-b zeros, and that the hypergeometric
distribution becomes the binomial distribution for bit sequences of length
m having b one bits,
Now, if we consider
and
then the factor
may be written
,
and in turn this may be approximated as exp(-m q). Similarly,
.
Finally, in this limit we have with
being the expected number of ones in m locations at rate q
that the hypergeometric distribution becomes the Poisson distribution
Thus, the hypergeometric, binomial, and Poisson distributions are simply
related - the binomial distribution is the hypergeometric distribution
for an urn with a large number of balls and a fixed fraction of one balls,
and the Poisson distribution occurs when additionally the fraction of one
balls in the urn is small. In the bit string system these cases correspond
to having a large unobserved portion of the bit string (binomial case)
and to having a small number of ones relative to the number of locations
of the bit string (Poisson case).