A simple bit-string system of weakly interacting bits is defined. The system consists of a microcanonical ensemble of bit strings of fixed length, with a fixed number of 1 and 0 bits for each member of the ensemble.
A fixed portion of the string is observed. The fixed overall length of the whole string and the fixed number of ones and zeros in the whole string induces weak correlations between the observed bit locations.
The relationships between the hypergeometric probability distribution, which describes the probability of having some observation of the bit string system with a fixed number of bits, the binomial probability distribution, and the Poisson probability distribution is discussed.
The first and second order entropies for this simple bit-string system are then computed. Then asymptotic limits are found for both large strings and small bit density. The exact first and second order entropies are compared. How each changes as the bit density of the system is changed is found.
A phase transition in the bit string system is found. This is not the usual temperature-dependent transition, as the system is temperature independent and we are considering a microcanonical ensemble. It is a phase transition that occurs as the bit density is changed. Specifically, the derivative of the second order entropy diverges logarithmically at two transition bit densities in the range (0,1). This phase transition may be useful in understanding the hypothesized Kirkwood phase transition in a hard-sphere gas. The bit string system is identical to a lattice hard sphere gas on the simplex lattice. Higher order entropies exhibit phase transitions at increased numbers of densities. The third order entropy has a phase transition at four densities, etc.
The computation is generalized to entropies of arbitrary order, and the information correlation functions for the system are computed, their asymptotics are noted, etc. These functions thus characterize a system having one of the weakest forms of correlations possible - correlations induced by a single constraint on a single extensive variable.