Information comes to the researcher or other system in untold forms. Information is carried in physical objects which interact with the observer or system that they influence. Living systems make use of information in subtle ways, to find and make use of sources of materials and energy.
In this work the focus is on the reduced distribution functions and when they yield information of relevance. Information correlation functions, correlation functions, and entropy are of primary interest. The estimation of functions of the underlying distribution is examined.
Several key theorems of statistical mechanics are shown to be consequences of a single theorem on counting labeled partitions, bringing together the cumulant expansion, linked cluster theorem, and Ursell development as consequences of this theorem. The information correlation functions provide a basis for the notion of the information between set of random variables.
The flow of information is closely examined, in both the classical and quantum frameworks. In the Hamiltonian context, when the unexamined part of the distribution is taken to be the maxent distribution, the information flow into the subsystem is shown to be zero.
The Ising model forms the basis of a non-trivial exactly solvable system for examining the correlations and information correlation functions. The expressions for the entropies of any subset of Ising spins are given, and it is shown that the information correlation functions give the mutual information between the first and last spins considered.
The quantum Heisenberg model forms the basis for a non-trivial system exhibiting dynamics. The measurement entropy and the intrinsic entropy are defined and are shown to be related by an inequality. A time-ordered mutual information that is of great interest when examining the setting and measurement of quantum states is introduced.
Estimating the values of functions of the underlying distribution (i.e. entropy, mutual information, etc.) and their uncertainties forms a large portion of the key results. Closed form expressions for the moments of the entropy and the mutual information are given.