Consider a system with m possible states and an associated m-vector of probabilities of those states, 
, 
, (
). The system is repeatedly and independently sampled according to the distribution 
. Let the total number of samples be N and denote the associated vector of counts of states by 
, , (
). The problem is to estimate a given function 
from 
, the samples. The functions considered are the entropy, mutual information, moments, average, variance, covariance and other correlations, and chi-squared.
Some previous work on estimating from , using frequency methods to generate correction terms, appears in [4, 21, 22, 31, 32, 33, 34, 51, 52, 61, 77].
Fully formal justifications of the manipulations carried out in this paper can be found as appendices 9.4-9.10.