LA-UR, Vol. 93, No. 833, 1993

David R. Wolf
 
 

Abstract:

This paper is the second in a series of two on the problem of estimating a function of a probability distribution from a finite set of samples of that distribution. In the first paper, the Bayes estimator for a funciton of a probability distribution was introduced, the optimal properties of the Bayes estimator were discussed, and the Bayes and Frequency-counts estimators for the Shannon entropy were derived and graphically contrasted. In the current paper the analysis of the first paper is extended by the derivation of Bayes estimators for several other functions of interest in statistics and information theory. These functions are (powers of) the mutual information, chi-squared for tests of independence, variance, covariance, and average. Finding Bayes estimators for several of these functions requires extensions to the analytical techniques developed in the first paper, and these extension form the main body of this paper. This paper extends the analysis in other ways as well, for example by enlarging the class of potential priors beyond the uniform prior assumed in the first paper. In particular, the use of entropic and Dirichlet priors is considered.
 
 

Download:

wolf_estmut.pdf  wolf_estmut.ps.gz  web0  web1