in Maximum Entropy and Bayesian Methods, Kluwer Academic Publishers, Dordrecht, ed. Van den Linden et. al., 1998
David R. Wolf
Abstract:
The topic of this paper is a novel Bayesian continuous-basis field representation and inference framework. Within this paper several problems are solved: The maximally informative inference of continuous-basis fields, that iswhere the basis for the field is itself a continuous object and not representable in a finite manner; the tradeoff between accuracy of representation in terms of information learned, and memory or storage capacity in bits; the approximation of probability distributions so that a maximal amount of information about the object being inferred is preserved; an information theoretic justification for multigrid methodology. The maximally informative field inference framework is described in full generality and denoted the Generalized Kalman Filter. The Generalized Kalman Filter allows the update of field knowledge from previous knowledge at any scale, and new data, to new knowledge at any other scale. An application example instance, the inference of continuous surfaces from measurements (for example, camera image data), is presented. The web links are to the long version, the local links are to the short version.
Download:
wolf_surface_maxent_98.pdf wolf_surface_maxent_98.ps.gz web0 web1
Entropy, Vol. 1, No. 3, pp. 69--98, 1999
David R. Wolf
Abstract:
The topic of this paper is a novel Bayesian continuous-basis field representation and inference framework. Within this paper several problems are solved: The maximally informative inference of continuous-basis fields, that iswhere the basis for the field is itself a continuous object and not representable in a finite manner; the tradeoff between accuracy of representation in terms of information learned, and memory or storage capacity in bits; the approximation of probability distributions so that a maximal amount of information about the object being inferred is preserved; an information theoretic justification for multigrid methodology. The maximally informative field inference framework is described in full generality and denoted the Generalized Kalman Filter. The Generalized Kalman Filter allows the update of field knowledge from previous knowledge at any scale, and new data, to new knowledge at any other scale. An application example instance, the inference of continuous surfaces from measurements (for example, camera image data), is presented.
Download:
wolf_bayref_entropy_99.pdf wolf_bayref_entropy_99.ps.gz web0 web1
Rev. R. Acad. Sci. Exacta. Fisica. Nat. - Monograph on Bayesian Methods in the Sciences, Vol. 93, No. 3, pp. 381--386, 1999
David R. Wolf and Edward I. George
Abstract:
In this paper we propose a Bayesian, information theoretic approach to dimensionality reduction. The approach is formulated as a variational principle on mutual information, and seamlessly addresses the notions of sufficiency, relevance, and representation. Maximally informative statistics are shown to minimize a Kullback-Leibler distance between posterior distributions. Illustrating the approach, we derive the maximally informative one dimensional statistic for a random sample from the Cauchy distribution. (Also presented Bayesian Statistics 6, Valencia, 1998)
Download:
wolf_maxinfo_arxiv_00.pdf preface.pdf preface.ps.gz wolf_maxinfo_arxiv_00.ps.gz web0 web1
Genomics, Vol. 13, pp. 1056--1064, 1992
J.W. Fickett, D.C. Torney, and D.R. Wolf
Abstract:
We model the base compositional structure of the human and Escherichia coli genomes. Three particular properties are first quantified: (1) There is a significant tendency for any region of either genome to have a strand-symmetric base composition. (2) The variation in base composition from region to region, within each genome, is very much larger than expected from common homogeneous stochastic models. (3) A given local base composition tends to persist over a scale of at least kilobases (E. coli) or tens of kilobases (human). Multidomain stochastic models from the literature are reviewed and sharpened. In particular, quantitative measurements of the third property lead us to suggest a significant shift in the style of domain models, in which the variation of A+T content with position is modeled by a random walk with frequent small steps rather than with large quantum jumps. As an application, we suggest a way to reduce the amount of computation in the assembly of large sequences from sequences of randomly chosen fragments.
Download:
in Maximum Entropy and Bayesian Methods, Kluwer Academic Publishers, Dordrecht, ed. Van den Linden et. al., 1998
David R. Wolf
Abstract:
The posterior moments of parameters specifying distributions are minimum mean square error Bayesian estimators for the corresponding moments of those parameters, and as such are ubiquitous in the Bayesian approach to statistical inference of distributions. The cauchy distribution is most notable for its wide tails, decided absence of high-order moments, and non-existence of less-than-data dimension sufficient statistics. Thus it vastly differs from the Gaussian distribution where tails are small, moments of all orders exist, and dimension-two sufficient statistics always exist. In this paper the posterior moments of the position parameter of the Cauchy distribution are found in closed form. (Estimating the width parameter is done using the same mathematics.) The interplay between the amount of data acquired for the estimation of the position parameter and the existence of higher order moments of the inferred posterior distribution for the position parameter is made explicit.
Download:
wolf_cauchymom_maxent_98.pdf wolf_cauchymom_maxent_98.ps
LA-UR, Vol. 93, April, 1993
David R. Wolf
Abstract:
The posterior moments of parameters specifying distributions are minimum mean square error Bayesian estimators for the corresponding moments of those parameters, and as such are ubiquitous in the Bayesian approach to statistical inference of distributions. The cauchy distribution is most notable for its wide tails, decided absence of high-order moments, and non-existence of less-than-data dimension sufficient statistics. Thus it vastly differs from the Gaussian distribution where tails are small, moments of all orders exist, and dimension-two sufficient statistics always exist. In this paper the posterior moments of the position parameter of the Cauchy distribution are found in closed form. (Estimating the width parameter is done using the same mathematics.) The interplay between the amount of data acquired for the estimation of the position parameter and the existence of higher order moments of the inferred posterior distribution for the position parameter is made explicit.
Download:
wolf_momcauchy_93.pdf wolf_momcauchy_93.ps.gz
Ph.D. Dissertation, University of Texas, Austin, Texas, USA, 1996
David R. Wolf
Abstract:
Information comes to the researcher or other system in untold forms. Information is carried in physical objects which interact with the observer or system that they influence. Living systems make use of information in subtle ways, to find and make use of sources of materials and energy. In this work the focus is on the reduced distribution functions and when they yield information of relevance. Information correlation functions, correlation functions, and entropy are of primary interest. The estimation of functions of the underlying distribution is examined. Several key theorems of statistical mechanics are shown to be consequences of a single theorem on counting labeled partitions, bringing together the cumulant expansion, linked cluster theorem, and Ursell development as consequences of this theorem. The information correlation functions provide a basis for the notion of the information between set of random variables. The flow of information is closely examined, in both the classical and quantum frameworks. In the Hamiltonian context, when the unexamined part of the distribution is taken to be the maxent distribution, the information flow into the subsystem is shown to be zero. The Ising model forms the basis of a non-trivial exactly solvable system for examining the correlations and information correlation functions. The expressions for the entropies of any subset of Ising spins are given, and it is shown that the information correlation functions give the mutual information between the first and last spins considered. The quantum Heisenberg model forms the basis for a non-trivial system exhibiting dynamics. The measurement entropy and the intrinsic entropy are defined and are shown to be related by an inequality. A time-ordered mutual information that is of great interest when examining the setting and measurement of quantum states is introduced. Estimating the values of functions of the underlying distribution (i.e. entropy, mutual information, etc.) and their uncertainties forms a large portion of the key results. Closed form expressions for the moments of the entropy and the mutual information are given.
Download:
dissert.html wolf_dissert_allchaps.tar.gz
LA-UR, Vol. 94, 1994
David R. Wolf
Abstract:
The problem of hypothesis testing is examined from both the historical and the Bayesian points of view in the case that sampling is from an underlying joint probability distribution and the hypotheses tested for are those of independence and dependence of the underlying distribution. Exact results for the Bayesian method are provided. Asymptotic Bayesian results and historical method quantities are compared and historical method quantities are interpreted in terms of clearly defined Bayesian quantities. The asymptotic Bayesian test relies upon a statistic that is predominantly mutual information.
Download:
wolf_mutind_94.pdf wolf_mutind_94.ps.gz web0 web1
in Review of Progress in Quantitative Nondestructive Evaluation, Plenum, New York, ed. D. O. Thompson and D. E. Chimenti, Vol. 14A, pp. 747-754, 1995
G. S. Cunningham, K. M. Hanson, G. R. Jennings, Jr., and D. R. Wolf
Abstract:
The Bayes Inference Engine (BIE) is a flexible software tool that allows one to interactively define models of radiographic measurement systems and geometric models of experimental objects so that the geometric properties of the objects being radiographed can be inferred from a limited amount of data. The BIE also allows a user to investigate confidence intervals on the estimated object geome try and compare the likelihoods of competing hypotheses. The BIE contains three components: a graphical programmer, for defining and interacting with the measurement system model, a geometric modeler, for defining and interacting with the object model, and an interactive optimizer. This article contains a description of these three components and an example of 2D geometry optimization from synthesized radiographic data using the BIE.
Download:
wolf_baytool_qnde_95.pdf wolf_baytool_qnde_95.ps.gz web0 web1
Physical Review E., Vol. 52, pp. 6841, 1992
David H. Wolpert, David R. Wolf
Abstract:
This paper addresses the problem of estimating a function of a probability distribution from a finite set of samples of that distribution. A Bayesian analysis of this problem is presented, the optimal properties of the Bayes estimators are discussed, and as an example of the formalism, closed form expressions for the Bayes estimators for the moments of the Shannon entropy function are derived. Then numerical results are presented that compare the Bayes estimator to the frequency-counts estimator for the Shannon entropy. In part II we also present the closed form estimators for the mutual information, chi 2 covariance, and some other statistics.
Download:
wolf_estent_pre_95.pdf wolf_estent_pre_95.ps.gz web0 web1
LA-UR, Vol. 93, No. 833, 1993
David R. Wolf
Abstract:
This paper is the second in a series of two on the problem of estimating a function of a probability distribution from a finite set of samples of that distribution. In the first paper, the Bayes estimator for a funciton of a probability distribution was introduced, the optimal properties of the Bayes estimator were discussed, and the Bayes and Frequency-counts estimators for the Shannon entropy were derived and graphically contrasted. In the current paper the analysis of the first paper is extended by the derivation of Bayes estimators for several other functions of interest in statistics and information theory. These functions are (powers of) the mutual information, chi-squared for tests of independence, variance, covariance, and average. Finding Bayes estimators for several of these functions requires extensions to the analytical techniques developed in the first paper, and these extension form the main body of this paper. This paper extends the analysis in other ways as well, for example by enlarging the class of potential priors beyond the uniform prior assumed in the first paper. In particular, the use of entropic and Dirichlet priors is considered.
Download:
wolf_estmut.pdf wolf_estmut.ps.gz web0 web1
in Maximum Entropy and Bayesian Methods, Kluwer Academic Press, Dordrecht, 1993
Kenneth M. Hanson and David R. Wolf
Abstract:
We discuss the properties of various estimators of the central position of the Cauchy distribution. The performance of these estimators is evaluated for a set of simulated experiments. Estimators based on the maximum and mean the posterior density function are empirically found to be well behaved when more than two measurements are available. On the contrary, because of the infinite variance of the Cauchy distribution, the average of the measured positions is an extremely poor estimator of the location of the source. However, the median of the measured positions is well behaved. The rms errors for the various estimators are compared to the Fisher-Cramer-Rao lower bound. We find that the square root of the variance of the posterior density function is predictive of the rms error in the mean posterior estimator.
Download:
wolf_estcauchy_maxent_93.pdf wolf_estcauchy_maxent_93.ps.gz web0 web1
LA-UR, Vol. 92, No. 4369, 1992
David H. Wolpert, David R. Wolf
Abstract:
This paper is the first of two on the problem of estimating a function of a probability distribution from a finite set of samples of that distribution. In this paper a bayesian analysis of this problem is presented, the optimal properties of the Bayes estimators are discussed, and as an example of the formalism, closed form expressions for the Bayes estimators for the moments of the Shannon entropy function are derived. Numerical results are presented that compare the Bayes estimator to the frequency counts estimator for the Shannon entropy.
Download:
wolf_estent_93.ps.gz wolf_estent_93.pdf web0 web1
in Proc. IEEE Int. Conf. Image Processing, IEEE, Vol. II, pp. 145--147, 1994
Hanson, G. S. Cunningham, G. R. Jennings, Jr., and D. R. Wolf
Abstract:
When dealing with ill-posed inverse problems in data analysis, the Bayesian approach allows one to use prior information to guide the result toward reasonable solutions. In this work the model consists of an object whose amplitude is constant inside a flexible boundary. The flexibility of the boundary is controlled by through a distortion energy. We present an example of reconstruction of the cross section of a blood vessel from just two projections.
Download:
wolf_flextom_icip_94.pdf wolf_flextom_icip_94.ps.gz web0 web1
in Medical Imaging, SPIE, Vol. 2167, pp. 914--923, 1994
Gregory S. Cunningham, Kenneth M. Hanson, George R. Jennings, Jr., and David R. Wolf
Abstract:
Object-oriented (OO) analysis, design, and programming is a powerful paradigm for creating software that is easily understood, modified, and maintained. In this paper we demonstrate how the OO concepts of abstraction, inheritance, encapsulation, polymorphism, and dynamic binding have aided in the design of a graphical-programming tool. The tool that we have developed allows a user to build radiographic system models for computing simulated radiographic data. It will eventually be used to perform Bayesian reconstructions of objects given radiographic data. The models are built by connecting icons that represent physical transformations, such as line integrals,exponentiation, and convolution, on acanvas. We will also briefly discuss ParcPlace's application development environment, VisualWorks, which we have found to be as helpful as the OO paradigm
Download:
wolf_oograph_spie_94.pdf wolf_oograph_spie_94.ps.gz web0 web1 web2
in Proc. IEEE Int. Conf. Image Processing, IEEE, Vol. III, pp. 826--830, 1994
G. S. Cunningham, K. M. Hanson, G. R. Jennings, Jr., and D. R. Wolf
Abstract:
We have described the implementation of a graphical programming tool in the object-oriented language, Smalltalk-80, that allows a user to construct a radiographic measurement model. The measurement model can be used to generate the measurements predicted by a given parameterized model of an experimental object. In this paper, we describe extensions to the graphical programming tool that allow it to be used to perform Bayesian inference on very large sets of object parameters, given real experimental data, by optimizing the likelihood or posterior probability of the parameters, given the real data.
Download:
wolf_oooptim_icip_94.pdf wolf_oooptim_icip_94.ps.gz web0 web1
Physical Review C, Vol. 45, No. 1, pp. 22--34, 1992
Gulmez, et. al.
Abstract:
23 p-d elastic scattering spin observables, most of them in linear combinations of two observables, were measured at 794 MeV over a range of four-momentum transfer t from -0.257 to -0.630 (GeV/c)2. By changing the beam polarization between S, N, and L type and by alternating the target vector polarization between negative and positive, six independent combinations of beam and target polarization were obtained. The target was dynamically polarized deuterated ammonia. An average target polarization of 30\% was achieved. Transverse polarization components of the scattered protons (S and N type) were measured directly by using a polarimeter. The longitudinal polarization component (L type) was measured after precessing it to N type. Typical uncertainties for two-spin coefficients were 0.02 and for three-spin coefficients 0.1–0.2. The data are compared with the theoretical predictions calculated by fitting the previously available data to a relativistic multiple-scattering model with derivative couplings. Although relativistic impulse approximation calculations do not predict the data well, they are also included in the comparisons.
Download:
wolf_pdpolar_prec_92.pdf wolf_pdpolar_prec_92.ps.gz web0
Entropy, Vol. 00, No. 027, 05, 2000
David R. Wolf
Abstract:
The topic of this paper is a novel Bayesian continuous-basis field representation and inference framework. Within this paper several problems are solved: The maximally informative inference of continuous-basis fields, that iswhere the basis for the field is itself a continuous object and not representable in a finite manner; the tradeoff between accuracy of representation in terms of information learned, and memory or storage capacity in bits; the approximation of probability distributions so that a maximal amount of information about the object being inferred is preserved; an information theoretic justification for multigrid methodology. The maximally informative field inference framework is described in full generality and denoted the Generalized Kalman Filter. The Generalized Kalman Filter allows the update of field knowledge from previous knowledge at any scale, and new data, to new knowledge at any other scale. An application example instance, the inference of continuous surfaces from measurements (for example, camera image data), is presented.
Download:
wolf_genkf_arxiv_00.pdf wolf_genkf_arxiv_00_fig.pdf wolf_genkf_arxiv_00.ps.gz wolf_genkf_arxiv_00_fig.ps.gz web0 web1
in Maximum Entropy and Bayesian Methods, Kluwer Academic Publishers, Dordrecht, ed. Ali Mohammed-Djafari, pp. 113--120, 1992
C.E.M. Strauss, D.H. Wolpert, and D.R. Wolf
Abstract:
A Bayesian analysis using an entropic distribution with hyperparameter alpha vs. evidence alpha estimation procedure, compared, contrasted. (Not original abstract).
Download: