Figure 9.1a
is a demonstration of the fact that the Bayes' estimator is better than
any other estimator in the least mean-square sense; in particular it is
better than the frequency counts estimator. Figure 9.1a
shows the mean-squared error of the Bayes' estimator and the frequency
counts estimator when m=2 and the prior is uniform, where the mean-squared
error for any estimator 
is given by expression 9.11.
As is immediately seen the Bayes' estimator has a smaller mean-squared
error than the frequency-counts estimator for all N (consistent
with section 9.3.5).
Figure 9.1b
depicts the average sample variance, that is
of the Bayes' and frequency counts estimators. For a particular sample
size N, the Bayes' estimator has a smaller sample variance.
Figure 9.2
shows the sample averages of the estimators as functions of the sample
size N for various values of fixed 
,
that is
Figure 9.3
shows the same sample averages (equation 9.42)
of the estimators, but now as functions of the true
for various values of the sample size N.
It is of interest to note that for a particular range of
values and sufficiently large N, the sample average of the frequency-counts
estimator actually comes closer to the true entropy than does the sample
average of the Bayes' estimator (see figures 9.2d-f
and 9.3d-f]).
To see how this is possible in light of the fact that the Bayes' estimator
has lower mean-squared error, first note that
so that the mean-squared error is the sum of the mean sample variance
and the mean-squared bias. The left hand side of equation 9.43
is depicted in figure 9.1a.
The first integral on the right hand side is depicted in figure 9.1b.
The integrand of the last integral on the right hand side (excluding the
prior) appears in figure 9.2
as the square of the difference between the curve for the estimator, and
the value of
being estimated. This quantity favors the frequency-counts estimator for
some values of
for sufficiently large N; however the first integral on the right
more than compensates to give a result favoring the Bayes' estimator.
Figure 9.4
depicts the sample averages of the estimators' square differences from
true as a function of r for various sample sizes N,
The integral of expression 9.44
multiplied by the prior (here uniform), depicted for various N,
is shown in 9.1a.
Finally, figure 9.5
shows
for a fixed ratio (1 : 15) of observed counts 
,
as a function of the number of counts 
.
Note the increasing density placed upon the entropy S(1/16, 15/16)
as the counts N increase. The average of s according to this
density is the Bayes' estimator given the observations.
Figure 9.1: Top. Mean square error. Bot. Sample variance.
Figure 9.4: Average square error from true.
Figure 9.5: Posterior density of entropy.
