Estimators for the first and second moments of the entropy

Define , the vector of all zeroes, except for a single one in the jth position.

. For , ,
 

Proof: Applying theorems 3, 4, and 5,

. For , ,
 

Proof: Similar to proof of theorem 7 noting that

In a similar manner, all higher moments of the entropy may be calculated by differentiation, leading to the observation that the posterior moment generating function of the entropy is given by

Note that for we have . Also note that , the frequency counts estimator.