Posterior variance

A few days ago I wrote a post entitled Does additional data always reduce posterior variance?. In a nutshell, the answer is no, not always.

That led the previous post which looked at posterior means for three Bayesian models, showing how the posterior mean is a weighted average of the prior mean and the mean of the new data. The weights are precisions, which means something different for each model.

For the beta-binomial model, variance may increase when seeing unexpected data (details here), but precision always increases.

For the normal-normal model precision is the reciprocal of variance. Every new data point makes precision go up and posterior variance go down.

The Poisson-gamma model may be the most interesting. As stated in the previous post, if data has a Poisson distribution with parameter , and has a gamma(₀, ₀) prior distribution, then the posterior distribution on after observing k events over time t has a gamma(₀ + k, ₀ + t) posterior distribution. Therefore the posterior variance is

(₀ + k) / (₀ + t)^2.

Note the posterior variance is an increasing function ofk and a decreasing function oft. This means that the posterior variance increasesevery time an event is observed, and it decreases quadratically between observations.

Here's an illustration. I simulated data from a Poisson process with and used a gamma(1, 1) prior on . Here's a plot of the posterior variance.

The post Posterior variance first appeared on John D. Cook.