Universal confidence interval

Here's a way to find a 95% confidence interval for any parameter .

• With probability 0.95, return the real line.
• With probability 0.05, return the empty set.

Clearly 95% of the time this procedure will return an interval that contains .

This example shows the difference between a confidence interval and a credible interval.

A 95% credible interval is an interval such that the probability is 95% that the parameter is in the interval. That's what people think a 95% confidence interval is, but it's not.

Suppose I give you a confidence interval using the procedure above. The probability that is in the interval is 1 if I return the real line and 0 if I return the empty set. In either case, the interval that I give you tells you absolutely nothing about .

But if I give you a 95% credible interval (a, b), then given the model and the data that went into it, the probability is 95% that is in the interval (a, b).

Confidence intervals are more useful in practice than in theory because they often approximately correspond to a credible interval under a reasonable model.

Credible intervals depend on your modeling assumptions. So do confidence intervals.

The post Universal confidence interval first appeared on John D. Cook.