When I was a kid there was a popular movie quote going around, “Love means never having to say you’re sorry” which was the basis behind this T-shirt my wonderful mother got me. It means very different things to different people, “Statistics means never having to say you’re certain”. I like the shirt, but these differences are very interesting to me.
Some see it as the shortcoming of statistics. It is like that other quote from Benjamin Disraeli. I have my own saying, “Numbers don’t lie. People lie with numbers.” It is not the infallibility of statistics that is the problem. It is the people’s understanding of what statistics are about.
So, back to the quote “Statistics means never having to say you’re certain”. It is absolutely true. The whole idea behind statistics is making judgments about a population or future outcome based upon a subset, a sample. That may cause some people to be discouraged and paint a picture instead (more to come on that in a future post). But, what about the statement, “Sending someone to prison means never having to say you are certain”? Hmmm.
It is the same, we start with a ‘belief’, we collect data/evidence to support or refute that belief and then we make an inference/judgment. We never know for certain that our judgment is correct. In statistics, assuming we follow correct procedures and don’t have a hidden agenda up front (people can lie with numbers) then we do have a major advantage over that court case.
In statistics we can quantify our uncertainty. We can use a one sided 99% confidence interval, a 1% alpha level of significance in classical statistics. Or in Bayesian inference we can speak in terms of actual probability to quantify this uncertainty. However, in the court case we never specifically quantify this likelihood. In either case, we may not be certain, but in many situations it is the best thing we have.
