Why do medical tests always have error rates?
Most people implicitly assume medical tests are infallible. If they test positive for X, they assume they have X. Or if they test negative for X, they're confident they don't have X. Neither is necessarily true.
Someone recently asked me why medical tests always have an error rate. It's a good question.
A test is necessarily a proxy, a substitute for something else. You don't run a test to determine whether someone has a gunshot wound: you just look.
A test for a disease is based on a pattern someone discovered. People who have a certain condition usually have a certain chemical in their blood, and people who do not have the condition usually do not have that chemical in their blood. But it's not a direct observation of the disease.
Usually" is as good as you can do in biology. It's difficult to make universal statements. When I first started working with genetic data, I said something to a colleague about genetic sequences being made of A, C, G, and T. I was shocked when he replied Usually. This is biology. Nothing always holds." It turns out some viruses have a Zs (aminoadenine) rather than As (adenine).
Error rates may be very low and still be misleading. A positive test for rare disease is usually a false positive, even if the test is highly accurate. This is a canonical example of the need to use Bayes theorem. The details are written up here.
The more often you test for something, the more often you'll find it, correctly or incorrectly. The more conditions you test, the more conditions you find, correctly or incorrectly.
Wouldn't it be great if your toilet had a built in lab that tested you for hundreds of diseases several times a day? No, it would not! The result would be frequent false positives, leading to unnecessary anxiety and expense.
Up to this point we've discussed medical tests, but the same applies to any kind of test. Surveillance is a kind of test, one that also has false positives and false negatives. The more often Big Brother reads you email, the more likely it is that he will find something justifying a knock on your door.
Related postsThe post Why do medical tests always have error rates? first appeared on John D. Cook.