medical tests


Study: Most Doctors Flunk Math Of Medical Test Accuracy


File under: “Does not inspire confidence.”

You’ve just been screened for a rare disease, one that only strikes 1 in 1,000 people. You test positive, your doctor tells you. Your heart drops into your stomach. “Is there any chance the test could be wrong?” you ask, your voice tinged with pleading.

“There’s a very small chance,” your doctor replies. “The test has a false positive rate of 5 percent. But that means there’s a 95 percent chance that you do have the disease.”

Bzzzzzzz. That’s the jarring sound of the game show buzzer that means “Wrong. Wrong. Wrong.”

A new study in the journal JAMA Internal Medicine posed just such a scenario to “24 attending physicians, 26 house officers, 10 medical students, and 1 retired physician” at a Boston‐area hospital. The hospital is not named, but all were affiliated with not-shabby medical schools: Harvard and Boston University.

And the vast majority blew the question. (Which was: “If a test to detect a disease whose prevalence is 1/1000 has a false positive rate of 5%, what is the chance that a person found to have a positive result actually has the disease, assuming you know nothing about the person’s symptoms or signs?”)

Close to half gave the answer “95 percent.” Not even close. The correct answer is 2 percent. (For an explanation of the math, check out Tom Siegfried’s excellent Science News post, which uses the analogy of a baseball player who flunks a drug test.)

The study — “Medicine’s Uncomfortable Relationship With Math” — replicated a similar math check done in 1978, and found little progress: 23 percent got the answer right in 2013, compared to 18 percent of a similar group in the 1978 study.

Yikes. What are patients — and doctors — to do about such medical innumeracy? I contacted the Boston-based Informed Medical Decisions Foundation, and spoke to research director Carrie Levin. Continue reading