I Was Wrong!
Posted by
tgirsch
Let it not be said that I never admit it. See a good discussion on Bayes Theorem here and here and here — it is in this last thread where I admit I was wrong. 
This is also a good example of why I’m an ex-math major. I always sucked at statistics, and it was usually because of the kind of mistake I made here, wherein I misread and misunderstood the givens. My logic was sound, but it was based on a horrible misunderstanding of the premises. The bright side is, based on this, I’m halfway to becoming a successful right-wing pundit. I just need to eliminate the sound logic part.
If you find statistical inference interesting, I recommend the book “Against the Gods”. It covers the evolution of risk analysis through the ages. What amazed me the most is how recently all the major advances in the field have occured. Principles that a 12 year old playing Texas Hold’em on the internet takes for granted were unknown before the 18th century. Celestial mechanics, no problem. The odds of being dealt a straight in five card stud? Dunno. that’s up to “the gods”.
Comment 6/28/2006
Over at the Dawn Treader you say, “mammograms are horribly unreliable. Which sucks.” And it certainly does suck that we don’t have a perfectly reliable, non-invasive way of determining whether a woman has breast cancer. But the situation becomes a lot less horrible when you understand the difference between a screening test and a diagnostic test.
A similar situation applies in pre-natal screening for birth defects. There’s a screening test you can take that’s non-invasive, but has a lot of false positives. There’s a diagnostic test that has virtually no false positives, but it’s invasive. And because it’s invasive, it carries a risk of fetal death.
Let’s say — and these numbers are close but not quite right — that one in 200 fetuses in women who become pregnant over the age of 35 will carry a serious birth defect, that one in 50 diagnostic tests will result in miscarriage, and that the screening test indicates a possible serious defect in one in 20 women. (Let’s leave aside the question of false negatives for now.)
By those numbers, if all women who become pregnant over the age of 35 had the diagnostic test, they’d suffer four miscarriages of healthy fetuses for every birth defect discovered. That really sucks.
If you give all women a screening test first, then five percent of women get a birth-defect scare, and are given the diagnostic test. Of those women, only ten percent are actually carrying a fetus with a birth defect — in that sense, the accuracy of the screening test, like the mammogram, is “horrible.”
But look at the real world results. Only five percent of women, rather than one hundred percent, are given the dangerous diagnostic test. The proportion of women who miscarry due to testing drops accordingly — from 2% to 0.1%. And the ratio of miscarriages to defects discovered flips from 4:1 to 1:5.
Screening tests with high false-positive rates can play a very positive role in the diagnostic process.
Comment 6/29/2006
Brooklynite:
Believe me, I know. It sucks not just because it’s inaccurate, but because the inaccuracy has a deterrent effect. It makes it harder to convince women to get them, when more should. With something like cancer, early detection is everything.
Comment 6/29/2006