Author Topic: Medical Tests and Probability  (Read 371 times)

0 Members and 1 Guest are viewing this topic.

Offline Friendly Angel

  • Frequent Poster
  • ******
  • Posts: 2858
  • Post count reset to zero in both forum apocalypses
Medical Tests and Probability
« on: May 16, 2017, 03:32:10 PM »
I watched this very interesting video essentially about how people are bad at statistics:  Reference 55m.

The Illusion of Certainty: Risk, Probability, and Chance




Now this guy says:
1.  You're a doctor and your patient has a positive test for breast cancer and wants to know what the probability is that she really has it.

2.  Overall incidence is 1% in gen pop.
3.  Actual breast cancer shows as positive test 90% of the time.
4.  Non breast cancer shows as positive test 9% of the time.

If there's a false positive rate of 9% then that means 91% of positive tests HAVE cancer, no? (unless there is some indeterminate result as well as pos or neg)  It doesn't matter how many women get tested or how many women test negative, because we already have the positive test and the statistic must come from that group.





Who's right?


« Last Edit: May 16, 2017, 03:38:13 PM by Friendly Angel »
Amend and resubmit.

Offline 2397

  • Seasoned Contributor
  • ****
  • Posts: 831
Re: Medical Tests and Probability
« Reply #1 on: May 16, 2017, 05:08:54 PM »
If there's a false positive rate of 9% then that means 91% of positive tests HAVE cancer, no? (unless there is some indeterminate result as well as pos or neg)  It doesn't matter how many women get tested or how many women test negative, because we already have the positive test and the statistic must come from that group.

He's using the term false positive rate correctly to my understanding. It's how many of the non-positive cases turn up as positive. How they arrived at the rates is a different matter.

So given the premises; in 1000 women tested, there will be 990 women without breast cancer where 9% of them test positive. And 10 women with breast cancer where 90% of them test positive. Adding up to 98.1 positive tests. 89.1 false positives and 9 true positives.

Offline Friendly Angel

  • Frequent Poster
  • ******
  • Posts: 2858
  • Post count reset to zero in both forum apocalypses
Re: Medical Tests and Probability
« Reply #2 on: May 16, 2017, 05:25:19 PM »
OK, but his explanation doesn't match the question -

Quote
A woman tests positive, what are the chances she has breast cancer?

Positive test - positive cancer 91%
Positive test - negative cancer 9%  (the false positive rate)

Negative test - irrelevant.
No test conducted - irrelevant.
Gen pop cancer rate - irrelevant
Amend and resubmit.

Offline 2397

  • Seasoned Contributor
  • ****
  • Posts: 831
Re: Medical Tests and Probability
« Reply #3 on: May 16, 2017, 05:44:28 PM »
The false positive rate has to do with all people who don't have breast cancer, that's why those other numbers matter. The people who test positive are a subset of all people (or all people who are tested), and the false positive rate is based on how many people without cancer are in that subset, vs. how many people without cancer there are in total.

You could ask the question "A woman tests negative, what are the chances she has breast cancer?".

The false negative rate is 10%, and the true negative rate is 91%. That doesn't mean there's a 10% chance she has breast cancer. Because 901.9 people tested negative, and there is only 1 person among those who tested negative who has breast cancer.

Offline Soldier of FORTRAN

  • Stopped Going Outside
  • *******
  • Posts: 5545
  • Cache rules everything around me.
Re: Medical Tests and Probability
« Reply #4 on: May 16, 2017, 05:46:41 PM »
If there's a false positive rate of 9% then that means 91% of positive tests HAVE cancer, no?

The false negative rate applies to cancer cases.  The false positive rate applies to negative cases. 

Which group is your positive in?  You can't tell.  What's your probability of being in the true positive versus the false positive? 

Edit: If you're interested, there was another thread on this from a few months back.
« Last Edit: May 16, 2017, 05:53:42 PM by Soldier of FORTRAN »
Every soup ladled to the hungry, every blanket draped over the cold signifies, in the final sense, a theft from my gigantic paycheck.

Offline Friendly Angel

  • Frequent Poster
  • ******
  • Posts: 2858
  • Post count reset to zero in both forum apocalypses
Re: Medical Tests and Probability
« Reply #5 on: May 16, 2017, 06:08:12 PM »
The false positive rate has to do with all people who don't have breast cancer, that's why those other numbers matter. The people who test positive are a subset of all people (or all people who are tested), and the false positive rate is based on how many people without cancer are in that subset, vs. how many people without cancer there are in total.


Aha, the copper has dropped.

In other words:

1.  The false positive rate is not the rate of positive tests that are wrong, it's the number of total tests that are wrong in the positive result.

2.  And the actual cancer rate is so low, that the sheer number of false positives overwhelms the number of true positives, yielding a low probability that any particular positive is true.

Thanks. 

They also discussed the Monty Hall problem, coin flipping sequences, and number of countries in Africa priming quiz... all previous topics on SGU.
Amend and resubmit.

Online superdave

  • Stopped Going Outside
  • *******
  • Posts: 4848
  • My name is not dave.
Re: Medical Tests and Probability
« Reply #6 on: May 16, 2017, 06:09:39 PM »
the short answer is, when a doctor prescribes a test, she thinks you are no longer part of the general population, she thinks you are in an at risk group, so your base rate is higher than the population at large.  therefore the odds of a false positive are much lower.

Offline Andrew Clunn

  • Poster of Extraordinary Magnitude
  • **********
  • Posts: 15179
  • Aspiring Super Villain
Re: Medical Tests and Probability
« Reply #7 on: May 16, 2017, 06:17:17 PM »
OK, but his explanation doesn't match the question -

Quote
A woman tests positive, what are the chances she has breast cancer?

Positive test - positive cancer 91%
Positive test - negative cancer 9%  (the false positive rate)

Negative test - irrelevant.
No test conducted - irrelevant.
Gen pop cancer rate - irrelevant

Statisticians think they're clever when they fool you with poorly phrased word problems.  We've had this very question here before.  The math wasn't hard for anybody.  The problem itself was the issue because those statistics are mutually exclusive.
Banned from both Hacker News and the xkcd forum.

Offline Friendly Angel

  • Frequent Poster
  • ******
  • Posts: 2858
  • Post count reset to zero in both forum apocalypses
Re: Medical Tests and Probability
« Reply #8 on: May 16, 2017, 06:20:17 PM »

Statisticians think they're clever when they fool you with poorly phrased word problems.  We've had this very question here before.  The math wasn't hard for anybody.  The problem itself was the issue because those statistics are mutually exclusive.

Yeah, I'm revisiting that thread, didn't pay much attention the first time around.

But in my case, the statistician was trying to help people understand statistics better - it wasn't a trick... I just didn't get it... to the extent that I thought he was wrong.

BTW, I think the actual numbers used are not accurate - chosen just to make the math easier... not to be used in any real calculations.
« Last Edit: May 16, 2017, 06:23:46 PM by Friendly Angel »
Amend and resubmit.

Online The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Reef Tank Owner
  • *****
  • Posts: 9264
Re: Medical Tests and Probability
« Reply #9 on: May 16, 2017, 07:15:04 PM »
the short answer is, when a doctor prescribes a test, she thinks you are no longer part of the general population, she thinks you are in an at risk group, so your base rate is higher than the population at large.  therefore the odds of a false positive are much lower.

Sorry, Dave, but that has nothing to do with this statistical issue.
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell

Offline Friendly Angel

  • Frequent Poster
  • ******
  • Posts: 2858
  • Post count reset to zero in both forum apocalypses
Re: Medical Tests and Probability
« Reply #10 on: May 16, 2017, 07:16:28 PM »
the short answer is, when a doctor prescribes a test, she thinks you are no longer part of the general population, she thinks you are in an at risk group, so your base rate is higher than the population at large.  therefore the odds of a false positive are much lower.

Sorry, Dave, but that has nothing to do with this statistical issue.

And indeed, if I'd followed your math quiz thread more closely I would've figured this out on my own.  Pretty much the same misunderstandings on terminology there.
Amend and resubmit.

Online The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Reef Tank Owner
  • *****
  • Posts: 9264
Re: Medical Tests and Probability
« Reply #11 on: May 16, 2017, 07:31:33 PM »
The numbers are actually just about right, except that the incidence is even lower than 1% even for 50 year olds.  The likelihood that a 50 year old who tests positive actually has breast cancer is only about 4%.  That means that 24 out of 25 women who tested positive for breast cancer will have stress, additional tests, and perhaps even an unnecessary biopsy.  And when you drop down to the old recommendation of starting mammograms at 40, the numbers are even worse.  And add onto that that it's very hard to determine how differences in detection affect outcomes (some suggest that most of the difference in 5- and 10-year survival rates for early detections is due almost entirely to the early detection rather than to women actually living longer), and it may be that one needs to test thousand or even tens of thousand of women and have hundreds of false positives to save one woman's life.

This is why, with my doctor's agreement, I've declined to have a PSA test done at 40.  Most studies seem to suggest that PSA screening of asymptomatic people has very little chance of significantly affecting outcomes and a high risk of needless treatments that can negatively affect quality of life.

My wife just got back from her OB/GYN, who recommended she start yearly mammograms at 40 despite having no family history of breast cancer.  My wife will of course make up her own mind, but I think it's ridiculous.
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell

Online superdave

  • Stopped Going Outside
  • *******
  • Posts: 4848
  • My name is not dave.
Re: Medical Tests and Probability
« Reply #12 on: May 16, 2017, 07:43:08 PM »
the short answer is, when a doctor prescribes a test, she thinks you are no longer part of the general population, she thinks you are in an at risk group, so your base rate is higher than the population at large.  therefore the odds of a false positive are much lower.

Sorry, Dave, but that has nothing to do with this statistical issue.
Sorry for not being clear.   I wanted to explain why, in light of these results, we do any medical testing at all.  The reason this math problem is counter intuitive is because the true answer seems shockingly low.  It makes the test seem just better than worthless.  But the percentages used in the way the problem set up are misleading compared to the true situation that doctors and patients face.  The way the problem above is set up, the base rate, the rate at which women get breast cancer, is very low compared to the false positive rate.  But if you are a 55 year old woman, or you have a family history of breast cancer, or you have a suspicious lump, you are no longer a member of the general population.  You are now a member of the at risk population.  So the base rate as given in the problem no longer applies.  You would use a higher base rate.  If you increase the base rate, then the odds of a true positive increase relative to that of the normal population.

Another way to think about it is this.  Say you are a man and you take an at home pregnancy test and get a positive result.  These tests are 99% accurate.  Does that mean you have a 99% of being pregnant?

Offline Soldier of FORTRAN

  • Stopped Going Outside
  • *******
  • Posts: 5545
  • Cache rules everything around me.
Re: Medical Tests and Probability
« Reply #13 on: May 16, 2017, 07:44:58 PM »
It's an easy and non-invasive test which triggers costly and invasive testing of greater accuracy.  You'll hassle a lot of people but save a lot of lives. 
Every soup ladled to the hungry, every blanket draped over the cold signifies, in the final sense, a theft from my gigantic paycheck.

Online The Latinist

  • Cyber Greasemonkey
  • Technical Administrator
  • Reef Tank Owner
  • *****
  • Posts: 9264
Re: Medical Tests and Probability
« Reply #14 on: May 16, 2017, 07:50:07 PM »
But Dave, mammograms are performed indiscriminately as a screening test on women over 40.  It's exactly the case that the numbers are as bad as they look -- actually significantly worse than in the video's example -- when the test is used as a universal screening measure.

It's an easy and non-invasive test which triggers costly and invasive testing of greater accuracy.  You'll hassle a lot of people but save a lot of lives.

But will you?  There's plenty of room for doubt about that.
I would like to propose...that...it is undesirable to believe in a proposition when there is no ground whatever for supposing it true. — Bertrand Russell