Author Topic: five thirty eight knows what they are doing  (Read 1116 times)

0 Members and 1 Guest are viewing this topic.

Offline jt512

  • Frequent Poster
  • ******
  • Posts: 2447
    • jt512
Re: five thirty eight knows what they are doing
« Reply #15 on: April 06, 2019, 04:53:39 PM »
I'm still not sure what you're getting at.  The chart shows that in this category—presidential elections—events that were given probabilities of 25% and 35% occurred with somewhat smaller frequency: 13% and 18%, respectively.  Overall, the chart suggests that for presidential elections the model is somewhat too timid, that is, forecasts tend to be too close to 50%.  So predicted probabilities in the range of 25% to 35%, like Trump winning in 2016 (28%), tended to be overstated; unlikely events, like Trump being elected, should have been given even smaller probabilities.  Is that what you're saying?

Not exactly. I'm saying that of their forecasts that had a 25% and a 35% probability came in at a significantly lower percentages.

That is what I am saying.  So events, like Trump winning in 2016, which the 538 model assigned probabilities of around 25%, should have been assigned lower probabilities, in the neighborhood of 13%.  The data in the chart suggest that for presidential elections, the 538 model is too timid, in general assigning probabilities closer to 50% than are warranted.  The data suggest, perhaps ironically, that 538 gave Trump to high a chance of winning. 
Ich verstehe nur Bahnhof.

Offline CarbShark

  • Poster of Extraordinary Magnitude
  • **********
  • Posts: 11341
Re: five thirty eight knows what they are doing
« Reply #16 on: April 06, 2019, 05:31:54 PM »
I'm still not sure what you're getting at.  The chart shows that in this category—presidential elections—events that were given probabilities of 25% and 35% occurred with somewhat smaller frequency: 13% and 18%, respectively.  Overall, the chart suggests that for presidential elections the model is somewhat too timid, that is, forecasts tend to be too close to 50%.  So predicted probabilities in the range of 25% to 35%, like Trump winning in 2016 (28%), tended to be overstated; unlikely events, like Trump being elected, should have been given even smaller probabilities.  Is that what you're saying?

Not exactly. I'm saying that of their forecasts that had a 25% and a 35% probability came in at a significantly lower percentages.

That is what I am saying.  So events, like Trump winning in 2016, which the 538 model assigned probabilities of around 25%, should have been assigned lower probabilities, in the neighborhood of 13%.  The data in the chart suggest that for presidential elections, the 538 model is too timid, in general assigning probabilities closer to 50% than are warranted.  The data suggest, perhaps ironically, that 538 gave Trump to high a chance of winning.

I don't think it's valid to apply that generalization to a specific case. All of the things to which they assigned a 25% probability of happening happened 13% of the time.

But that doesn't imply that that specific example should have been given a lower probability.  It implies that in general those predictions were too high.

and Donald Trump is President of the United States.

I'm not a doctor, I'm just someone who has done a ton of research into diet and nutrition.

Offline jt512

  • Frequent Poster
  • ******
  • Posts: 2447
    • jt512
Re: five thirty eight knows what they are doing
« Reply #17 on: April 06, 2019, 05:39:14 PM »
I'm still not sure what you're getting at.  The chart shows that in this category—presidential elections—events that were given probabilities of 25% and 35% occurred with somewhat smaller frequency: 13% and 18%, respectively.  Overall, the chart suggests that for presidential elections the model is somewhat too timid, that is, forecasts tend to be too close to 50%.  So predicted probabilities in the range of 25% to 35%, like Trump winning in 2016 (28%), tended to be overstated; unlikely events, like Trump being elected, should have been given even smaller probabilities.  Is that what you're saying?

Not exactly. I'm saying that of their forecasts that had a 25% and a 35% probability came in at a significantly lower percentages.

That is what I am saying.  So events, like Trump winning in 2016, which the 538 model assigned probabilities of around 25%, should have been assigned lower probabilities, in the neighborhood of 13%.  The data in the chart suggest that for presidential elections, the 538 model is too timid, in general assigning probabilities closer to 50% than are warranted.  The data suggest, perhaps ironically, that 538 gave Trump to high a chance of winning.

I don't think it's valid to apply that generalization to a specific case. All of the things to which they assigned a 25% probability of happening happened 13% of the time.

But that doesn't imply that that specific example should have been given a lower probability.  It implies that in general those predictions were too high.

Say you roll a die.  What probability would you give it, before the roll, that it would come up "six."  If it then does comes up "six," was the probability you gave it wrong?
Ich verstehe nur Bahnhof.

Offline CarbShark

  • Poster of Extraordinary Magnitude
  • **********
  • Posts: 11341
Re: five thirty eight knows what they are doing
« Reply #18 on: April 06, 2019, 06:05:44 PM »
This is more like rolling 1000 die each with 20 sides, and the die is loaded, and you're trying to predict the probability of each number turning up based on what you think you know about the loading and how you think the loading affects the probability. And the loading might change with each roll. And based on the accuracy of your overall predictions you're making a claim about the probability of a single row, where the loading was unique.



and Donald Trump is President of the United States.

I'm not a doctor, I'm just someone who has done a ton of research into diet and nutrition.

Offline jt512

  • Frequent Poster
  • ******
  • Posts: 2447
    • jt512
Re: five thirty eight knows what they are doing
« Reply #19 on: April 06, 2019, 06:18:46 PM »
This is more like rolling 1000 die each with 20 sides, and the die is loaded, and you're trying to predict the probability of each number turning up based on what you think you know about the loading and how you think the loading affects the probability. And the loading might change with each roll. And based on the accuracy of your overall predictions you're making a claim about the probability of a single row, where the loading was unique.

That is incoherent.
Ich verstehe nur Bahnhof.

Offline CarbShark

  • Poster of Extraordinary Magnitude
  • **********
  • Posts: 11341
Re: five thirty eight knows what they are doing
« Reply #20 on: April 06, 2019, 07:08:40 PM »
This is more like rolling 1000 die each with 20 sides, and the die is loaded, and you're trying to predict the probability of each number turning up based on what you think you know about the loading and how you think the loading affects the probability. And the loading might change with each roll. And based on the accuracy of your overall predictions you're making a claim about the probability of a single row, where the loading was unique.

That is incoherent.

So is comparing a totally random occurrence (tossing a die) with an anything but random outcome (voting results).
and Donald Trump is President of the United States.

I'm not a doctor, I'm just someone who has done a ton of research into diet and nutrition.

Offline jt512

  • Frequent Poster
  • ******
  • Posts: 2447
    • jt512
Re: five thirty eight knows what they are doing
« Reply #21 on: April 06, 2019, 08:12:26 PM »
This is more like rolling 1000 die each with 20 sides, and the die is loaded, and you're trying to predict the probability of each number turning up based on what you think you know about the loading and how you think the loading affects the probability. And the loading might change with each roll. And based on the accuracy of your overall predictions you're making a claim about the probability of a single row, where the loading was unique.

That is incoherent.
So is comparing a totally random occurrence (tossing a die) with an anything but random outcome (voting results).

A die roll is no more fundamentally random than an election result — hell, a die roll is completely determined by Newtonian physics.  Can we say that about election results? 

The random events in the 538 forecast are the polling results.  Each poll takes a random sample of voters.  Polling results are statistical estimates.  Their results depend on the random sample of respondents they happen to select.  In addition, polls make systematic errors, such as incorrectly estimating the probability that a "likely voter" will actually vote.  538 converts uncertainty in polling results into probabilistic forecasts of elections results.  If there were no uncertainty in the polls, 538's predictions would always be 100% or 0% for a particular candidate.  But uncertainty in the polls makes this impossible. 
Ich verstehe nur Bahnhof.

Offline Harry Black

  • International Man of Mystery
  • Global Moderator
  • Poster of Extraordinary Magnitude
  • *****
  • Posts: 15845
Re: five thirty eight knows what they are doing
« Reply #22 on: April 06, 2019, 08:45:56 PM »
It will be interesting to see if they can continue such accuracy in what seems to be quite a different political landscape to previous elections.
The likelihood (and hope) is that Trump and the effect of social media is an anomaly, but if not then accounting for such variables may be difficult.

I would still put a fair amount of stock in their opinion though, given that we have little else to go off of.

Offline xenu

  • Stopped Going Outside
  • *******
  • Posts: 4009
  • Chicago Blackhawks 2010,13,15 Stanley Cup Champion
Re: five thirty eight knows what they are doing
« Reply #23 on: April 06, 2019, 09:48:14 PM »
I think some of the issue of maybe a low probability of Trump's odds of winning the election was that a lot of people were kind of embarrassed about saying they would vote for him. I don't think this next election will be the case. Also the way they pole the public has changed. Caller ID makes it so people do not answer the phone unless they know the number. So polling companies will need to change the way they get their data.
"In the beginning the Universe was created. This has made a lot of people very angry and has been widely regarded as a bad move."
Douglas Adams

Offline Gigabyte

  • Not Enough Spare Time
  • **
  • Posts: 114
  • If you believe, they put a man on the moon
Re: five thirty eight knows what they are doing
« Reply #24 on: April 07, 2019, 09:50:09 AM »
they did an analysis of how well their predictions have gone over the years and they are pretty damn good.
I know this is true because they told me it is true
I don't understand some things, but at least I know I don't know

Offline jt512

  • Frequent Poster
  • ******
  • Posts: 2447
    • jt512
Re: five thirty eight knows what they are doing
« Reply #25 on: April 07, 2019, 10:04:20 AM »
they did an analysis of how well their predictions have gone over the years and they are pretty damn good.
I know this is true because they told me it is true


Dave, unlike you, actually looked at the analysis.
Ich verstehe nur Bahnhof.

Offline Gigabyte

  • Not Enough Spare Time
  • **
  • Posts: 114
  • If you believe, they put a man on the moon
Re: five thirty eight knows what they are doing
« Reply #26 on: April 07, 2019, 10:25:29 AM »
they did an analysis of how well their predictions have gone over the years and they are pretty damn good.
I know this is true because they told me it is true
Dave, unlike you, actually looked at the analysis.
That must be true, because you said it is.
I don't understand some things, but at least I know I don't know

Offline superdave

  • Too Much Spare Time
  • ********
  • Posts: 6040
Re: five thirty eight knows what they are doing
« Reply #27 on: April 07, 2019, 04:53:44 PM »
they did an analysis of how well their predictions have gone over the years and they are pretty damn good.
I know this is true because they told me it is true
Dave, unlike you, actually looked at the analysis.
That must be true, because you said it is.

do not confuse skepticism with cynicism.
I disavow anyone in the movement involved in any illegal,unethical, sexist, or racist behavior. However, I don't have the energy or time to investigate each person and case, and a lack of individual disavowals for each incident should not be construed as condoning such behavior.

Offline brilligtove

  • Too Much Spare Time
  • ********
  • Posts: 7280
  • Ignorance can be cured. Stupidity, you deal with.
Re: five thirty eight knows what they are doing
« Reply #28 on: April 07, 2019, 04:59:06 PM »
I looked at their report, but I am not able to grok most of it. They do seem to have the most reliable political forecasts. (I do get the difference in how they use 'forecast' and 'predict'.)
evidence trumps experience | performance over perfection | responsibility – authority = scapegoat | emotions motivate; data doesn't

Offline Billzbub

  • Stopped Going Outside
  • *******
  • Posts: 4265
  • I know you know I know
Re: five thirty eight knows what they are doing
« Reply #29 on: April 08, 2019, 03:56:43 PM »
This is more like rolling 1000 die each with 20 sides, and the die is loaded, and you're trying to predict the probability of each number turning up based on what you think you know about the loading and how you think the loading affects the probability. And the loading might change with each roll. And based on the accuracy of your overall predictions you're making a claim about the probability of a single row, where the loading was unique.

I bought twenty 20-sided dice at Dragon*Con last year so that whenever I get a couple of low rolls in a row when playing D&D, I can throw the offending dice in the trash as a warning to the other dice to roll higher.  It hasn't been working, I assume, because dice don't mind being in the trash.  I am now looking for a low-cost way to shatter uncooperative dice in front of the others with a hydraulic press or something.  That should do the trick.
Quote from: Steven Novella
gleefully altering one’s beliefs to accommodate new information should be a badge of honor

 

personate-rain
personate-rain