<!DOCTYPE html>

An Analysis of Expert Computer Sports Predictions: The Wisdom of the Crowd and PredictionTracker.com

An Analysis of Expert Computer Sports Predictions: The Wisdom of the Crowd and PredictionTracker.com

Author

Jake Stoetzner

Published

September 3, 2024

Introduction

This post analyzes data from one of my favorite websites: The Prediction Tracker.

The premise is simple. Predictions for every NFL and NCAA game from over 50 different computer systems are aggregated for NFL, NCAA Football, NCAA Basketball and NBA games. The computer predictions are then analyzed - average prediction, median prediction, min, max etc. The actual results of each game are then compared against each system to see which is the “most” accurate.

For instance, the screenshot below shows current NCAA football predictions.

For example, Air Force hosts San Jose State.

  • The line open is at 6 points, the current line is at 6.5 points and the prediction avg is at 5.04 points
  • All lines are in reference to the home team. +6.5 means the home team is favored to win by 6.5 points and -6.5 means the visitor is favored by 6.5 points.
  • The bookmaker believes Air Force will win by 6.5 points.
  • The average of all the predictions indicates that Air Force is 5.04 points “better” than San Jose St.
  • If we “believe” the prediction avg then we should take San Jose St.and the 6.5 points because they are projected to only lose by 5.04 points

The above is just one example of how to use the prediction avg and the other information on the site.

Below is a list of the historical data available:

Additionally, here is a list of all computer systems used for NCAA football, NFL, NBA and NCAA basketball I have compiled in csv format.

Summary

As discussed further in this post across the four leagues I analyzed (NCAAF, NFL, NBA and NCAAB) and data from the 2001 to 2023 seasons:

  • The average of the computer predictions is not more accurate at picking the actual straight-up winner than the posted line.
  • The average of the computer predictions was slightly more accurate at predicting the actual final score differential of the event as compared with the closing line.
  • A betting system that used the average of the computer predictions to choose winners against the spread would be a net loser across all of the leagues

Research

Much of the research around the aggregation of computer predictions centers around the usage of prediction markets and the wisdom of the crowd in sports forecasting.

“The wisdom of the crowd is the collective opinion of a diverse and independent group of individuals rather than that of a single expert.” See Wisdom of the crowd - Wikipedia - Link - DB. In other words, a large group of individuals aggregated forecast has been found to be better than that of a single person (regardless of expertise).

“Prediction Markets are based on various people taking a stance on the same event and willing to back their hunch by paying (or collecting) money should their hunch be wrong (or right). Given that a number of people are taking a stance on the same event, it can be seen as a predictive model of the event.” See Sports Forecasting: A Comparison of the Forecast Accuracy of Prediction Markets, Betting Odds and Tipsters – Professor Graham Kendall Web Site - Link - DB.

Examples of the use and accuracy of the wisdom of the crowd theory and prediction markets abounds, including in sports betting. The idea in its most basic form is to average a large group of informed and committed forecasts to come to the correct prediction. Ideally, this eliminates individual prediction errors and leads to greater accuracy. Several articles have shown that using the wisdom of the crowds approach has value in sports betting (note that some of these articles are linked in the Notes & Research section below).

One paper in particular looked at the use of an online tipster community on OddsPortal.com as the source of the “wisdom of the crowd” to see the effect of betting returns when incorporating the crowd’s opinion. See The Wisdom of Amateur Crowds: Evidence from an Online Community of Sports Tipsters - Link - DB

In this paper we analyse predictions collected on Oddsportal, a betting comparison website which also hosts an online community of sports tipsters. Members of the Oddsportal community are ranked according to the betting return on their tips. The crowds on Oddsportal are smaller than Twitter, for example, but because of the ranking criteria these crowds are specifically tasked with identifying when betting markets are mispriced (i.e. when there is information not in betting prices). This setting therefore provides small, but highly-targeted, crowd-sourced predictions. We set out to answer two questions. 1) Can Oddsportal tips be used to improve betting returns? And 2) does the informational content of these crowd-sourced predictions stem from the full crowd, or a subset of skilled or experienced individuals?

On the first question, the researchers concluded that “a strategy of betting when a majority of tipsters predict an outcome produces average returns of 1.317% for 68,339 events.”

But on the second question, there was no additional betting return gained by focusing on those tipsters with more experience or more skill. “This suggests that the accuracy of these crowd forecasts stems from the whole crowd, rather than just a select few tipsters.”

Sports betting is inherently challenging. The idea that the PredictionTracker site rests on and the data analyzed below is this: is there a model (or collection of models) that are significantly better at predicting the outcome of a sporting event than the oddsmaker? But here is the rub: it doesn’t matter if the models are slightly better at predicting the outcome. For reasons outlined below, the model must be significantly better at either predicting the actual winner of the event or of predicting the team that covers more often than the sportsbook.

The bettor is up against it for several reasons. First, the bettor must overcome the vig charged by most sportsbooks and win at least 52.35% of their bets just to breakeven assuming -110 (1.91) odds. Any winning percentage below this is a losing proposition.

Assuming that most spread bets are offered at -110,

Additionally, outside of a few props and the exotic approach of “selling” points, the bettor doesn’t gain anything from hitting the point spread more accurately than the sportsbook. For example, if the bettor forecasts the final spread as home +7 and the sportsbook offers the line at +3 and the home team actually wins by 7, the bettor gains nothing from being more accurate than the sportsbook.

Data

The historical prediction data was acquired from The Prediction Tracker.

Overview

The first broad questions is how often did the closing line and computer average predict the actual winner straight up (NOT against the spread)?

The line and computer average “predict” the winner by use of the point spread. If the line is positive, the home team is predicted to win. If the line is negative, the road teamn is predicted to win.

The columns in the table are defined as follows:

  • count the number of games analyzed in the given leagues
  • line_win total number of straight up winners based on the closing line
  • line_loss total number of straight up losers based on the closing line
  • line_win_pct percentage of straight up winners based on the closing line
  • pred_win total number of straight up winners based on the prediction average
  • pred_loss total number of straight up losers based on the prediction average
  • pred_win_pct percentage of straight up winners based on the prediction average
league count line_win line_loss line_win_pct pred_win pred_loss pred_win_pct
NCAAF 14674 10908 3766 0.7433556 10826 3848 0.7377675
NFL 6277 4157 2120 0.6622590 4019 2258 0.6402740
NBA 17533 11710 5823 0.6678834 11585 5948 0.6607540
NCAAB 73522 52806 20716 0.7182340 52737 20785 0.7172955

Using the predicted average as a proxy to predict the straight up winner of the event is no better (and is actually slightly worse) than simply using the closing line.

The second question is how accurate was the closing line and the computer average at predicting the ACTUAL final score difference?

  • line_error the average of the closing line minus the actual final score difference
  • pred_error the average of the prediction average minus the actual final score difference
  • line_mse and pred_mse mean squared error found by taking the average of (line_error)^2 and (pred_error)^2 respectively
league count line_error line_mse pred_error pred_mse
NCAAF 14674 0.1934441 0.0374206 -0.0059655 0.0000356
NFL 6277 -0.1079745 0.0116585 0.0871361 0.0075927
NBA 17533 0.0841989 0.0070894 0.0676802 0.0045806
NCAAB 73522 -0.1049096 0.0110060 -0.0099333 0.0000987

Overall, the prediction average had a lower absolute error than the closing line. This is likely due to the line moving up or down in response to bets being placed on one side more than the other.

The final question is how accurate the aggregate average of the computer predictions were in predicting which team would cover the spread. Recall that this analysis is more important to bettors than the accuracy of the final score difference prediction.

league total win_no loss_no win_pct
NCAAF 14674 7244 7410 0.4936623
NFL 6277 3181 3076 0.5067708
NBA 17533 8481 8653 0.4837164
NCAAB 73522 35823 35947 0.4872419

NCAA Football

Below is a table grouped by year showing the win percentage of using the prediction_avg as the predictor.

  • If the predicted average is greater than the closing line, bet the home team
  • If the predicted average is less than the closing line, bet the road team
year total win_no loss_no win_pct
2003 698 338 357 0.4842407
2004 656 336 320 0.5121951
2005 665 313 346 0.4706767
2006 280 141 139 0.5035714
2007 712 362 350 0.5084270
2008 718 343 375 0.4777159
2009 714 354 360 0.4957983
2010 718 342 376 0.4763231
2011 715 350 365 0.4895105
2012 732 385 346 0.5259563
2013 738 380 355 0.5149051
2014 760 389 371 0.5118421
2015 724 341 379 0.4709945
2016 760 382 375 0.5026316
2017 780 379 401 0.4858974
2018 773 383 390 0.4954722
2019 659 328 331 0.4977238
2020 534 256 278 0.4794007
2021 770 361 409 0.4688312
2022 776 391 385 0.5038660
2023 792 390 402 0.4924242

I repeated this process using the opening line rather than the closing line.

year total win_no loss_no win_pct
2003 698 343 351 0.4914040
2004 656 330 326 0.5030488
2005 665 316 342 0.4751880
2006 280 135 145 0.4821429
2007 712 354 358 0.4971910
2008 718 339 377 0.4721448
2009 714 366 348 0.5126050
2010 718 345 371 0.4805014
2011 715 353 362 0.4937063
2012 732 393 338 0.5368852
2013 738 383 352 0.5189702
2014 760 359 399 0.4723684
2015 724 339 381 0.4682320
2016 760 381 376 0.5013158
2017 780 374 406 0.4794872
2018 773 392 381 0.5071151
2019 659 337 322 0.5113809
2020 534 264 270 0.4943820
2021 770 365 405 0.4740260
2022 776 381 395 0.4909794
2023 792 383 409 0.4835859

The chart below compares the performance by year using the prediction average and the results using the opening line and closing line.

Next, I calculated the difference between the predicted line and the actual line to see how well the average predictions performed when divided into percentiles. Only the closing line is used.

For example, did the winning percentage increase or decrease when there was a large difference between the average prediction and the closing line?

bin total bin_low bin_high win_no win_pct
1 367 -6.920000 -19.1700000 181 0.4931880
2 367 -5.620000 -6.9200000 174 0.4741144
3 367 -4.860000 -5.6100000 176 0.4795640
4 367 -4.280000 -4.8600000 178 0.4850136
5 367 -3.850000 -4.2800000 195 0.5313351
6 367 -3.490000 -3.8400000 188 0.5122616
7 367 -3.140000 -3.4900000 187 0.5095368
8 367 -2.860000 -3.1400000 199 0.5422343
9 367 -2.570196 -2.8600000 198 0.5395095
10 367 -2.320000 -2.5700000 181 0.4931880
11 367 -2.060000 -2.3200000 195 0.5313351
12 367 -1.840000 -2.0600000 194 0.5286104
13 367 -1.610000 -1.8331250 187 0.5095368
14 367 -1.420000 -1.6100000 168 0.4577657
15 367 -1.199778 -1.4100000 190 0.5177112
16 367 -0.970000 -1.1972500 202 0.5504087
17 367 -0.760000 -0.9700000 189 0.5149864
18 367 -0.560000 -0.7528000 192 0.5231608
19 367 -0.370000 -0.5600000 182 0.4959128
20 366 -0.180000 -0.3700000 182 0.4972678
21 366 0.010000 -0.1793182 184 0.5027322
22 366 0.210000 0.0100000 176 0.4808743
23 366 0.420000 0.2100000 174 0.4754098
24 366 0.630000 0.4200000 178 0.4863388
25 366 0.820000 0.6300000 175 0.4781421
26 366 1.030000 0.8200000 176 0.4808743
27 366 1.247727 1.0300000 177 0.4836066
28 366 1.450000 1.2500000 192 0.5245902
29 366 1.700000 1.4500000 187 0.5109290
30 366 1.940000 1.7000000 170 0.4644809
31 366 2.190000 1.9400000 155 0.4234973
32 366 2.480000 2.1900000 182 0.4972678
33 366 2.780980 2.4800000 164 0.4480874
34 366 3.110000 2.7891667 172 0.4699454
35 366 3.460000 3.1100000 183 0.5000000
36 366 3.890000 3.4600000 154 0.4207650
37 366 4.410000 3.8900000 186 0.5081967
38 366 5.100000 4.4100000 176 0.4808743
39 366 6.270000 5.1000000 170 0.4644809
40 366 20.010000 6.2700000 175 0.4781421
NA 15 -Inf Inf 0 0.0000000

The final analysis I completed for NCAAF is by examining whether the standard deviation of the predictions mattered to the final outcome. A smaller standard deviation indicates that the individual predictions were closer to the mean. A larger standard deviation then would indicate data that is more spread out.

Does “agreeement” among the individual predictions - meaning a smaller standard deviation where each individual prediction is relatively closer to the average - affect the winning percentage?

bin_std total bin_low bin_mean bin_high win_no win_pct
1 734 -46.75 -21.930 2.89 355 0.4836512
2 734 2.90 3.075 3.25 375 0.5108992
3 734 3.25 3.380 3.51 370 0.5040872
4 734 3.51 3.620 3.73 349 0.4754768
5 734 3.73 3.835 3.94 379 0.5163488
6 734 3.94 4.035 4.13 361 0.4918256
7 734 4.13 4.215 4.30 367 0.5000000
8 734 4.30 4.390 4.48 359 0.4891008
9 734 4.48 4.575 4.67 357 0.4863760
10 734 4.67 4.755 4.84 361 0.4918256
11 734 4.84 4.940 5.04 375 0.5108992
12 734 5.04 5.145 5.25 347 0.4727520
13 734 5.25 5.355 5.46 392 0.5340599
14 734 5.46 5.575 5.69 367 0.5000000
15 733 5.69 5.825 5.96 372 0.5075034
16 733 5.96 6.115 6.27 337 0.4597544
17 733 6.27 6.460 6.65 366 0.4993179
18 733 6.65 6.925 7.20 348 0.4747613
19 733 7.20 7.625 8.05 359 0.4897681
20 733 8.05 1178.625 2349.20 348 0.4747613

NFL

A table grouped by year showing the win percentage when using the prediction_avg to choose the ATS winner and the closing line for the NFL from 2000 to 2023:

year total win_no loss_no win_pct
00 259 131 127 0.5057915
01 259 121 136 0.4671815
02 267 144 123 0.5393258
03 267 124 143 0.4644195
04 267 145 122 0.5430712
05 260 128 132 0.4923077
06 128 64 64 0.5000000
07 267 151 116 0.5655431
08 264 124 140 0.4696970
09 260 141 118 0.5423077
10 267 133 134 0.4981273
11 264 142 122 0.5378788
12 267 137 130 0.5131086
13 266 147 119 0.5526316
14 266 133 120 0.5000000
15 267 143 124 0.5355805
16 267 120 147 0.4494382
17 267 137 130 0.5131086
18 267 140 127 0.5243446
19 256 130 126 0.5078125
20 271 141 128 0.5202952
21 284 128 156 0.4507042
22 285 137 147 0.4807018
23 285 140 145 0.4912281

A table grouped by year showing the win percentage when using the prediction_avg to choose the ATS winner and the opening line for the NFL from 2000 to 2023:

year total win_no loss_no win_pct
00 259 0 0 0.0000000
01 259 0 0 0.0000000
02 267 151 116 0.5655431
03 267 129 138 0.4831461
04 267 152 115 0.5692884
05 260 126 134 0.4846154
06 128 67 61 0.5234375
07 267 156 111 0.5842697
08 264 129 135 0.4886364
09 260 130 122 0.5000000
10 267 134 132 0.5018727
11 264 144 120 0.5454545
12 267 131 136 0.4906367
13 266 142 124 0.5338346
14 266 134 119 0.5037594
15 267 139 128 0.5205993
16 267 111 156 0.4157303
17 267 129 138 0.4831461
18 267 134 133 0.5018727
19 256 117 139 0.4570312
20 271 135 134 0.4981550
21 284 118 166 0.4154930
22 285 138 146 0.4842105
23 285 141 144 0.4947368

A chart showing a comparison of the opening and closing lines for the NFL.

The table and chart below group the average predictions by percentile (ie in bins).

bin total bin_low bin_high win_no win_pct
1 313 -4.24 -16.97 171 0.5463259
2 313 -3.16 -4.24 153 0.4888179
3 313 -2.46 -3.16 175 0.5591054
4 313 -1.91 -2.46 174 0.5559105
5 313 -1.47 -1.90 173 0.5527157
6 313 -1.10 -1.47 160 0.5111821
7 313 -0.77 -1.10 162 0.5175719
8 313 -0.43 -0.77 172 0.5495208
9 313 -0.12 -0.43 157 0.5015974
10 313 0.19 -0.12 170 0.5431310
11 313 0.50 0.19 146 0.4664537
12 313 0.81 0.50 156 0.4984026
13 313 1.14 0.81 158 0.5047923
14 313 1.49 1.14 152 0.4856230
15 313 1.87 1.49 149 0.4760383
16 313 2.32 1.87 150 0.4792332
17 313 2.86 2.32 155 0.4952077
18 313 3.45 2.86 147 0.4696486
19 313 4.69 3.45 159 0.5079872
20 312 17.66 4.70 142 0.4551282
NA 18 -Inf Inf 0 0.0000000

The table and chart below group the average predictions by percentile (ie in bins).

bin_std total bin_low bin_high win_no win_pct
1 275 1.259516 2.162134 108 0.3927273
2 275 2.162215 2.360601 149 0.5418182
3 275 2.361402 2.540693 139 0.5054545
4 275 2.540713 2.674152 137 0.4981818
5 275 2.674433 2.794694 144 0.5236364
6 275 2.794718 2.918484 147 0.5345455
7 275 2.919767 3.027186 146 0.5309091
8 275 3.028577 3.157752 143 0.5200000
9 275 3.158387 3.281494 142 0.5163636
10 275 3.281562 3.408045 151 0.5490909
11 274 3.408142 3.533564 133 0.4854015
12 274 3.533778 3.674240 148 0.5401460
13 274 3.674245 3.830343 129 0.4708029
14 274 3.831244 3.994727 139 0.5072993
15 274 3.996741 4.205013 136 0.4963504
16 274 4.205559 4.445218 129 0.4708029
17 274 4.446354 4.734807 157 0.5729927
18 274 4.736355 5.167118 139 0.5072993
19 274 5.169862 5.873340 136 0.4963504
20 274 5.874305 16.030910 133 0.4854015
NA 787 Inf -Inf 396 0.5031766

NBA

A table grouped by year showing the win percentage when using the prediction_avg to choose the ATS winner and the closing line for the NBA from 2004 to 2021:

year total win_no loss_no win_pct
04 1285 597 675 0.4645914
05 1253 608 639 0.4852354
06 1261 590 659 0.4678826
07 1257 615 595 0.4892601
08 1146 590 548 0.5148342
09 1216 623 560 0.5123355
14 1415 679 633 0.4798587
15 1309 626 669 0.4782277
16 1196 588 566 0.4916388
17 1313 621 678 0.4729627
18 1315 625 622 0.4752852
19 967 484 481 0.5005171
20 1241 594 639 0.4786463
21 1359 641 689 0.4716703

A table grouped by year showing the win percentage when using the prediction_avg to choose the ATS winner and the opening line for the NBA from 2004 to 2021:

year total win_no loss_no win_pct
04 1285 599 667 0.4661479
05 1253 569 635 0.4541101
06 1261 573 670 0.4544013
07 1257 594 615 0.4725537
08 1146 587 551 0.5122164
09 1216 623 560 0.5123355
14 1415 682 628 0.4819788
15 1309 617 678 0.4713522
16 1196 594 560 0.4966555
17 1313 636 659 0.4843869
18 1315 627 620 0.4768061
19 967 474 491 0.4901758
20 1241 596 637 0.4802579
21 1359 656 672 0.4827079

A chart showing a comparison of win percentage using average prediction against the opening and closing lines for the NBA.

The table and chart below group the average predictions by percentile (ie in bins).

bin total bin_low bin_high win_no win_pct
1 867 -4.46 -691.88 421 0.4855825
2 867 -3.42 -4.46 442 0.5098039
3 867 -2.74 -3.42 448 0.5167243
4 867 -2.22 -2.74 465 0.5363322
5 867 -1.79 -2.22 439 0.5063437
6 867 -1.40 -1.79 412 0.4752018
7 867 -1.03 -1.40 432 0.4982699
8 867 -0.72 -1.03 447 0.5155709
9 867 -0.40 -0.72 420 0.4844291
10 867 -0.11 -0.40 451 0.5201845
11 867 0.21 -0.11 427 0.4925029
12 867 0.53 0.21 423 0.4878893
13 867 0.85 0.53 406 0.4682814
14 867 1.19 0.85 395 0.4555940
15 867 1.60 1.19 423 0.4878893
16 867 2.05 1.60 435 0.5017301
17 867 2.58 2.05 411 0.4740484
18 866 3.29 2.58 386 0.4457275
19 866 4.51 3.29 406 0.4688222
20 866 808.11 4.51 392 0.4526559
NA 196 -Inf Inf 0 0.0000000

The table and chart below group the average predictions by percentile (ie in bins).

bin_std total bin_low bin_high win_no win_pct
1 130 0.503291 1.401647 55 0.4230769
2 130 1.402322 1.620796 55 0.4230769
3 130 1.621359 1.770917 63 0.4846154
4 130 1.771406 1.891712 75 0.5769231
5 130 1.891849 1.995554 65 0.5000000
6 130 1.995736 2.097832 63 0.4846154
7 130 2.098321 2.201303 66 0.5076923
8 130 2.202207 2.305500 75 0.5769231
9 130 2.305765 2.397890 65 0.5000000
10 129 2.398709 2.506781 72 0.5581395
11 129 2.506882 2.608685 58 0.4496124
12 129 2.609376 2.741244 62 0.4806202
13 129 2.743929 2.879725 55 0.4263566
14 129 2.881169 3.008474 68 0.5271318
15 129 3.008613 3.190744 57 0.4418605
16 129 3.192097 3.367923 58 0.4496124
17 129 3.369894 3.649926 52 0.4031008
18 129 3.651833 3.986014 58 0.4496124
19 129 3.989641 4.574999 59 0.4573643
20 129 4.575533 11.116674 54 0.4186047
NA 14944 Inf -Inf 7246 0.4848769

NCAAB

A table grouped by year showing the win percentage when using the prediction_avg to choose the ATS winner and the closing line for NCAAB from 2003 to 2021:

year total win_no loss_no win_pct
03 2844 1407 1384 0.4947257
04 3007 1489 1440 0.4951779
05 2964 1494 1431 0.5040486
06 3340 1652 1674 0.4946108
07 3431 1499 1626 0.4368989
08 3243 1493 1555 0.4603762
09 3576 1797 1716 0.5025168
10 3631 1673 1703 0.4607546
11 3573 1652 1777 0.4623566
12 3933 1914 1862 0.4866514
13 3875 1916 1910 0.4944516
14 3939 1952 1964 0.4955572
15 3941 1961 1947 0.4975894
16 4013 2044 1923 0.5093446
17 4079 1973 2080 0.4836970
18 5564 2739 2764 0.4922717
19 5288 2537 2574 0.4797655
20 3880 1949 1903 0.5023196
21 5401 2682 2714 0.4965747

A table grouped by year showing the win percentage when using the prediction_avg to choose the ATS winner and the opening line for the NCAAB from 2003 to 2021:

year total win_no loss_no win_pct
03 2844 0 0 0.0000000
04 3007 1405 1464 0.4672431
05 2964 1391 1447 0.4692982
06 3340 1601 1653 0.4793413
07 3431 1493 1611 0.4351501
08 3243 1481 1563 0.4566759
09 3576 1790 1723 0.5005593
10 3631 1688 1688 0.4648857
11 3573 1625 1703 0.4547999
12 3933 1903 1866 0.4838546
13 3875 1890 1930 0.4877419
14 3939 1899 1981 0.4821021
15 3941 1976 1924 0.5013956
16 4013 2012 1937 0.5013705
17 4079 1985 2061 0.4866389
18 5564 2772 2730 0.4982027
19 5288 2549 2562 0.4820348
20 3880 1953 1884 0.5033505
21 5401 2666 2730 0.4936123

A chart showing a comparison of win percentage using average prediction against the opening and closing lines for NCAAB.

The table and chart below group the average predictions by percentile (ie in bins).

bin total bin_low bin_high win_no win_pct
1 3645 -3.21 -66.76 1770 0.4855967
2 3645 -2.36 -3.21 1832 0.5026063
3 3645 -1.86 -2.36 1842 0.5053498
4 3645 -1.48 -1.86 1806 0.4954733
5 3645 -1.16 -1.48 1836 0.5037037
6 3644 -0.89 -1.16 1870 0.5131723
7 3644 -0.63 -0.89 1873 0.5139956
8 3644 -0.39 -0.63 1828 0.5016465
9 3644 -0.16 -0.39 1826 0.5010977
10 3644 0.06 -0.16 1861 0.5107025
11 3644 0.28 0.06 1734 0.4758507
12 3644 0.51 0.28 1704 0.4676180
13 3644 0.76 0.51 1748 0.4796926
14 3644 1.02 0.76 1778 0.4879254
15 3644 1.30 1.02 1770 0.4857300
16 3644 1.62 1.30 1754 0.4813392
17 3644 2.02 1.62 1777 0.4876509
18 3644 2.56 2.02 1727 0.4739297
19 3644 3.43 2.56 1680 0.4610318
20 3644 150.92 3.43 1807 0.4958836
NA 637 -Inf Inf 0 0.0000000

The table and chart below group the average predictions by percentile (ie in bins).

bin_std total bin_low bin_high win_no win_pct
1 1805 0.11 1.12 885 0.4903047
2 1805 1.12 1.31 899 0.4980609
3 1805 1.31 1.44 914 0.5063712
4 1805 1.44 1.57 899 0.4980609
5 1805 1.57 1.68 896 0.4963989
6 1805 1.68 1.80 916 0.5074792
7 1805 1.80 1.91 944 0.5229917
8 1805 1.91 2.02 865 0.4792244
9 1805 2.02 2.13 874 0.4842105
10 1805 2.13 2.26 892 0.4941828
11 1805 2.26 2.39 889 0.4925208
12 1805 2.39 2.53 885 0.4903047
13 1805 2.53 2.69 895 0.4958449
14 1805 2.69 2.86 830 0.4598338
15 1805 2.86 3.06 872 0.4831025
16 1805 3.06 3.30 875 0.4847645
17 1804 3.30 3.61 898 0.4977827
18 1804 3.61 4.04 897 0.4972284
19 1804 4.04 4.81 904 0.5011086
20 1804 4.81 46.27 905 0.5016630
NA 37426 Inf -Inf 17989 0.4806552

Notes & Research

  • GitHub - stanford-policylab/wisdom-of-crowds: Studying the "wisdom of crowds" at scale - Link - DB
  • Studying the “Wisdom of Crowds” at Scale - Link - DB
  • The Wisdom of Todd’s Crowd - Link - DB
  • 7.7 Prediction Markets and the Wisdom of Crowds – Information Systems - Link - DB
  • Expert performance and crowd wisdom: Evidence from English Premier League predictions - ScienceDirect - Link - DB
  • Project MUSE - NFL Betting Biases, Profitable Strategies, and the Wisdom of the Crowd - Link - DB
  • Forecasting football match results: Are the many smarter than the few? - Link - DB
  • The wisdom of amateur crowds: Evidence from an online community of sports tipsters - ScienceDirect - Link - DB
  • [PDF Version] The Wisdom of Amateur Crowds: Evidence from an Online Community of Sports Tipsters - Link - DB
  • The Wisdom of Smaller, Smarter Crowds - Link - DB
  • Rationality and Efficiency in NFL Gambling Markets - Link - DB
  • Crowd Wisdom In NFL Point Spread and Over/Under Betting - Link - DB
  • Sports Forecasting: A Comparison of the Forecast Accuracy of Prediction Markets, Betting Odds and Tipsters – Professor Graham Kendall Web Site - Link - DB