5

And how does this skill vary during the season?

For decades we've heard the presentation that polls are biased, especially in preseason, and lazily voted on.
Does empirical data generally bear this out?

  • 1
    "How skillful are major polls at predicting the winner..." I got two words for you. Donald Trump! – CodeNewbie Dec 16 '16 at 13:30
  • 1
    @CodeNewbie That was the first thing that ran through my mind as well. – user527 Dec 16 '16 at 13:31
5

In college basketball since 2011, ranked teams are a combined 3933-868 (81.9%) against unranked teams. However, if we break it down month-by-month, we get this:

enter image description here

Note the drop-off from December to January. This is when conference games start, and ranked teams start playing 2-3 quality opponents every week. Overall, the early polls seem to perform just as well as the late polls (considering strength of competition) or better.


In college football, we see the exact same thing. Using just the AP poll, and looking back over the last 3 seasons (this data wasn't as ready and I did it half by hand), we see that polls do slightly better in the first ~4 weeks, during the nonconference schedule, and then do worse once conference play starts. The dropoff is likely due to the increase in level of competition; again, this is ranked teams vs unranked teams:

enter image description here


So, overall, the AP Poll actually does better at the beginning of a season, when most ranked teams play cupcakes.

Source for baseketball, playing around with the month: http://www.sports-reference.com/cbb/play-index/tgl_finder.cgi?request=1&match=career&comp_schl_rk=le&val_schl_rk=25&comp_opp_rk=eq&val_opp_rk=NR&game_type=A&game_result=L&is_range=N&order_by=date_game

For football, I went to this page for each week and copied the table into excel, which automatically extracted wins/losses against unranked teams: http://www.espn.com/college-football/rankings/_/poll/1/week/15/year/2014/seasontype/2

Formula in excel for pulling out the W/L data for anyone who wants to keep going this way: =AND(IF(ISNUMBER(SEARCH(" W or L ",Next Week Column)),1,0),IF(ISNUMBER(SEARCH(" # ",Next Week Column)),1,0))

  • Great results Pawr, I really appreciate you doing this! – JeopardyTempest Jun 1 '17 at 16:09
  • Good links too, I'd played with downloading a few things of such, but think those will help. Very interesting consistent-looking downward trend, even within the OOC in basketball and most of the season (even in-conference) in football... could be as more conferences play their matchups, but wonder if there's more to it. I always find myself trying to get bigger datasets, as it might sure up the football trends (though with so many other variables in play over time, certainly no guarantee it'd be any sharper of a trend). But this is great. – JeopardyTempest Jun 1 '17 at 16:14
  • I think I was at one point DLing the ESPN dataset over long periods, looking to see how well it could predict matchups by rank, both between two teams in the rankings... and also at how strong the win % drops off by rank versus unranked teams. But you've definitely made a more solid start than I ever put together. Solid stuff. – JeopardyTempest Jun 1 '17 at 16:17
  • 1
    Definitely. I focused on ranked vs unranked, because home field/court advantage could potentially skew ranked-ranked matchups. If #14 loses to #16 on the road, is that really an upset based on the polls? Getting the football data was definitely tedious (college football reference doesn't let you sort by rank in their game finder for some reason) so I made the most of it. Massey Rating Data also omits rank. – Pawr Jun 2 '17 at 15:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.