https://fivethirtyeight.com/features/no-team-can-beat-the-draft/
During this week's NFL draft, 32 team executives will select 256 prospects in the most-hyped, most-scrutinized event of its kind. Whatever happens will make or break talent-evaluation careers, and help plot the course of each franchise over the next decade or more. And it all revolves around what is essentially a very public set of predictions.
Like traders bidding for commodities and speculating on their relative worth, each pick a team makes is essentially a statement about how it expects a player's career to turn out. Overvalue the commodity (i.e., draft a guy too early) and you end up with a bust; undervalue it and risk another team walking away with a prized prospect. Because of all of the effort and examination being poured into these predictions, the draft is a robust market that, in the aggregate, does a good job of sorting prospects from top to bottom. Yet despite so many people trying to "beat the market," no single actor can do it consistently. Abnormal returns are likely due to luck, not skill. But that hasn't stopped NFL executives from behaving with the confidence of traders.
If certain teams had superior talent-evaluation abilities then we'd expect them to achieve a greater return on their draft picks than the average team, after adjusting for where the picks were made in the draft. But if the NFL Draft follows the same general guidelines financial markets do (at least, according to the efficient-market hypothesis), there wouldn't be much of a relationship between a team or an executive's drafting performance across multiple years' worth of drafts.
If teams showed any consistency in their ability to out-draft the market, it would show up in these deviations. But, as Chase Stuart of FootballPerspective.com has also found, there's practically no correlation between a team's picking performance from one draft to the next.
While some veteran general managers were able to sustain positive returns above average over six or more years, even theirs were not unqualified success stories. Along with former Green Bay Packers GM Ron Wolf, ex-San Diego Chargers GM A.J. Smith and ex-Indianapolis Colts GM Bill Polian were the three best drafting executives in our data set on a per-pick basis. But as Pro-Football-Reference's Stuart notes, despite Smith and Polian's track records, both were fired from their posts after a series of poor drafts.
In fact, Polian and Smith merely might have been examples of what's called the "Wyatt Earp Effect." It's named for 19th-century gunslinger, whose fame came from the seeming improbability of an individual surviving countless consecutive gunfights. Any feat seems improbable in hindsight from the perspective of the people involved, but given the volume of gunfights in the Old West, the odds were actually pretty high that someone would make it through a large number of battles unscathed, simply by chance alone.
Likewise, even over a half-decade or more, some GMs would appear to beat average by chance alone. But as we saw with Polian and Smith, eventually that luck runs out.
All of this means that the NFL draft's mechanism for sorting players is largely an efficient system, in the sense that none of its individual actors have the ability to "beat the market" in the long run. Some do see short-term deviations from the mean, but those prove unsustainable over larger samples.
But there's another interpretation. Cade Massey and Richard Thaler's seminal paper (PDF), "The Loser's Curse," argues that NFL decision-makers shouldn't be so quick to attribute the apparent efficiency of the draft market to an abundance of picking skill. To do so is hubris.
As Massey and Thaler point out, the more that teams study players and gather information about them, the more assured they become in their ability to differentiate among prospects of roughly the same talent level. This leads to overconfidence, and the tendency to make what they call "non-regressive predictions" — forecasts that don't appropriately account for the uncertainty in projecting college players' performance in the NFL — about the future value of potential draftees.
. Research by TheBigLead's Jason Lisk (then writing for Pro-Football-Reference) shows that teams with top-five picks in the draft correctly identify the player who goes on to have the best career only 10.3 percent of the time, a success rate that only gets worse as things progress deeper into the draft. So a team that believes it could somehow beat the market if only it controlled its own fate can end up doing more harm than good if it trades away lower picks to move up in the draft. This is especially the case if a team uses Johnson's unrealistically optimistic chart as justification for such behavior.
