Amateur Analysis of 31 Years of the NFL Draft

09/11/2014

Do you have that annoying friend who absolutely hates your sports teams? I am describing the person who sends a weekly barrage of emails full hate and over indulges in schadenfreude when your team loses. I have that friend and he is a Miami Dolphins fan. I am a Patriots fan and season ticket holder for more than twenty years. The Tom Brady era has been toxic for the Miami Dolphins and the AFC East in general. This toxicity manifests itself in a weekly barrage about Patriot cheating, film crews, playbook theft, hometown refs, video recording innuendo and general hatred towards Bill Belichick.

A consistent theme for admonishment from my friend is the drafting record of the Patriots. This manifests itself weekly on sports radio and websites in the form that the Patriots have cost Tom Brady at least three Super Bowls because they cannot draft and Bill Belichick is a terrible GM. The Evil Genius Bill let players like Milloy go, would not pay Law, traded Seymour, traded Branch, would not pay Welker and now he has traded Mankins! His drafts are terrible and we have squandered the Tom Brady era!

Sports radio is full of people who call up and say “the Patriots should have picked Randy Moss in 1998!” That is actually correct, but Bill had nothing to do with this as he was employed by the NY Jets organization in 1998. I am driving home on Wednesday of this past week. It should be noted that the Patriots just blew out the best team in football three days before and what am I subjected to? A diatribe of whining from a Patriot fan who is still upset with the Lawrence Maroney pick, the Adalius Thomas signing and the Brandon Lafell signing over Emanuel Sanders. Really? The Patriots have a 70% winning percentage in the years 1998-2013 and we have to listen to a crazy person upset about Lawrence Maroney and Adalius Thomas?   Really? This is the alluring nature of the NFL draft and the elements of draft porn available to the casual fan.

Teams draft for need and their system, they do not often draft the best player on the board and it should be noted that there is a high error selection rate. More on that later, but it should be noted that teams often have hot and cold streaks in the draft. Sometimes they put together a few really good years and this is highly correlated with a Super Bowl appearance and sometimes the put together a cold streak as the Patriots did in the late middle part of last decade.

The real draft failures for the Patriots occurred in the years 2006, 2007 and 2008.

• 2006: Pats draft L. Maroney, C. Jackson and D. Thomas. Pats could have been J. Evans, M.J. Drew, G. Jennings or N. Mangold.

• 2007: Pats draft B. Meriwether with pick #24 when they could have picked J. Beason, L. Woodley or J. Staley.

• 2008: Pats draft J. Mayo, T. Weatley, S. Crable, J. White. Nothing wrong with J. Mayo, but the others picks could have been R. Rice, C. Nicks and P. Garcon.

All of this good-natured banter about the NFL Draft started me thinking about the draft. I then watched the 30 for 30 Shorts on the trade of Herschel Walker and how this turned the Cowboys around. A flurry of question came to mind: How well do teams draft? Does drafting lead to championships? Who are the best drafting teams? Does trading down make sense? Is the drafting record of the Patriots under Bill Belichick good or bad? Are all teams bad? If all teams are bad should teams be graded on a curve? What would that curve look like? Does drafting well equal wins or is it just a quarterback driven league? Is it easy to draft well? Are there any teams that stand out? Is the best strategy to pick the best available player or draft for need?

Frame 1: Player Data

The first step I took was to go look for websites that tracked all drafts and stack ranked them on the quality of the draft. My thought was this would take a few minutes as surely one of the advanced football metric sites had already done this work. Turns out I was mistaken or I could not find the site with the data and ended up creating my own analysis. I did find a lot of player data at www.pro-football-reference.com. I took their player and draft data from 1983 forward to build the following analysis. The data is divided into two eras. 1983 to 1997, which is the John Elway Era and 1998 to present day, which is the Peyton Manning Era. It could be divided many ways, but the 1983 draft was one of the most talented drafts with six Hall of Fame players selected in that draft and was the beginning the first generation of the quarterback driven league. I would argue that Peyton Manning was the second iteration of the quarterback driven league.

Before you read any further, here are some caveats about the data:

• If the source I used for player data (pro-football-reference.com) is unacceptable, read no further. There are no player rankings, no player evaluations in this analysis. I simply used historical player data to create an analysis around (1) the quality of selection and (2) the ability of teams to retain talent and can these metrics be correlated with (3) winning percentage and (4) Super Bowl appearances.

• There are a number of charts and tables, about 29 of them, within this post.  I am certain I made errors.  If you find one and point it out, I will correct the error.

• It is about Super Bowl appearances. It is really about Super Bowl wins, but getting to a Super Bowl is actually quite difficult so credit is given to teams who can acquire enough talent to make it to the final game. The Super Bowl is an outlier of game. It has a two-week buildup, winner take all format with an unusual halftime length. For those reasons and more, I think a lot of credit should be given for being able to build a program that can deliver a team to the final game.

• The structure of the game started to change with the start of the John Elway and Peyton Manning Eras, meaning the game increasingly became more of a passing game with each of these drafts. That is the reason for the division of the two eras.

• The league went from 28 teams to 32 teams. This had an effect on the draft. Those four additional selection slots or 16 slots over the first four rounds means that late round picks are worthless. Yes, Tom Brady was selected in the sixth round, but historically only 6% of the total value of the draft is selected in this round of the draft.

• The ability to study players has become easier with the proliferation of the Internet.

• The player dataset is complete for 1983 to 1997 as there are no active players from the 1997 draft or earlier currently in the NFL.

• The player dataset from 1998 forward is still on-going as in the value of active players is still changing

• Pro-Football Reference measures the value of player by two numbers. DrAV is the value that the team who drafted the player realized. CarAV is the value of the player over the career of the player. An example would be Baltimore Colts who drafted John Elway in 1983, but then traded him to Denver. The Colts received zero DrAV value for Elway because he never played a down for Colts and all his career value was realized by Denver.   The Indianapolis Colts drafted Peyton Manning and realized fourteen years of his career value (CarAV) with the last 2.5 years (ongoing) being realized by Denver.

• Answering the earlier complaints from Patriot fans regarding Milloy, Law, Seymour, Branch and Mankins:
  • Milloy CarAV of 77 and Patriots realized 49 of the 77 points (63%)
  • Seymour CarAV of 89 and the Patriots realized 69 of the 89 points (77%)
  • Law had a CarAV of 87 and the Patriots realized 67 of the 87 points (77%)
  • Branch had a CarAV of 52 and the Patriots realized 39 of the 52 points (75%)
  • Welker was undrafted
  • Mankins has a CarAV of 91, but his final value will be determined over his remaining years.

Frame 2: Conclusions

There is no need to put the conclusions at the end; here are some facts about the draft that are relevant to the analysis:

• 94% of the total CarAV (i.e. Career Value) of each draft is available in first 128 picks (rounds 1-4)

• 57% of the total CarAV of each draft is available in the first 64 picks (rounds 1-2)

• 5% of the total CarAV of each draft is available in first 32 picks (round 1)

• On average only 30.2% of the total CarAV is selected in the first round

• Of the CarAV drafted in the first round, teams will fail to retain 23.8% of that value over the career of the player

• On average teams leave 33.9% of total possible value in the 1^st round to the following rounds. In other words, only 66.1% of the picks are the correct choice with the aid of hindsight. That would be a grade of D on a high school test.

• Drafts contain some element of Game Theory which I recently wrote about here; meaning teams are not always drafting the best player available. Teams are drafting as to their need, their scheme and within that motivation is found the element of Game Theory. For example, in the 2001 draft Drew Brees was available to the Colts with the 30 pick, but they chose to select Reggie Wayne.

• Super Bowl contenders highly correlate with 3-5 very strong draft years prior a SB appearance

• Super Bowl contenders are really built through the draft, hence the value of having a program over having a individual player or great coach.  The value is in the system or the program.

• The draft is not easy to master

Frame 3: How is Player Value Spread Across Draft Rounds?

Tom Brady was drafted in the sixth round of the 2000 draft. James Harrison went undrafted in 2002. This is what I call draft porn. We love to cite factoids like this as if we knew how great these players would turn out to be. If our team had just taken these players the past would be different. Let’s set aside one debate. No NFL team knew or projected that Tom Brady to be a Hall of Fame player.

If any team had on the scouting report that Brady was HOF potential and passed on him for five to sixth rounds, then those employees should be fired. No GM or scout was sitting in their team’s draft room in 2000 saying “Brady is a HOF player, but we can get him latter in the draft, let’s take a few players in rounds 2-5 before we pick him.”   As stated in Frame 2, ~94% of career value within each draft is available in rounds 1-4, or the first 128 picks. This value is diffused over the sixth rounds of the draft because teams are drafting for need, not selecting the best player available. Throw in the variability of what will the player become and it is highly subjective which is what makes it so much fun.

Frame 4: DrAV Value

This chart shows the drafted value (DrAV) of all the players taken per round since 1983. The distribution of value selected per round is shown in the chart. This is a measurement of the quality selection, the ability of teams to make wise player selections. 30.2% of the total CarAV of the players chosen in the draft are selected in the first round. 74.9% of the total CarAV of the players chosen in the draft are selected in rounds 1-4. As stated previously, 94% of CarAV is available in rounds 1-4, but historically only 74.9% is chosen.

Frame 5: CarAV Value Lost

This is a measurement of retention, the ability of teams to keep the talent selected in each draft by round. On average 23.8% of first round drafted value is not retained by teams over the career of the player. This was detailed specific to the Patriots at the end of Frame 1. Players are released for many reasons; not a fit to the 53-man roster, beat out by a better player and did not make team, player is traded, the team did not want to pay them, etc. On average 56.4% total career value (CarAV) of the players selected in rounds 1-4 is unrealized by the teams that selected the player, which means that value is available to other teams through free agency and trades.

Frame 6: Team Rankings by DrAV

Using the DrAV metric, the first table lists each team in alphabetical order from 1983 to 2013. Yellow shading indicates Super Bowl loser, dark green indicated Super Bowl winner and the light green represents strong drafts leading to a Super Bowl appearance. The gray shaded part of the table indicates drafts with no active players in the NFL. The chart after the table shows the trend of the trends of the DrAV metric. The blue line is total DrAV available by year in the draft and is indexed to the right. The falloff from 2007 is because the value for those players in drafts from 2007 forward is still in flux.

This second chart contains the same data, but is sorted by highest average DvAV over 31 years. 51% of teams making a Super Bowl appearance over the past thirty-one years come from the ten teams with the highest DrAV average. In terms of draft quality or draft depth, there is a 1,850 point difference between the ’83 class and the ’91 draft class. That is the equivalent of 13 players as talented as John Elway. In other words, the ’83 draft had 13 more Elways in player value available than the ’91 draft class.

The last DrAV value sorted table is from 1998 forward. 1998 was the Peyton Manning draft year and this sorting shows how the five most recent drafts are still maturing. There is a lot of truth to the statement that it takes about 5-7 years for the value of a draft to be understood. That is why the DrAV value of the 2007-2013 drafts show low total value scores. However, the top 10 drafting teams from 1998-2013 account for 50% of the Super Bowl appearances in this time frame.

Frame 7: Team Rankings by Simple Stack Rank

If we removed the DrAV qualitative metric and just ranked the teams 1-32 with the score of 32 being the best and 1 being the worst, how does the table change?

We often forget how strong the Oilers and Titans were in the 1980s and 1990s. The Oilers under Warren Moon had several great years, unfortunately for them the Buffalo Bills played just a little bit better one day. The Titans were in the Super Bowl and one miracle short of victory.

The following table shows that the Patriots are a top 3 drafting team since 1998, but this does not mean they are flawless. The 2006-2007-2008 drafts were devastating for the team. New England had five top 64 picks in those three years (1 in top 16, 2 in top 32, 1 in top 48 and 1 in top 64).  There is no point in analyzing these failures in detail, but what it does show is that teams have ups and downs and I would speculate these are more related to turnovers in the coaching, scouting and management organizations. Different philosophic views will dictate different player selections. Before 2006-2008, the Patriots were a very good drafting team and they became a very good drafting team after, therefore it would be interesting to know what was different in those three years considering that the coach, GM, owner and QB were the same.

Frame 8: How Well Do NFL Teams Choose?

A big point of agitation from fans of every team is around draft quality. I think most fans think their team’s drafts are horrible. The following table and charts were created by taking the total CarAV available (only known in full with time), the total 1^st Round CarAV possible, the actual 1^st Rd CarAV selected and calculating the selection rates. The column 1^st Rd CarAV Delta is difference between what would have been a perfect draft sequence and what actually occurred. The average delta between ’83 and ’06 is 741 CarAV points, which is equivalent to four Peyton Manning caliber players missed in each draft.

Some trending around these metrics show that teams are getting better at selection and this is shown in the following two charts with percentage of possible CarAV trending up and delta between CarAV-DrAV trending lower.

Ultimately what NFL fans want to know is how well do NFL teams choose? The answer is: not well, but the spread between the best and the worst is not as great as one would think and we will cover that in the last frame. The following chart shows the quality or depth of the draft in the top blue line, followed by want would have been the perfect selection choices if NFL teams could predict the future, followed by the actual selection quality in the green line. This is also what makes the NFL draft so much fun for fans. We can all be the perfect GM.

The above chart shows trending to 2006, because we have a good base of statistics around the value of the players selected in the draft up to 2006. The blue line is the total CarAV available in the draft or what draftnicks would call draft depth. The red line would be a perfect draft with the aid of hindsight and the green line is the actual CarAV selected.

Frame 9: Does Pick Location and the Number of Picks Matter?

Does draft position and the number of picks matter? I hear this often from Patriot fans because the Pats have had a tendency to trade down to accumulate more picks. Does having more picks matter?

The following table shows the number of picks for each team over the initial 64 picks in the draft. The draft positions are broken down into four groups of 16 (1-16, 16-32, 33-48, 69-64). Over thirty-one years, 53% of the total career value of the players available, are taken in the first 64 picks.   Since 1998 this percentage has risen to 57.9%.   The remaining 42% is spread over the next ~160 picks.  For that reason it is focused on the first 64 picks. I did not adjust for trades. The picks are what they are and did not go back and hypothesize if team X had not traded out of the slot. Since 1998 there have been 1,024 picks in the first 64 slots.   There are five charts here to break down all the picks in the Peyton Manning Era.

Table 1 – Sorted by Team Alphabetical Listing / With Weighted Pick Value (4, 3, 2, 1)      

Table 2 – Sorted by Total Number of Picks / With Weighted Pick Value (4, 3, 2, 1)      

Newsflash, having the most picks does not make you a winner unless you are really good at selection and if you are really good at selection you will have less picks and lower quality position from which to choose over time. PIT is the highest ranked team over the past fifteen years, yet they are in the middle of pack when the number of picks is stacked ranked. That tells us a few things about PIT, (i) they are very good at selection and (ii) they have system or program that allows them to draft players that are productive in their system. From the 1998 draft forward, PIT has the highest DrAV average score of 109.4, the second lowest CarAV average score of 16.8 all while having a total of 30 picks which is one below the median of 31 across all teams over the same period of time. This is a mark of a high quality drafting team.

In contrast Washington has had 25 picks, they are middle of the stack in terms of retention and they are ranked as the worse team in terms of selection. Detroit has had the most high quality picks, but is a bottom five team in terms of selection and middle of the pack in terms of retention.

Table 3 – Total Number of Picks / Sorted by Weighted Pick Value (4, 3, 2, 1)      

Table 4 – Sorted by Highest Number of Picks in 1-16 / Sorted by Highest Number of Picks 17-32      

Table 5 – Sorted by Highest Number of Picks in 33-48 / Sorted by Highest Number of Picks 49-64      

As for my Patriots, they have had selections in the first 16 positions only three times and that is fewest over the time period compared to the top teams having 11-13. When the Patriots did have a pick in the first 16, they did pick well as shown by selecting Seymour in 2001, Ty Warren in 2003 and J. Mayo in 2008. The only quibble a Patriots fan could make would be rather than Ty Warren in 2003, the team should have taken Troy Polamalu, Calvin Pace or Jason Witten. It should be noted that the Ty Warren pick is a perfect example of game theory. In 2003 the Patriots did not need a safety or tight end, hence they drafted for need, which was the defensive line, thus allowing other teams to draft higher quality players.

Frame 10: Putting it All Together / Does Draft Class Quality = Wins?

The two tables below are sorted differently. The left hand table is sorted by the highest difference between CarAV – DrAV, which indicates which teams drafted the best and retained their talent. The right hand table is sorted by which teams have the highest DrAV average from 1998 through 2013.

Question…does quality drafting = wins?

The first observation about the left hand table is it is divided in quarters and it is sorted by DrAV-CarAV delta. The teams in top 25% have a wining percentage of 58.1% versus 50.5% for the next 25%, which translates into 155 more wins and that stretches to 235 wins over the bottom 25%.

Looking at the right side table, it is sorted by highest DrAV average. This chart is segmented into thirds. The gray shaded part of the table highlight teams that are average drafters. Sometimes these teams have great drafts, but great drafts are often followed by a poor draft. These teams lack consistency. The bottom third of the table are teams that are consistently poor drafting teams. The teams in the top third of right hand sort account for 56% of Super Bowl appearances in the 1998-2013 time frame compared to 43% for the bottom two thirds. In other words 33% of the teams appear in the Super Bowl 56% of the time and 66% of the teams appear in the Super Bowl 33% of the time.

The following two tables rank teams first by difference between DrAV-CarAV and then by DrAV only. No surprise from Frame 9 that PIT is top of the chart.

This next chart sorts by winning percentage.

I often hear from my Miami friend that winning in the NFL is all about having an elite QB. There is some truth to that, but not nearly as much as we would like to believe. DEN clearly believes in this theory because twice they have gone out and trading for an elite QB in John Elway and Peyton Manning. Drew Brees to NOR changed the fortune of that franchise, but it did not do wonders for SDG from 2002-2005. To play in and win Super Bowls, it does require more than just a quarterback.

Frame 11: 31 Years of Draft Rankings

These tables show the teams from top to bottom for each year based the total DrAV of the players selected. The final value of the draft is only known in time, but the delta between the top team and the bottom team is quite large. From 1983-2006, the difference between the top position and the bottom position has averaged 228 points. To put that into perspective, Dan Marino had a CarAV of 145 points.

Frame 12: Selected Team Drafting Charts

I did not take the time to produce a draft chart for every team in the NFL, but I did a few of the popular teams as well as best and worst drafting teams over the past 31 and 16 years.

 

 

 

 

/wrk