Expectation vs Reality: Exploring the Output of the #1 Draft Pick

The #1 Pick in the AFL Draft is the football off-season’s most treasured commodity, but by how much should we really value it over the picks that follow?

As a near-religious spectator of the AFL as well as it’s accompanying digital landscape, it feels to me as though every football off-season is marked with a sensational shift in the tone of the broader AFL media.

Throughout the season the headline writers tend to routinely dish up a main course consisting of endless chastising of underperforming sides, while only leaving enough room for an easily forgotten side salad of praise for those at the top of their game.

From what I’ve seen over the years, this practice within the media then seems to habitually flip on it’s head come the post-season. The pessimism and doomsaying which dominates so much of the focus during the season is all of a sudden replaced with boundless positivity.

“They’ve got the next superstar of the game on their hands.”

“This recruit is the missing piece of the premiership puzzle.”

“He’s looking Dusty-like.” (too soon?)

Football always sounds so easy when written about after the season is said and done. In my opinion, perhaps the biggest spotlight of the media’s new-found optimism shines in the lead up to the AFL Draft.

What a time it must be to be a prospective draftee! Not only are you on the verge of cracking the majors to live out what for many is a childhood dream, but for the absolute cream of the crop you are also the subject of seemingly infinite praise for your prodigious skills and apparent limitless potential to make a big difference to the club which ends up drafting you. And why shouldn’t this be the case? We are after all talking about some of the most talented teenage footballers the country has to offer.

This is the truth for none more so than those youngsters in contention to be drafted with the #1 overall pick.

I don’t think I can remember a year in my football-following life where the touted top pick was not discussed as a sure thing for a 200+ game career to be decorated with all the accolades and top billings the AFL has to give. Such is the one in-all in nature of the Australian sporting media landscape.

But here’s the thing – while some #1 picks have gone on to fulfil the prophecy that was laid out for them in their draft year, it has not been the case for all of those whose names were first to be read out on draft night.

Despite the raving pre-draft buzz that surrounds each and every one of them, for every successful career begun as the #1 pick – Luke Hodge, Nick Riewoldt, Brendon Goddard – there are also those who unfortunately never quite reach the same heights – think Paddy McCartin, Jon Patton, Tom Boyd (September ’16 heroics aside of course – Thank You from this grateful Bulldogs supporter).

This phenomenon is not limited to the AFL alone of course. Basketball’s NBA Draft has also seen a comparable mix of hits and misses at #1 as the years have gone by.
LeBron James, Dwight Howard, Anthony Davis – franchise players and household names – sit on one side of this coin. Kwame Brown, Greg Oden and Anthony Bennett – not so big-time – on the other.
The trials and mistrials of #1 picks are not limited to football alone it would seem.

The point I am trying to get across here is that while we can (and do) speculate about a draftee’s destined greatness until the cows come home, the reality is that we don’t know for sure what their career will look like or whether they will achieve the lofty ambitions set before them – even for those taken with the first pick of the Draft.

This year it is both Jason Horne-Francis as well as Nick Daicos who are the subjects of the aforementioned praise that comes with being a potential #1 pick.
With dazzling highlights packages, proven performances against big bodies on even bigger stages and in Daicos’ case a famous surname to put the cherry on the already well-decorated cake, it’s hard for fans, media outlets and recruiters alike not to get romantic about imagery of the apparent greatness that may be ahead for these two youngsters, just like those that came before them.

As stated though, not every top pick is a sure thing. There are misses too, and these misses should be acknowledged all the same.

Now of course there will be misses with every other pick in the draft after Pick #1 too – and one would assume at a higher rate too – but is it at all possible that the media’s annual outpour of adoration for top draft prospects is actually masking the fact that there is far more uncertainty around whether a draftee will go on to have an exceptional career than what is currently widely recognised?

Before the draft each year, I now find myself wondering just how big is the gap in output across the whole career of a typical #1 pick and that of those that follow?

Journalists will tell you it’s huge if their coverage is any indication. The trade strategies employed and offers tabled throughout the trade period would suggest list management at AFL clubs think the same. Just look at the offer put forward by the Crows and the Tigers to try to pry the top pick away from North Melbourne this year!

I however have a theory that the absolute market value of the #1 draft pick is never higher than before the pick has been actually been used, barring the occasional exceptional circumstance/draftee.
Would Jamarra Ugle-Hagan net as much on the open market now as the pick to nab him would have a year ago? What about Matt Rowell versus his pick? And Sam Walsh? Cam Rayner?
The answer for each is far from clear cut and probably inconsequential given the trades for these players are entire hypothetical and never took place, but it may be a useful line of questioning in helping to ascertain the exact value of Pick #1.

The AFL Draft Value Index is a good guide for what a draft pick is typically worth at the trade table, but does it do a good job of actually predicting the output of a draftee’s career? Personally, I don’t believe so.

What I want to investigate here is whether the actual output of a typical #1 draft pick is discernibly better than that of a #2 pick, a #3 pick and all those that follow? And if so, by how much?

Let’s get to it.

Quantifying career output using Replacement Player logic

The first step I want to take here is to identify a metric whic takes into account a player’s entire body of work in their career to-date, and can then be used to compare to that of their peers.

The method I have chosen to use for this study is loosely based on the concept of Value Over Replacement Player (VORP). VORP is a commonly-used player evaluation metric which originated in baseball as a means of assessing a player’s quality and output as compared to a “Replacement Player” – a hypothetical player who could be obtained with absolute ease and at minimal cost, and whose output reflected just that.

Baseball list management works a little differently to that of the AFL, so the system I have chosen to implement has a few tweaks compared to the way it is applied to baseball.

Using AFL Player Ratings from 2010 (the earliest that Player Ratings are currently available) to 2021 as a baseline, I’ve defined a Replacement Level Player’s output as the season-long average of the minimum AFL Player Rating scored for a given field position within a team’s selected 22. A hard concept to articulate, but read on and it’ll become clearer.

A quick side note – using AFL Player Ratings for this study isn’t a flawless system, I’ll admit. Like every metric it has it’s strengths and it’s drawbacks, however for the sake of a quick and relatively easy exercise I feel it is the most suitable of any statistic readily available to the general public (with big thanks to the FitzRoy crew of course!).

The approach adopted is a somewhat simplified version of VORP which I have appropriately renamed as Rating Above Replacement (RAR).
A player’s RAR in a match will be their actual AFL Player Rating less the Replacement Rating in accordance to the player’s position and the season in which the match occurred. I.e.:

For both the defining of Replacement Ratings as well as each player’s comparison to said ratings, instances where a player spent less that 50% time on ground (e.g. due to injury, starting sub etc.) have been filtered out as they do not give a true indication of the player’s output versus that of a would-be replacement.

In layman’s terms, a hypothetical Replacement Player would perform as well as the worst performed player in each position of a team’s selected 22. The general idea of this is that in reality the worst-performing player in each position of a side can (usually) be replaced with player of similar quality from the twos.

Let’s see a couple of examples:

• In 2015, the season-long average for the minimum Player Rating scored by midfielder-forwards (as defined by Champion Data) within a selected side was 7.66.
  In Round 3 2015, when Dustin Martin collected 31 touches and kicked two majors in a win against the Lions, his AFL player rating was 24.4. This means his RAR was 24.4 – 7.66 = 16.74.
  
• In 2021, the average minimum Player Rating for a general defender within a selected side was 3.72.
  Taylor Duryea’s 11 touches versus Richmond in Round 7 only earned him 1.0 Rating Points which means his RAR for the match was 1.0 – 3.72 = -2.72. Duryea’s RAR being negative means his performance for the match was below that of a Replacement Level Player.

For reference, a few examples of players who performed similar to that of a Replacement Player in 2021 for their respective positions include:

• Caleb Graham (Key Defender) – cumulative RAR of 0.43 from 12 appearances (averaged 6.1 Rating Points)
• Will Phillips (Midfielder) – cumulative RAR of 1.43 from 13 appearances (averaged 6.9 Rating Points excl. 1 match below 50% TOG)
• Toby Nankervis (Ruck) – cumulative RAR of 1.14 from 16 appearances (averaged 11.1 Rating Points)
• Ben Long (General Defender) – cumulative RAR of -2.53 from 12 appearances (averaged 3.5 Rating Points)

By cumulatively summing a player’s RAR across the course of their career (to-date), we can begin to understand how their career’s trajectory compares to players who play a similar on-field role.

Here’s a look at how Replacement Ratings have changed through the past decade within each of Champion Data’s eight defined positions.

We can see that over the last decade, the Replacement Rating of rucks has lifted significantly, perhaps due to the phasing out of the dedicated second ruck (which means in some scenarios the worst-performed ruckman in a team is actually a genuine first-choice big man).
Conversely there has been a slight decline in the Replacement Rating of both general defenders and general forwards (By “general”, Champion Data mean smalls and mediums).
The RAR system should – and does – take into account these temporal changes.

Comparing draft picks

Replacement Ratings give us the means to compare a player’s output versus a hypothetical Replacement Player within a match and season – the next step is to expand the scope of the comparison to cover their entire careers. By assessing the RAR across players’ entire careers – both finished and unfinished, we can analyse whether #1 draft picks tend to have greater output on average than any other draft position, as would be theoretically expected.

What we are looking at here is the cumulative RAR of top 10 draft picks since the 2009 AFL Draft, and what an interesting story this visual tells!

First things first – if theory translated perfectly to practice, we’d expect to see the lines perfectly ordered starting with Pick #1 (darkest green) at the top, then working down to Pick #10 (darkest red) at the bottom. As we can see, theory does not translate perfectly.
Of the top 10 picks since 2009, the #1 overall has the third-lowest cumulative output according to RAR, only topping picks #8 and #10.

Despite this, we can also see that after one season in the league, #1 picks have on average performed better than any other pick within the top 10. Could it be that #1 picks continue with their pre-draft trajectory in their first season before being reigned in by the pack?

It feels unwise to make such a sweeping generalisation when we are lumping the performances of only a dozen individuals together to form a single-number summary, however I do find it interesting to see that there appears to be no genuinely compelling evidence that a typical #1 pick will have out-performed every other pick in the draft after 10+ seasons in the league.
If we are to believe the story told by the above visual, it looks as though any pick in the top six or seven have historically performed at a similar level to that of the first overall choice.

Now don’t get me wrong here – I don’t think what I am showing here is anywhere even close enough to say having Pick #5 or Pick #6 is just as good as having Pick #1 on Draft night. Similarly, based on this sample I cannot definitively refute the notion that #1 picks perform better across their careers than other picks in the top 10.
With only 12 drafts in the scope of the study, most of whose draftees are still in the infancy of their careers, there is nowhere near enough data to make take any particularly strong stances about the fortunes or misfortunes of the collective group of #1 picks.
Nonetheless, it is interesting to see that there is not yet a distinguishable separation in RAR between Pick #1 and those that follow.

Some later picks that have fared particularly well include:

• Pick #26 – RAR 4372 (4th highest overall) – includes Zach Merrett, Jack Darling, Toby McLean, Jack Viney, Liam Ryan, Brandan Parfitt, Travis Colyer
• Pick #24 – RAR 3133 (13th overall) – Jamie Cripps, Tim Kelly, Jack Steele, Jake Carlisle, Ben Keays, Bobby Hill
• Pick #40 – RAR 3096 (14th overall) – Luke Parker, Tom Stewart, Alex Neal-Bullen, Allen Christensen

Let’s dive a little deeper into the which individual players have positively or negatively contributed to the typical career trajectory of #1 picks seen in the chart above.

Rating Above Replacement (RAR) of #1 picks

Looking at the career trajectories of the 12 #1 picks since 2009, I feel the RAR system adopted herein does a decent job of roughly matching the general public sentiment around that player.

David Swallow may not earn the plaudits of some of the household names around the competition, but a long and consistent career at the Suns which started at half back before moving full time into the midfield sees him as one of the best performed #1 picks in the last decade.
Sam Walsh‘s trajectory puts him on pace to out-do Swallow, however with only three seasons under his belt, he has not yet matched Swallow’s 11-year body of work. Lachie Whitfield, Jacob Weitering and Andrew McGrath are others who have performed well for their respective tenures in the league.

At the other end of the spectrum, we have a couple of notable “failures”. Using the word failure feels a touch harsh for this context given the specific circumstances of the careers of the players in question, but this is also as good of a time as any to reiterate the fact that holding the #1 pick is no sure thing.
Paddy McCartin and Tom Boyd both see career-long RARs below 0, which suggests that they both performed worse that a Replacement Level key forward would have given the same opportunity.
Like I said, very harsh assessment given McCartin’s concussion issues and Boyd’s well-documented mental health struggles, but the whole point of this exercise is to measure actual output rather than potential, so the numbers should be taken as they come.

The offers for North’s #1 pick

I couldn’t ignore the elephant in the room in writing this post by failing to acknowledge the monster offers for North Melbourne’s #1 Pick put forward by Adelaide and Richmond now, could I?

Looking at the proposed deal from the Crows – which at face value looked to be the better of the two offers – if North were to have accepted, it would have seen:

• North Melbourne receive Adelaide’s Pick 4 and two 2022 first round picks (tied to Adelaide & Melbourne’s finishing positions)
• Adelaide receive Pick 1 (so Horne-Francis) and a 2022 second round pick (tied to North Melbourne)

There are a few things to consider here. First of all, it would seem likely that Adelaide’s #4 pick in this year’s draft actually ends up being pick #6 after bids come for Nick Daicos and Sam Darcy, which will ultimately be matched by Collingwood and the Western Bulldogs respectively.
Does this affect the quality of player available at that pick, whatever number it ends up being? No it does not, as Daicos and Darcy are effectually off the table no matter what as it is, however it is still worth bearing in mind.

Adelaide and Melbourne’s finishing positions in 2022 come into the equation too. As does North’s long-term unfulfilled desire for an out-and-out star midfielder, Horne-Francis’ South Australian roots and the calibre of player that would likely be available at pick 4-6.
All in all it’s obviously a very complicated state of affairs.

The biggest question of course is just how good will Jason Horne-Francis be?

Well that’s just it – we really don’t know.

Neither you, nor I, nor even the wise and experienced recruiting staff of AFL clubs know the exact quality of player Horne-Francis will develop into throughout his footballing career either. We can sit here and marvel at what he has already achieved as a teenager yet to enter the league, however only time will definitively tell the answer to this question, such is the inexact science that is player recruitment in the AFL.

For that very reason, I won’t sit here and naively state that North Melbourne definitely should or shouldn’t have accepted Adelaide’s offer. I am obviously seriously lacking in subject knowledge having not lived and breathed AFL list management this season nor watched any considerable amount of Under 18s football, unlike those in the decision making positions at the two clubs in question.

But for what it’s worth anyway, given this investigation’s suggestion that pick #1s may not definitively outperform their fellow top 10 draftees, if I were in Glenn Luff’s position at North Melbourne I probably would have very genuinely considered accepting the offer put on the table by the Crows.

Final Thoughts

One of the things I like most about the AFL Draft as the main passage for new players into the league is that there is never such a thing as a certainty. Just because a player is taken in the first round does not mean he will definitely go on to be a best 22 player across a full and fruitful career. It takes intellect, masses of research and subject matter expertise for clubs to be able to draft well with the picks they have.
There is no simple cut-and-dry approach to player recruitment which beats all others, which is why I think the results of this investigation need to be taken with a sizeable grain of salt.

The insinuation found here that holding Pick #1 may be no more valuable than holding any other pick in the top half dozen or so is fraught with uncertainty. One club taking a particular player with the first pick does not mean every other club would have done the same, so it is difficult to attribute any shortcomings to the #1 Pick itself, as opposed to the club wielding it.
Similarly, it is impossible to tell if the players who ultimately were taken with the first pick of the Draft would’ve experienced the same career trajectories had they landed at a different club, via a different pick. We are dealing with people here after all, not just numbers on a page.

Still though, my own personal learnings from having written this post are that I will try to go into this year’s Draft with an open mind. I – like many others – will continue to acknowledge that Jason Horne-Francis and Nick Daicos are currently considered to be the two best prospects in the nation, however I’ll also be sure to recall that just because they currently have more runs on the board than their fellow draftees does not mean they will be assured to remain the best performed of their crop in the years to come.
I’ll also keep in mind the outcome of the Rating Above Replacement (RAR) analysis discussed earlier. The handful of picks immediately following the #1 Pick historically have had just as good of a track record in the AFL than the first picked player, so perhaps the gap between the first pick and those than follow truly is smaller than the media make it out to be?

Admittedly sample sizes are the other big issue with this study. Ideally we would have another 10-20 years worth of AFL Player Ratings data to play around with or otherwise having that same amount of additional draft crops to analyse. Only assessing back to the 2009 Draft means that individual stories of success or failure tend to have an unreasonably large of an impact on the results of the investigation. This however is unfortunately often the nature of AFL-related studies where sufficient historic data dating back multiple decades is typically not readily available.

I’ll put it in my diary to revisit this in topic 20 years to see how well these findings are holding up. Someone remind me if I forget.