### The Approach

At the time this post will go live, the draft will be 10 days away! With that in mind, its time for the second of my series of blog posts about using mock draft data to predict the NFL Draft. In my last blog post, I layed out my theory of mock drafts and how I think about using mock drafts as a way of assessing a player’s stock throughout the NFL Draft process. This post aims to get a bit deeper into the data and statistical aspects of using mock draft data to predict the actual outcomes of the NFL Draft.

I decided to start down the road for this project by exploring mock draft data for the 2018 NFL Draft, which means that 2018 is the only draft I have data for, which is a small sample but its all I’ve got and as good a place to start as any. For my 2018, I collected the data by hand (more on that choice another time) with a focus on getting a diverse sample of mock drafts from media members, fans, and draft experts. Ultimately, I collected *395 mock drafts*: *229* from fans, *90* from the media, and *76* from experts. This might sound like a lot but for the 2019 draft, I’ve collected more than double this number of mock drafts. This will ideally make any estimates I make using the mock draft data less-noisy in terms of the spread of mock draft selections for each draft-eligible player.

Because we have to start from somewhere, I thought it was a good idea to explore how well the basic measures of central tendenency in data: the average, the median, and a weighted average based on how far (in numbers of days) a mock draft is from the date of the actual NFL Draft. From there, we can move into model-based estimates using two basic statistical estimation tools: the linear regression and the Loess regression.

A linear regression take a series of points and tries to fit a line of best fit in the data based off where the points cluster to minimize the difference between your prediction and the actual outcomes you’re trying to predict. In this case, we’re trying to use mock draft data to predict actual NFL draft selections. A loess regression takes a slightly different approach in that it attempts to fit a line through your data but attempts *smooth* the data you provide it to account for change over time, which is often why its known as a “moving regression”.

Let’s begin by setting up the data and calculating the basic summary statistics for our draft eligible players in mock drafts.

```
options(stringsAsFactors = FALSE, scipen = 7)
library(tidyr)
```

`## Warning: package 'tidyr' was built under R version 3.6.2`

`library(dplyr)`

```
##
## Attaching package: 'dplyr'
```

```
## The following objects are masked from 'package:stats':
##
## filter, lag
```

```
## The following objects are masked from 'package:base':
##
## intersect, setdiff, setequal, union
```

```
library(ggplot2)
library(plotly)
```

```
##
## Attaching package: 'plotly'
```

```
## The following object is masked from 'package:ggplot2':
##
## last_plot
```

```
## The following object is masked from 'package:stats':
##
## filter
```

```
## The following object is masked from 'package:graphics':
##
## layout
```

```
library(teamcolors)
library(ggrepel)
library(ggthemes)
library(broom)
`%notin%` <- function(x, y) !(x %in% y)
read.csv("https://github.com/benjaminrobinson/2018NFLMockDrafts/raw/master/data/2018%20NFL%20Mock%20Draft%20Data%20-%20Projections.csv") %>%
filter(
name %notin% c(
"Christian Wilkins",
"Clelin Ferrell",
"Mitch Hyatt",
"Austin Bryant",
"Trey Adams",
"Dre'Mont Jones",
"Adonis Alexander",
"Bryce Love",
"Iman Marshall",
"Cam Smith",
"Martez Ivey",
"Clayton Thorson",
"Jarrett Stidham",
"Ken Webster",
"Parris Campbell",
"Damien Harris",
"Dante Booker",
"Beau Benzschawel",
"Jake Browning",
"Porter Gustin",
"Brian Hill",
"Daylon Mack",
"Grant Newsome",
"LJ Scott",
"Michael Dieter",
"Nick Fitzgerald",
"TJ Edwards",
"Will Grier",
"Kendall Joseph",
"Jerry Tillery",
"Brock Ruble",
"Andre Dillard",
"Byron Cowart",
"CJ Conrad",
"George Panos",
"Caleb Wilson",
"Dontavius Russell",
"Sam Beal",
"Chase Hansen",
"Adam Breneman",
"Jaylon Ferguson",
"Casey Tucker",
""
) & !(name == 'Josh Allen' & position == 'LB')
) %>%
mutate(
date = as.Date(date, format = "%m/%d/%Y"),
draft_weight = 1 / ((max(date) + 1) - date) %>% as.numeric,
draft_year = 2018
) -> prj
read.csv("https://github.com/benjaminrobinson/2018NFLMockDrafts/raw/master/data/2018%20NFL%20Mock%20Draft%20Data%20-%20Actuals.csv") %>%
mutate(date = as.Date(date, format = "%m/%d/%Y"),
draft_year = 2018) -> act
prj %>%
left_join(
act %>%
rename(actual = pick) %>%
distinct(round, actual, name, position, school, team)
) %>%
mutate(team = ifelse(is.na(actual), "Undrafted", team),
round = ifelse(is.na(actual), 8, round),
actual = ifelse(is.na(actual), 257, actual)) -> pick
```

`## Joining, by = c("name", "position", "school")`

```
prj %>%
mutate(n_drafts = n_distinct(paste0(site, date))) %>%
group_by(name, position, school, draft_year, n_drafts) %>%
summarize(
draft_count = n(),
average_draft_position = mean(pick, na.rm = TRUE),
median_draft_position = median(pick, na.rm = TRUE),
weighted_average_draft_position = weighted.mean(pick, draft_weight, na.rm = TRUE),
sd = sd(pick, na.rm = TRUE),
sd = ifelse(is.na(sd), NA, sd)
) %>%
ungroup %>%
mutate(
draft_share = draft_count / n_drafts,
draft_share = ifelse(draft_share > 1, 1, draft_share)
) %>%
select(-n_drafts) %>%
left_join(act %>% distinct(round, pick, name, position, school, team)) %>%
mutate(pick = ifelse(is.na(pick), max(act$pick) + 1, pick),
round = ifelse(is.na(round), max(act$round) + 1, round)) %>%
gather(metric,
value,
-name,
-position,
-school,
-draft_year,
-pick,
-round,
-team,
-draft_share,
-draft_count,
-sd) %>%
mutate(
metric = gsub("_", " ", metric),
metric = gsub("(^|[[:space:]])([[:alpha:]])", "\\1\\U\\2", metric, perl = TRUE),
lwr = ifelse(is.na(sd), value, value - 1.96*sd),
lwr = ifelse(lwr <= 1, 1, lwr),
upr = ifelse(is.na(sd), value, value + 1.96*sd),
upr = ifelse(upr >= 256, 256, upr)
) %>%
left_join(
teamcolors %>%
filter(league == 'nfl') %>%
rename(team = name)
) %>%
group_by(metric) %>%
mutate(rank = dense_rank(value)) %>%
as.data.frame -> agg
```

`## Joining, by = c("name", "position", "school")`

`## Joining, by = "team"`

## The Metrics

Now that we’ve done that, let’s use a univariate linear regression to compare the three basic measures of mock draft position against the actual mock draft position to see which one is most accurate:

```
agg %>%
filter(!is.na(sd)) %>%
group_by(metric) %>%
do(mock = lm(pick ~ value, data = .)) %>%
glance(mock) %>%
as.data.frame
```

```
## metric r.squared adj.r.squared sigma statistic
## 1 Average Draft Position 0.5423606 0.5409077 59.61236 373.3148
## 2 Median Draft Position 0.5063135 0.5047462 61.91562 323.0567
## 3 Weighted Average Draft Position 0.6332899 0.6321257 53.36249 543.9892
## p.value df logLik AIC BIC deviance df.residual
## 1 2.072960e-55 2 -1744.653 3495.306 3506.583 1119394.7 315
## 2 3.292161e-50 2 -1756.670 3519.340 3530.617 1207566.5 315
## 3 1.354256e-70 2 -1709.544 3425.087 3436.364 896979.8 315
```

So it turns out that in a simple linear regression the basic metrics explain about **TWO-THIRDS** of the variation in actual mock draft selections. That’s pretty great on its face for a single metric!

However, this takes into account the whole universe of mock draft and most folks only care about the first round of the NFL Draft and that’s probably the only part of the draft worth using the mock draft data to predict the outcome of the draft en-masse. Let’s focus on the first round of the draft:

```
agg %>%
filter(!is.na(sd)) %>%
ggplot(aes(
value,
pick,
color = factor(team, levels = team %>% unique %>% sort),
fill = factor(team, levels = team %>% unique %>% sort)
)) +
geom_point(size = 3) +
geom_text_repel(aes(label = name)) +
geom_abline(slope = 1,
intercept = 0,
size = .5) +
geom_smooth(aes(group = metric), method = 'lm', formula = 'y ~ x') +
scale_color_manual(
values = agg %>% filter(!is.na(team)) %>% distinct(team, primary) %>% arrange(team) %>% select(-team) %>% unlist %>% unname
) +
scale_fill_manual(
values = agg %>% filter(!is.na(team)) %>% distinct(team, secondary) %>% arrange(team) %>% select(-team) %>% unlist %>% unname
) +
scale_x_continuous(limits = c(1, 32), breaks = c(1, 16, 32)) +
scale_y_continuous(limits = c(-10, 174),
breaks = c(1, 32, 64, 96, 137, 174)) +
facet_wrap(~metric, ncol = 1) +
theme_fivethirtyeight() +
theme(legend.position = "none") +
labs(x = "Mock Draft Position",
y = "Actual Draft Position",
title = "2018 NFL Draft 1st Round Projections",
subtitle = "Compared to Actual Draft Position",
caption = "Data and Graph by @benj_robinson"
)
```

`## Warning: Removed 828 rows containing non-finite values (stat_smooth).`

`## Warning: Removed 828 rows containing missing values (geom_point).`

`## Warning: Removed 828 rows containing missing values (geom_text_repel).`

```
agg %>%
filter(!is.na(sd) & rank <= 32) %>%
ggplot(aes(
value,
pick,
color = factor(team, levels = team %>% unique %>% sort),
fill = factor(team, levels = team %>% unique %>% sort)
)) +
geom_point(size = 3) +
geom_text_repel(aes(label = name)) +
geom_abline(slope = 1,
intercept = 0,
size = .5) +
geom_smooth(aes(group = metric), method = 'lm', formula = 'y ~ x') +
scale_color_manual(
values = agg %>% filter(!is.na(team)) %>% distinct(team, primary) %>% arrange(team) %>% select(-team) %>% unlist %>% unname
) +
scale_fill_manual(
values = agg %>% filter(!is.na(team)) %>% distinct(team, secondary) %>% arrange(team) %>% select(-team) %>% unlist %>% unname
) +
scale_x_continuous(limits = c(1, 32), breaks = c(1, 16, 32)) +
scale_y_continuous(limits = c(-10, 174),
breaks = c(1, 32, 64, 96, 137, 174)) +
facet_wrap(~metric, ncol = 1) +
theme_fivethirtyeight() +
theme(legend.position = "none") +
labs(x = "Mock Draft Position",
y = "Actual Draft Position",
title = "2018 NFL Draft 1st Round Projections",
subtitle = "Compared to Actual Draft Position",
caption = "Data and Graph by @benj_robinson"
)
```

`## Warning: Removed 8 rows containing non-finite values (stat_smooth).`

`## Warning: Removed 8 rows containing missing values (geom_point).`

`## Warning: Removed 8 rows containing missing values (geom_text_repel).`

As we can see, even with the first round, there is quite a bit of noise in the data. Due to a number of outliers, mostly in the direction of players getting projected to go in the first but going later (as well as players who were projected to go in later rounds but went in the first), the metric does not predict the first round as well overall.

It seems that if we are to use a metric Weighted Average Draft Position metric then we can remove some of those NFL Draft false positives: players that we thought might go in the 1st round based on data points from earlier in the draft process and that really didn’t have much business going in the 1st round as the process played out. Let’s look at a common performance metric before we move on to modeled estimates of Mock Draft Position:

```
## Player Subset Metric
## 1 First Round Actual Draft Picks Average Draft Position
## 2 First Round Actual Draft Picks Median Draft Position
## 3 First Round Actual Draft Picks Weighted Average Draft Position
## Mock Draft Position Mean Squared Error Mock Draft Ranking Mean Squared Error
## 1 191.8729 364.59375
## 2 168.0625 98.84375
## 3 283.6368 423.12500
## Mock Draft Metric Correlation Mock Draft Ranking Correlation
## 1 0.6918107 0.6831410
## 2 0.6947786 0.6671132
## 3 0.6252490 0.6017523
```

```
## Player Subset Metric
## 1 First Round Mock Draft Rank Average Draft Position
## 2 First Round Mock Draft Rank Median Draft Position
## 3 First Round Mock Draft Rank Weighted Average Draft Position
## Mock Draft Position Mean Squared Error Mock Draft Ranking Mean Squared Error
## 1 754.2557 756.9375
## 2 2809.6447 3183.6491
## 3 110.7990 103.8125
## Mock Draft Metric Correlation Mock Draft Ranking Correlation
## 1 0.6165475 0.5905570
## 2 0.4558563 0.4680832
## 3 0.7820824 0.8011319
```

```
## Player Subset Metric
## 1 First Round Mock Draft Position Average Draft Position
## 2 First Round Mock Draft Position Median Draft Position
## 3 First Round Mock Draft Position Weighted Average Draft Position
## Mock Draft Position Mean Squared Error Mock Draft Ranking Mean Squared Error
## 1 692.6951 686.3590
## 2 1920.4643 2220.3265
## 3 437.5016 407.5833
## Mock Draft Metric Correlation Mock Draft Ranking Correlation
## 1 0.5530910 0.4949965
## 2 0.4988623 0.4920950
## 3 0.6105045 0.6331873
```

What we see confirms the need to have multiple metrics for comparison purposes. While Weighted Average Draft Position minimizes the Mean Squared Error for a player’s estimated draft position and draft rank, with the benefit of hindsight, the Median Draft Position metric predicts the players who actually went in the 1st round best, mostly because as part of its definition, the median does not react as strongly to outliers. However, based on my assumptions and the quantity and quality of my 2019 NFL Mock Draft data, I will use the Weighted Average Draft Position going forward as my central tendency metric of choice for predicting actual draft position.

## The Models

From the world of aggregated metrics, let’s move to the more abstract world of models. I’ll test two models based on the raw mock draft data (not aggregates): good old linear regression and the Loess regression. First of all, let’s put together the dataset:

```
suppressWarnings(
bind_rows(
prj %>%
group_by(name, position, school, draft_year, metric = "Linear Regression") %>%
do(mock = lm(pick ~ date, data = .)) %>%
mutate(value = predict(mock, data.frame(date = as.Date("2018-04-26")), interval = 'confidence', level = .95)[1],
lwr = predict(mock, data.frame(date = as.Date("2018-04-26")), interval = 'confidence', level = .95)[2],
upr = predict(mock, data.frame(date = as.Date("2018-04-26")), interval = 'confidence', level = .95)[3],
lwr = ifelse(lwr <= 1, 1, lwr),
upr = ifelse(upr >= 256, 256, upr)) %>%
select(-mock) %>%
filter(!is.na(lwr)),
prj %>%
left_join(
agg %>%
distinct(name, position, school, draft_count)
) %>%
filter(draft_count > 2) %>%
group_by(name, position, school, draft_year, metric = "Loess Regression") %>%
do(mock = loess(pick ~ date %>% as.numeric, data = .)) %>%
mutate(value = predict(mock, data.frame(date = as.Date("2018-04-26")), se = TRUE)[1] %>% unlist,
se = predict(mock, data.frame(date = as.Date("2018-04-26")), se = TRUE)[2] %>% unlist,
lwr = value - 1.96*se,
lwr = ifelse(lwr <= 1, 1, lwr),
upr = value + 1.96*se,
upr = ifelse(upr >= 256, 256, upr)
) %>%
select(-mock, -se) %>%
filter(!is.na(value) & !is.na(lwr))
) %>%
left_join(
act
) %>%
left_join(teamcolors %>%
filter(league == 'nfl') %>%
rename(team = name)
) %>%
group_by(metric) %>%
mutate(rank = dense_rank(value),
residual_pick = abs(value - pick),
residual_rank = abs(rank - pick)) %>%
as.data.frame
) -> models
```

```
## Joining, by = c("name", "position", "school")Joining, by = c("name", "position",
## "school", "draft_year")Joining, by = "team"
```

Let’s run the same diagnostics we did for the aggregate metrics:

```
models %>%
group_by(metric) %>%
do(mock = lm(pick ~ value, data = .)) %>%
glance(mock) %>%
as.data.frame
```

```
## metric r.squared adj.r.squared sigma statistic p.value df
## 1 Linear Regression 0.6005095 0.5985981 44.04117 314.1664 1.614930e-43 2
## 2 Loess Regression 0.6071465 0.6049760 40.99118 279.7315 1.436171e-38 2
## logLik AIC BIC deviance df.residual
## 1 -1097.0526 2200.105 2210.161 405381.6 209
## 2 -938.2045 1882.409 1892.038 304130.0 181
```

Looks like overall, the linear and Loess regression modesl perform similarly well with an R squared value that explains about 60% of the variation in the actual 2018 NFRL Draft selections.

```
models %>%
ggplot(aes(
value,
pick,
color = factor(team, levels = team %>% unique %>% sort),
fill = factor(team, levels = team %>% unique %>% sort)
)) +
geom_point(size = 3) +
geom_text_repel(aes(label = name)) +
geom_abline(slope = 1,
intercept = 0,
size = .5) +
geom_smooth(aes(group = metric), method = 'lm', formula = 'y ~ x') +
scale_color_manual(
values = models %>% filter(!is.na(team)) %>% distinct(team, primary) %>% arrange(team) %>% select(-team) %>% unlist %>% unname
) +
scale_fill_manual(
values = models %>% filter(!is.na(team)) %>% distinct(team, secondary) %>% arrange(team) %>% select(-team) %>% unlist %>% unname
) +
scale_x_continuous(limits = c(1, 32), breaks = c(1, 16, 32)) +
scale_y_continuous(limits = c(-10, 174),
breaks = c(1, 32, 64, 96, 137, 174)) +
facet_wrap(~metric, ncol = 1) +
theme_fivethirtyeight() +
theme(legend.position = "none") +
labs(x = "Mock Draft Position",
y = "Actual Draft Position",
title = "2018 NFL Draft 1st Round Projections",
subtitle = "Compared to Actual Draft Position",
caption = "Data and Graph by @benj_robinson"
)
```

`## Warning: Removed 446 rows containing non-finite values (stat_smooth).`

`## Warning: Removed 446 rows containing missing values (geom_point).`

`## Warning: Removed 446 rows containing missing values (geom_text_repel).`

That conclusion from the regression summaries is borne out in the graph above but what about when we focus in on the first round: where the majority of the mock draft data comes from. Suddenly, the Loess regression looks a lot stronger.

```
models %>%
filter(rank <= 32) %>%
ggplot(aes(
value,
pick,
color = factor(team, levels = team %>% unique %>% sort),
fill = factor(team, levels = team %>% unique %>% sort)
)) +
geom_point(size = 3) +
geom_text_repel(aes(label = name)) +
geom_abline(slope = 1,
intercept = 0,
size = .5) +
geom_smooth(aes(group = metric), method = 'lm', formula = 'y ~ x') +
scale_color_manual(
values = models %>% filter(!is.na(team)) %>% distinct(team, primary) %>% arrange(team) %>% select(-team) %>% unlist %>% unname
) +
scale_fill_manual(
values = models %>% filter(!is.na(team)) %>% distinct(team, secondary) %>% arrange(team) %>% select(-team) %>% unlist %>% unname
) +
scale_x_continuous(limits = c(1, 32), breaks = c(1, 16, 32)) +
scale_y_continuous(limits = c(-10, 174),
breaks = c(1, 32, 64, 96, 137, 174)) +
facet_wrap(~metric, ncol = 1) +
theme_fivethirtyeight() +
theme(legend.position = "none") +
labs(x = "Mock Draft Position",
y = "Actual Draft Position",
title = "2018 NFL Draft 1st Round Projections",
subtitle = "Compared to Actual Draft Position",
caption = "Data and Graph by @benj_robinson"
)
```

```
## Player Subset Metric
## 1 First Round Actual Draft Picks Linear Regression
## 2 First Round Actual Draft Picks Loess Regression
## Mock Draft Position Mean Squared Error Mock Draft Ranking Mean Squared Error
## 1 238.9097 303.7188
## 2 308.9240 384.0625
## Mock Draft Metric Correlation Mock Draft Ranking Correlation
## 1 0.6654195 0.6601037
## 2 0.6060257 0.6113017
```

```
## Player Subset Metric
## 1 First Round Mock Draft Rank Linear Regression
## 2 First Round Mock Draft Rank Loess Regression
## Mock Draft Position Mean Squared Error Mock Draft Ranking Mean Squared Error
## 1 510.6507 477.0312
## 2 97.4431 97.7500
## Mock Draft Metric Correlation Mock Draft Ranking Correlation
## 1 0.6424860 0.6878918
## 2 0.7558404 0.7460792
```

```
## Player Subset Metric
## 1 First Round Mock Draft Position Linear Regression
## 2 First Round Mock Draft Position Loess Regression
## Mock Draft Position Mean Squared Error Mock Draft Ranking Mean Squared Error
## 1 485.4605 458.9706
## 2 438.5455 391.7027
## Mock Draft Metric Correlation Mock Draft Ranking Correlation
## 1 0.6021054 0.6200909
## 2 0.6245101 0.6628927
```

While it seems that the Linear Regression is best for the ex post facto analysis of looking back at 2018 NFL Draft 1st round actuals, the Loess Regression has the lower of the two modeled estimates in terms of minimizing the Mean Squared Error of the predicted draft positions relative to actuals.

## The Conclusion

I don’t want to belabor the point too much, since that was a lot of math (but this is how we do exploratory analysis!), but the math based on the mock draft data does tell us quite a bit (in the 60% range) about what the actual draft looks like. However, that still means that there is 40% of the variation in the actual draft picks that is not explained by the mock drafts. To me, this means that we should not use data alone to predict the actual draft but that, combined with human intelligence, this data can do quite a bit to inform us about when a player might go in the draft.

Trades can and will occur a lot more than mock drafts likely predict (an analysis for another day) but I still think that in an uncertain world, the NFL Draft is not as unpredictable as we would like to think. In fact, I’m especially bullish about my projections for the 2019 NFL Draft given my twice as larger mock draft dataset to work from and the knowledge I’ve gathered from working on this post about which metrics to pay closest attention to.

Look for my own (very meta) data-informed 2019 NFL Mock Draft on April 25th, the day of the draft, as well as a healthy amount of Twitter content the first two days of the draft. I’m as excited as you all to see what happens!