Like any other sport, the game of football is at its core all about extracting more value from each individual play than the opponent. There are plenty of ways that this can and has been done by NFL teams. As time passed, NFL teams began to throw the ball more often as they realized the untapped value of the passing game, they began to run from spread formations as they realized the value of spacing on the field, and they began to blitz more as they realized the value of pressuring the quarterback.
One thing NFL teams don’t seem to have a handle on yet is the value of the downfield pass. Even as the percentage of passing plays run by teams has consistently crept up over the past 50 years, the rate at which teams throw the ball downfield has remained low since the mid 80s. We haven’t ever seen an NFL in which pass rates were high and teams consistently attacked down the field with deep passes. It’s probably a case of coaches trying to hedge their bets as they transition more aggressive pass-heavy offenses. Throwing the ball a majority of the time suddenly becomes much less risky when you’re only passing short distances past the line of scrimmage.
NFL coaches, or for that matter anyone working in a position of power within an NFL franchise, take an especially conservative approach to their profession. There’s no obvious reason for them to take risks when doing so puts them in danger of losing their jobs. No one notices when you’re wrong in the same way everyone else is, but as soon as soon as that step out on a limb turns into a misstep there’s a quick target on your back. The only time NFL coaches take risks is when the majority of them essentially agree to do so, thus eliminating much of the risk. This tendency to preserve job security over possible success leads to plenty of untapped value left for the taking.
There isn’t much logic that leads to the decision of throwing the ball short an overwhelming majority of the time, but I suspect NFL decision makers are assuming that nearly guaranteed small gains are worth more in the long run than the possibility of far less consistent large gains. Whether or not this is actually true depends on how dependable the result of each respective gain is.
Looking at the numbers bears out that this conservative behavior in search of predictable results comes at a very large cost. Teams are sacrificing huge amounts of offensive value every season simply because they hold the incorrect view that the improvement in going from a 1st and 10 to a 2nd and 4 is worth sacrificing the possibility of another 1st and 10 much further down the field. It’s a classic case of chasing consistency at the expense of more volatile, long term value.
Before we dive into the numbers that show where this decision making is leading, let’s quickly explain the methodology.
Using an Expected Points (explained in depth here) model (Markov Drive Analysis from Drive-By Football) and 2016 NFL play-by-play data from NFLSavant.com it takes just some simple formatting in Excel to find data points for every passing play of the 2016 season.
Passing plays from the second half of the 2nd and 4th quarters were eliminated in order keep the constraints of game time from affecting the results.
We can find Expected Points Added (EPA), Completion %, Touchdown %, and Interception % data points, which will be the basis of our analysis.
Now here we go.
Surface-level examination is a bit misleading here. Short passes have a far higher Completion Percentage and far lower Interception Rate, although they also have a far lower Touchdown Rate. A two-to-one advantage for short passing ignores that Touchdown Percentage is by far the most important measure among the three. One touchdown on average holds far more value to an offense than even numerous completions do. A single touchdown also brings a team far more value than an interception takes away from it on average.
The most important metric in the table above is EPA/Attempt. On a single play, a deep pass is expected to yield nearly twice the value of a short pass. Over the course of the season short passes have yielded offenses only a little over twice as much value as deep passes, despite being attempted 4.5 times more often.
Let’s look at a distribution of EPA for short and deep passes across all of 2016 to get a better picture for the disparity in value.
Ignoring the huge spike in the middle of either graph, which we’ll get to in a minute, the distribution is about what you would intuitively expect. Deep passes lead to bigger gains and also larger losses, while short passes lead to more consistent small gains. The part of the distribution graphs that seems out of place is the small disparity between deep passes that lead to a loss in value and short passes that lead to a loss in value.
In order to get a better picture of the distributions we can look only at completed passes. This will effectively remove the huge spike in the center of the graphs since every incomplete pass, regardless of the down, distance, and field position scenario save for 4th downs, falls between and 0.99 and 0.01 expected points loss. This will give a much clearer picture of the upside/downside relationship between short and deep passes.
It’s not surprising that completed passes farther downfield lead to more gained value than completed passes closer to the line of scrimmage. After all, gaining more yards is obviously better.
However, by looking at only completed passes, which in the play-by-play data we’re working with includes interceptions and thus preserves the big downside aspect of deep passes, we can see that deep passes provide disproportionately greater upside to short passes without sacrificing the same disproportionate downside. In fact, a plurality of short passes even leads to a loss in value. Greater than one third of the time that a short pass is completed the offense ends up in a worse position after the completion than they were before the play.
As explained briefly in the intro, the rationale for this strategy is that even a small gain is better than an incomplete pass and no gain, which is an outcome that occurs greater than 50% of the time on deep pass attempts. In other words, a small loss is always better than a slightly bigger loss. What is being overlooked here is that the difference between losses is limited to a small downside, less than 1 Expected Point. On the other hand, the difference between gains has a theoretical limit of just under 14 Expected Points. Placing a bet with a small defined loss and large possible gain is almost never a bad decision.
In a vacuum, the no-brainer decision would be to throw deep on every single offensive play, but of course there are outside conditions in a football game that aren’t present in a vacuum. For one example, on 3rd and short a team stands more to gain from maximizing its odds at a first down rather than chasing Expected Points upside. Let’s take a look at how the outcomes of deep and short passes differ when you separate them by down.
What immediately stands out about 1st Down is the disparity in EPA value between short and deep passes. As far as Completion %, Interception %, and Touchdown % 1st Downs seem to be essentially reflective of all pass attempts, so what happened to cause such a huge difference in EPA and EPA/Attempt?
An overall decrease in EPA/Attempt numbers isn’t the surprising part, as you would expect short passes to suffer due to being farther from the first down line and deep passes to suffer as defenses are more guarded against downfield attempts. What is the surprising part is how much value short passes lose compared to deep passes. At 0.002 EPA/Attempt, attempting a short pass on first down is essentially sacrificing a down. It’s an indication that NFL teams are so risk averse that they would rather sacrifice any hope at creating positive value in order to ensure they don’t suffer a loss in value. Despite the fact that deep passes offer 95 TIMES the value per attempt as short passes do, they are attempted less than one fourth as often. Put another way, 95% of the value of first down passes came from deep attempts yet they only account for 18.8% of the total first down pass attempts.
Throwing short of the sticks on first down is quite possibly the worst strategic decision out there for NFL teams to make, and yet it was made over 4,000 times last season alone.
There’s not much to address with 2nd Down because its results are so reflective of overall pass attempts, so let’s move on to see if there’s anything to glean from 3rd Down numbers.
Since the outcome and value of a play on 3rd Down is so inherently tied to the yardage needed to gain a first down, it’s necessary to analyze the value of 3rd Down passes with respect to Yards to Go. Doing so yields results that are surprising to the say the least, but not unexplainable.
Deep passes enjoying a monopoly on value for 3rd Down attempts is striking. In a situation where a first down conversion is essentially the only element of the outcome that matters, short passes with a much higher Completion % should enjoy a surplus of value. Relative to overall EPA/Attempt numbers short passes on 3rd Down do gain value relative to deep passes, just not enough to become more valuable.
As you’ll notice above, throwing deep on 3rd and short (1-3 YTG) is the most valuable play examined so far. I suspect it is one of the most valuable offensive plays in the NFL altogether. This likely stems from defenses over-guarding against short passes in order to prevent 1st downs, in effect leaving their deep coverage more vulnerable to the slim number of deep passes attempted by offenses on 3rd and short.
There also seems to be a sort of alternation in the numbers above. Deep passes are clearly more valuable with 1-3 YTG and 7-9 YTG, while the difference in value between short and deep passes is more minute with 4-6 YTG and 7-9 YTG. The alternating structure has a possible explanation as well, although I’m not sure it’s based as much in observation as it is in speculation. Sometimes educated speculation is necessary, though, in a game where decision making leans so heavily on the decision making of the opponent. We already addressed the likely reason for why deep passes hold so much value per play with 1-3 YTG. I suspect short passes recoup value with 4-6 YTG because defenses are less focused on specifically stopping short passes and thus they become more valuable as completions are more likely. With 7-9 YTG deep passes once again gain more relative value because offenses are farther away from the line to gain and completed short passes are no longer nearly guaranteed to end up in first downs. Lastly, deep passes once again lose value with 10+ YTG because defenses no longer need to guard against short passes as a serious threat and can focus all their attention on preventing deep completions. That’s not a wholly satisfying explanation, but it is the one I have settled on being at least somewhat likely.
Now that we have a handle on how downs affect the value of deep and short passes, let’s move on to address how field position affects EPA/Attempt numbers.
Once again, about what you’d expect based on what we saw earlier. For reference, in the chart above the YL numbers signify the distance from the opponent’s end zone. Deep passes consistently hold more value than short passes except near either end zone. Backed up in your own end short passes hold more value because the cost of an turnover is magnified and thus playing it safe with short passes is the better option.
Inside the opponent’s 30 yard line, I suspect the difference is because essentially the only deep pass to be made is into the end zone, which is always more strongly defended than any other 10 yard stretch on the field. While this seems like a logical explanation, I certainly wouldn’t have expected short passes to gain so much value on deep passes near the opponent’s end zone.
Now that we have information on the overall value of deep and short passes and how down, distance, and field position affects that value, what does any of this mean for actual strategy in the NFL?
The first obvious conclusion is that teams should be throwing deep much more often than they currently are. Despite holding roughly twice as much value per play, deep passes are attempted less than a quarter as often as short passes. If you limit the sample to 1st Down only then deep passes hold 95 times the value per play of short passes according to EPA/Attempt, yet are still attempted less than a quarter as often. To put it simply: that’s absurd.
There’s certainly an element of game theory to the decision. It’s not as simple as throwing deep every time because deep passes hold more value on a per play basis. Throwing deep more often will lead to defenses adjusting to the tendency and playing farther off the ball to guard against deep passes. That, in turn, will open up underneath passing lanes and the value pendulum will begin to swing back towards short passing.
There is some ideal split in the proportion of deep to short passing where the value of each is essentially equal, that way defenses have to respect both short and deep passes and the value of passing plays in general is maximized. There’s no way to tell what that ideal split is until it’s naturally found, but it’s certainly much different than the current split. It speaks to how NFL offenses are so grossly misunderstanding the value of deep passing that you can count on two hands the situations in which throwing the ball short is currently the better option. Yet offensive strategy is being approached as though deep passing is a more often used trick play and short passes are the bedrock on which an offense is built.
Passing depth of target is certainly more complicated than just “deep” and “short” passes, so a more descriptive way of separating (such as Depth of Target) would likely help parse the issue even more. Unfortunately the play-by-play data used only separates the passes into deep and short. I’m not sure there’s a hard barrier between the two but it seems to be about 15 yards.
One other note of value would be that the risk of sacks is not baked into the analysis above, since a sack prevents a pass attempt from occurring. Sack rates are necessarily higher for plays with deeper drop backs which often coincide with deeper passes. Sacks occur on about 7.8% of designed deep dropbacks, 6.8% of designed intermediate dropbacks, and 5.2% of designed short dropbacks. This adds another element of risk to deep passes in addition to higher incompletion rates and higher interception rates when compared with short passes. Deep passes still almost certainly carry enough upside to offset the limited added risk, but it is something to note as a flaw in the data.
It’s unfortunate that there’s no hard and fast way to determine exactly how much more often teams should be throwing the ball deep, at least with the limited scope of the game that can be seen from this data.
In a way, though, that’s what makes football great.