The decision to pass or run on a given play in the NFL is a big one. Each has different risks, benefits, and specific situations where it is best. Right now, NFL offenses usually call passes when there are many yards until a first down and runs when there are fewer yards to gain. However, there are still many situations where the play-calling is balanced, such as 1st & 10. Using data from previous seasons, we can see which scenarios are best suited for passing or running.
Using First Down Rate
The simplest way to determine the success of passes or runs on different down and distance scenarios is to use first down percentage. Using play by play data from since 2018, the first down rates for the most common down and distance scenarios are shown below. The red line represents the first down rates for passes, while the blue line represents the first down rates for runs. The black dotted line is the average first down rate for all plays. Because this graph is supposed to show the impact of play-calling, all plays with a quarterback dropback (including pass attempts, sacks, and qb scrambles) are put in the pass category, even if the play was ultimately a run by the quarterback.
We can draw several conclusions from this graph. On first downs with less than 5 yards to go, there does not seem to be a large discrepancy between passing and running. These scenarios usually occur in goal-to-go situations, meaning that a first down results in a touchdown. On first downs with more than 5 yards to go, it is usually better to pass. A similar situation arises with other downs. On 2nd downs, it seems that running is the better option with 3 or fewer yards to go, but otherwise passing is more advantageous. The same occurs with 3rd downs and 4th downs. However, teams often are not always trying to gain a first down. Many times, NFL offenses will call run plays because they want to make future downs easier. This is the primary reason that NFL teams run the ball frequently on long distances like 1st and 10. They are not particularly aiming for a first down but rather a 2nd and short instead of 2nd and long. Therefore, a different metric to measure the success of runs and passes is expected points added and success rate.
Using Success Rate
Expected points added, or EPA for short, is the difference between the expected points scored on a drive before a play and after a play. The expected points is based on down, distance, yard line, and other factors. A more in-depth explanation of EPA can be found here.
A positive value of EPA indicates that a play was successful since the expected points of the drive increased. The opposite is true for a negative value of EPA. The distribution of Expected points added is centered around 0, meaning that there are approximately equal numbers of plays with positive EPA and negative EPA. Therefore, we can use a different statistic to measure the efficiency of a play: Success Rate. The Success Rate is the percentage of plays that result in a positive EPA. This gives a greater advantage to running the ball and is a better measure of efficiency than first down percentage as it accounts for the fact that a good play does not have to end in a first down. Below is a graph for the success rate of passes and runs based on different down and distance combinations.
At first glance, one can see that there is a lower difference between the efficiency of passing and running for most situations. For example, by looking at the first down graph we can see that passing and running have about the same benefit until there is 10 or more yards to go. Additionally, the success rate graph seems to have more confusing results, especially on third downs. The reason that the run line graph spikes greatly on third & longs is because there is not a lot of data for running in these situations, creating more randomness in the results.
Analyzing First Downs
Using the success rate statistics for first downs based on yards to go, I constructed a table showing the best option for each scenario, along with the confidence of each choice. The suggestion column shows whether is is better to pass or run based on the data, and the confidence column shows the confidence of the suggestion (based on the p-value of the Z-Test).
Based on the data, there is no large advantage for passing or running on 1st and less than 10. However, on 1st & 10’s, 1st & 15’s, and 1st & 20’s, it is far more advantageous to pass than run, with passes having a success rate at least 10% greater than rushing success rate. While teams pass very frequently on 1st & 15 and 1st & 20, the pass-run split is close to 50-50 on 1st & 10, the scenario that starts almost every single series. This is the biggest play-calling flaw in the NFL. Teams should be passing on 1st & 10 at a much higher rate, closer to 60-70% of plays instead of 50%. The graph below shows the tendencies of teams in the 2021 season on 1st & 10.
The graph above compares the difference in pass and run success rate of 1st & 10 to the pass rate on 1st & 10 for NFL teams in 2021. Teams in the bottom right corner are losing the most from running a lot on 1st & 10. The Titans, Cardinals, and Falcons are three teams that are much more successful when they pass on 1st & 10, but still run more often anyways. The Cardinals and Titans already have potent offenses, so calling more pass plays on 1st & 10 could boost their offenses even further. In contrast, teams like the Bills and Chargers are mostly passing on 1st & 10, gaining efficiency by avoiding the run in these situations. In general, though, teams should start to call more pass plays on 1st & 10.
Second Downs
The success rates and suggestions for second down scenarios are shown in the chart below.
On 2nd downs with less than 4 yards to go, it is usually slightly better to pass, although the difference between passing and running is not major. However, when there are at least 4 yards to go on 2nd downs, the optimal decision is to pass. While most NFL teams understand that passing is far more efficient than running on 2nd & 11+, they still do not call enough passes when the yards to go is between 4 and 10. Similar to how teams are losing efficiency by running on 1st & 10, teams would be more efficient is they passed more often on 2nd & 4 to 2nd & 10. The tendencies of teams in these situations is shown below.
Just like the 1st & 10 tendencies plot, the teams in the bottom right quadrant have the most flawed play-calling on 2nd & 4 to 2nd & 10. The Titans (again) and Saints should be passing a lot more in these situations, even if they do have talented running backs in Derrick Henry and Alvin Kamara. The Titans’ rushing success rate on 2nd & 4 to 2nd & 10 is just 27%, 2nd worst in the NFL. Meanwhile, they have a top 7 passing success rate of 57%, but they only pass 54% of the time. On the other hand, the Buccaneers, Falcons, and Chiefs have realized that passing is far more beneficial than running in these scenarios, as each team has called a pass play on 80% or more 2nd & 4’s to 2nd & 10’s. Only 5 teams (Chargers, Seahawks, Steelers, Cowboys, 49ers) are actually more successful running than passing in these instances.
Third and Fourth Downs
In the NFL, 3rd down plays are usually dominated by passes. Teams realize that they need to get a first down to avoid punting on 4th down, so they try to get all the yardage possible. Whenever there are 5+ yards to go on a 3rd down, the best choice is to call a pass. The confidence of the 3rd down choices towards the end are low with high standard errors because of the lack of run plays called in 3rd & Long situations. That’s why there are some cases in which the run success rate is actually greater than the pass success rate (like 3rd & 13, 3rd & 16, and 3rd & 19-20). However, these are just a result of a lower sample size and passing is still the better option. Unlike 1st & 10 and 2nd & Longs, NFL teams usually make the right decision to pass on 3rd downs.
NFL teams also usually make the correct decisions on 4th downs. The only time it is really advantageous to run is on 4th & 1, with passing being the better option otherwise. There are very high standard errors because there is not a lot of data for 4th downs, since teams usually do not attempt a play on 4th down. The run suggestion for 4th & 3 is also likely a result of low sample size since all the other scenarios have a pass suggestion.
Team Tendencies
Using the suggestions from each of the 1st, 2nd, 3rd, and 4th down scenarios, we can see which teams are calling passes and runs at the right times, based on the previous tables. To evaluate this, I created a metric called Match Rate. The match rate is the percentage of plays that the team called what was suggested (called a pass when pass success rate is higher; a run when run success rate is higher). The plays were weighted by the confidence of the suggestion, where the weight of a high confidence suggestion was 3 times greater than the weight of a low confidence suggestion (and 1.5 times greater than the weight of a medium confidence suggestion). Only the scenarios which were included in the previous tables were counted for the match rate. The Chiefs and Buccaneers currently have the greatest match rate, meaning they are calling the right plays at the right times.
The Chiefs and Buccaneers both call lots of passes, especially in down + distance scenarios where pass success rate is greater than run success rate. The Chiefs have a pass rate over 60% on 1st & 10, which increases their Match Rate significantly, and both the Chiefs and Bucs pass often on 2nd & 4 to 2nd & 10, an area where teams usually run too often. Meanwhile, on plays where the suggestion is to run, the chiefs run most of the time (they have a top 5 run rate in both low and medium confidence run suggestion plays). The Buccaneers similarly run often when they should, especially in high confidence run suggestion plays.
Overall, NFL teams call too many run plays and not enough pass plays. The first scenario where this is especially apparent is the most common one: 1st & 10. Even though the passing success rate is almost 15% greater than the rushing success rate on 1st & 10, NFL teams call passes only slightly over 50% of the time. Another scenario where teams run far too often is on 2nd downs with between 4 and 10 yards to go. Of course, the passing and running tendencies of a team depend heavily on the team’s relative strength of passing vs running, so the general suggestions shown in the tables do not apply to every team. If a team is far more efficient running the ball, they should run more often than suggested. Additionally, another reason teams run more often than expected is because running has less of a risk: pass plays have the chance to result in a turnover or sack, while run plays are a safer option due to a lower chance of turnover. Unfortunately, this is not accounted for by success rate since it is binary. However, in general, NFL teams should opt to pass more often, especially on 1st & 10, 2nd & Medium, and 2nd & Long.
Best Ball Stats 