Introducing Adjusted Quarterback Efficiency

Using advanced stats and charting to adjust quarterback EPA for luck and surroundings

Using advanced stats, like EPA per play, to assess quarterbacks has its flaws. EPA measures the value added by the entire offense on a play, yet in doing so assigns all the value to the quarterback.

Yes, EPA is a team stat, but so are all the other advanced and traditional stats. The quarterback, in and of himself, isn’t fully responsible for passing yards or touchdowns. Even plays seemingly in their control, like interceptions, are influenced by the performance of teammates and players on the other side of the field. PFF grading looks to isolate quarterback play, but even grading has it’s issues with assigning proper blame to quarterbacks for sacks, and a myopic focus on the throws made instead of the whole process. EPA solves the calibration of value missing in other measures, but is more results based for plays of luck and surrounding context, like tipped or dropped interceptions or a poor offensive line.

We can’t strip out all the context and surroundings to focus in exactly on the quarterback, but charting data can help us get part of the way. In this analysis, I’m using the nflfastR data for EPA, in conjunction with FTNData’s play-level charting data. Combining the two, we can adjust for receiver and defender drops, fumble recovery luck, excess value from penalties and strength-of-schedule adjustments.

Next week, I’ll utilize some measures of receiver strength, perhaps the player-tracking receiver metrics from ESPN, to adjust quarterback efficiency for his pass catchers. But that’s a more complicated process, and I want to take a little more time to make sure I get right.

For now, let’s look at the non-receiver quality adjustments of quarterbacks, illustrated below comparing their non-adjusted EPA per play to after adjustments.

The two biggest movers in either direction are Russell Wilson (down) and Matthew Stafford (up). Josh Allen sees a relatively big decline near the top, widening the gap with Jalen Hurts. Jimmy Garoppolo, everyone’s favorite example for the flaws of EPA, drops down fourth in EPA per play unadjusted to seventh with adjustments, in a virtual tie with Geno Smith right below.

Turnover (dropped, tipped INTs, fumble luck) and drops adjustments (drops) have the biggest impacts, with sack adjustments - based on my survival curve analysis - next, then strength of schedule movement based on the same methodology as my power rankings.

The nflfastR data has expected air and YAC EPA numbers that can be applied to drops to assumed value if caught, interception EP loss can be reversed for plays not charted and turnover worthy, and fumbles can have their value adjusted based on recovery rates, giving a more severe adjustment for those fumbled by the quarterback and recovered by a different offensive player.

Breaking out the adjustments categorically gives more insight into the measures cause the movement. The table below lists the values in total EPA, not a per-play basis. Its easier to conceptualize the numbers as whole points gained or lost, rather than the fractions left after dividing for plays.

Dig through the numbers and see exactly what category of adjustment is moving your favorite quarterback up or down the rankings.

Interesting numbers to keep in mind when thinking about different adjustments:

• Only 35% of interception-worthy throws end up interceptions.

• 57.5% of fumbles are lost if not recovered by the quarterback himself.

• Roughly 3.5% of all passes are dropped, the highest rate is for short passes in the middle of the field.

• Defensive pass interference is called on only 1% of passes, but are worth around 5% of the total expected points on completed passes.

I’ll be back with an update to this analysis next week, with some tweak and additions that will change around the numbers. Until then, leave in the comments any suggestions for a better process or surprises you see in the results.