World of NFL Player Value Metrics Has Gone Crazy
Nobody is perfect in the land of NFL player valuations.
Welcome back, everyone!
If you are a new subscriber to the Cap & Trade Newsletter, welcome! This newsletter covers the NFL salary cap and news across the league with additional coverage of the Houston Texans. I have been covering the Houston Texans since 2012 and 2017 as credentialed media, and have been working at OverTheCap.com for the past 7 years. I hope you find this newsletter informative and digestible. I do subscribe to many paywall-type websites for data sourcing and aggregation, but always looking for more sources. While this newsletter remains free, you can activate a paid subscription to assist in making this newsletter even better:
What a week of NFL Wild Card action this weekend, including the complete domination by the Houston Texans with their victory over the Cleveland Browns.
The game plan on both sides of the line of scrimmage was pure magic and quite frankly a joy to watch. The team has a huge challenge, and opportunity, this coming Saturday against the #1 seed, the Baltimore Ravens in Baltimore.
Pivoting off the Houston Texans, I wanted to share some thoughts on a major topic on X (formerly Twitter) verse specific to PFF player grades.
ProFootball Focus (PFF) got the fireworks started with this tweet showcasing the Top 4 graded quarterbacks from the Wild Card slate of games.
One notable player was missing from the graphic…C.J Stroud. On Saturday, Stroud put together a fantastic performance finishing the game with a 16/21/3/0 line with a near-perfect passer rating of 157.2. This statistical line was nearly identical to that of Jordan Love with Green Bay.
Yet Love was graded higher than Stroud. A social media firestorm would ensue with grenades from both sides of the fence.
Fast forward ~8 hours with former Houston Texans defensive end and future Hall of Famer J.J. Watt added his thoughts on the PFF post.
This post set off the PFF detractors even further with the lone Sam Monson standing like Jon Snow trying to defend the PFF shield. In PFF’s defense, the usage of “algorithm” by Watt was inaccurate as PFF utilizes a team of human game graders.
Pulling from the fantastic book by
on the PFF origin story, Sam Monson has a history of wearing personal protective gear in defense of the PFF brand. If you haven’t read (or my case listened to) Matthew’s book, and you are interested in a deeper understanding of PFF and how they came to be; you can purchase the book on Amazon here. I truly enjoyed the book myself.Fast forward to today to the wildly popular Pat McAfee show on ESPN/YouTube where J.J. Watt continues his opinion on PFF player grades.
This is the point where things start to go sideways for all involved. Comparing PFF player grades to NFL coaching grades is not an apples-to-apples comparison. As soon as I read this tweet, I knew where this was headed.
And Geoff Schwartz, former offensive lineman, did not disappoint.
This is a solid counter argument to J.J. Watt’s initial point about coaching grades. Coaches could potentially exhibit bias in their assessment of their own players, and the game result could further exaborate that bias…even citing their own job as part of the reasoning.
So what about PFF Grades? I’m not going to tell you a definitive yes or no if you should trust PFF player grades. That is up for you to decide. What I can do to explain my thoughts on the grades and how I use them in my work.
Do I trust the player grades generated by PFF? Absolutely…on a season long scale.
Do I trust the player grades generated by PFF on a week to week basis? Not always. Sometimes, not often, I have a disagreement with their generated player grade.
In the past, I leaned heavily on PFF player grades to assist me in determing a player’s performance. I did not make time to watch film and make the assessment for myself. Nor am I an expert to make a definitive assessment on that type of scale.
For the 2023 season I put fourth the effort to watch film on a regular basis with the limited spare time I have at my disposal. Watching the film has assisted me in generating a more confident assessment on the players I am watching.
Do I ignore PFF if I disagree with their generated grade in a specific week? Not specifically, but I will make a note of it.
At the end of the day I use what I see on film in conjuction with the PFF grades. More often than not, we (PFF and I) agree with the player performance.
The player grade becomes another data point in the overall assessment of the player. I believe that is the the most imporant take away for myself.
I truly believe what Neil Hornsby set out to do, and that was “to get it right” with the grading system. If you read (or listen to) the book and do some truthful research you will find that PFF’s grading system has merit with strong correlation to future performance.
Other Player Value Metric Sources
One data source I utilize quite a bit is Sports Info Solutions (SIS), specifically their Total Points metric.
Simply defined as “Total Points takes nearly everything that SIS measures about a play and uses it to evaluate each player on a scale that allows you to compare them more easily.”
For an in-depth explanation I will refer you here.
As the drama with the PFF Player Grades continued on my X timeline, I wondered how SIS’s Total Points metric compared against PFF Player Grades.
This is a very small sample size, so keep that in mind. I took the quarterbacks with the most “aimed passes” pulling the Top 30 quarterbacks and placed their respective offensive player grade and their respective total point number into a scatter plot.
Full interactive chart here: Data Viz
Based on this plot, there is a strong correlation between the PFF player grades and SIS Total Point value. I am surprised at the distance from the trend line given the high correlation. Something I would need to further investigate.
The correlation between the two value metrics gives me confidence that I could trust using either metric in reference to my work.
You can find the free data from SIS at their Data Hub here.
Closing
Who’s right and who’s wrong? That’s up for you to decide. If you do not like PFF player grades, I respect that. My only request is to avoid trashing the company that has singlehandely transformed how we look at the game of football.
But if you feel the need to trash the product, I am not here to stop you.
I utilize a number of data sources in addition to player film to reach my personal conclusions; including PFF, SIS, NGS, and NFLFastR public data.
I encourage you to find reputable sources for data AND reputable people to follow who understand the game and can break it down professionally. Luckily there are an abundance of qualified individuals and organizations that offer quality insight and game break downs (some free) on social media and their respective websites. Search those folks and companies out, do your research, and enjoy.
P.S.
I hope you enjoyed last week’s video post. I believe there is a place for video within the Cap & Trade Newsletter, and you can expect more video posts this year.
And with that…#GoTexans!
-TC
Hey, really appreciate the mention! Great write up