Steals Don’t Mean Squat

If there’s one thing we know about defensive statistics in the NBA, it’s that steals don’t mean squat. James Harden was second in the league in steals per game last year, and Andre Drummond was eighth. This was not merely an illusion created by the two playing lots of minutes, as both ranked within the top 20 in the league in steals per 36 minutes. In 2016-17, Manu Ginobili and T.J. McConnell were first and second in the league, respectively, in steals per 36 minutes. Steph Curry and Nerlens Noel were both in the top seven in the league in 2015-16, while the same two players along with Pablo Prigioni all ranked in the top eight in the league the previous season. Andray Blatche was eighth in the league in steals per 36 in 2012-13. The illusion extends back as far as steals go in the statistical record.

The Real Reasons Why Steals Don’t Mean Squat

          These examples are merely anecdotal evidence, though; what really makes steals unreliable indicators of defensive performance are the many different chains of events which can lead to a player being credited with a steal in the box score. Many of those sequences involve plays made by teammates of the player who gets credit for the steal. Any observer can recognize these plays when they happen: tipped passes, saves on balls headed out of bounds, instances in which an on-ball defender pokes the ball away from the dribbler and another defender grabs the loose ball, traps, double teams, errant passes caused by pressure on the ballhandler, etc. Sometimes a steal is the result of a phenomenal play by one defender, but oftentimes a steal is the result of one player disrupting the offense and another player recovering the ball. Steals create an immediate problem of attribution; the player who gets credit for the steal is not always the player who truly created the turnover.

          Beyond the individual problem, steals are a problem at the team level. Steals don’t mean squat for the team, and here’s why: they don’t tell us very much about how well a team defends. Using data from the 1976-77 through 2009-10 seasons (900 team-seasons), we find a weak connection between a team’s steals and the team’s points allowed. More specifically, the correlation coefficient between the two variables is .543. Opponent field goal percentage, by comparison, explains .715 of variance in points allowed. This is not rocket science, or even difficult math. Steals made up only 8.3% of possessions in the data set, so of course they don’t do much to explain how well the team performs. Opponents shoot at least once on a vast majority of possessions, so opponent field goal percentage naturally correlates much more strongly with team defense.

          If steals don’t tell us about how good a team’s defense is, why would anyone think that steals can tell us how good an individual player is on defense? The question leads us back to the problem of attribution specificity, which we already know is a dead end. At the individual level and the team level, steals don’t mean squat.

How Do We Measure Non-Shooting Defense?

          In the first part of my Matchup-Based Defense model, we analyzed the effect of players’ defense on the scoring performance of the player he is assigned to guard. What we have not directly addressed is the impact made by players on possessions which do not include a field goal attempt or free throw attempt. Clearly, possessions that result in a turnover by the offense are valuable in and of themselves. Is it possible to determine how much each player contributes to opponent turnovers, or can we only evaluate them accurately at the team level?

          Like any good analyst, I’ll answer with a confident, “Hmm, let’s see.” There are two categories which encompass a vast majority of turnovers:

1. the offense loses or mishandles the ball (this includes swipes, bad passes, travels, double dribbles, the ball going out of bounds, and the player going out of bounds)
2. offensive fouls

Turnovers which do not fit into these categories do happen, but they are rare (backcourt violations, three seconds in the lane, etc.). Some turnovers are solo defensive plays, while others require coordinated efforts by multiple defenders. If we are to have any chance of estimating how much each player contributes to the turnovers his team forces, I propose that we begin with the known and move cautiously toward the unknown.

The Known

          The last few seasons have been like the Renaissance for acolytes of “hustle plays,” with the NBA tracking charges drawn for the first time in league history. Though we do not know with certainty what causes offensive fouls that are not charges, we can safely give defenders full credit for the charges they draw (adjusted for pace).

          We can also use a smidgen of deductive reasoning to convert steals into a known quantity that is useful. Among the hustle plays that the league has tracked and made publicly available in a progressive rollout since 2013-14 are loose ball recoveries, and the data is now even disaggregated into offensive and defensive loose ball recoveries. All defensive loose ball recoveries are steals, but not all steals are loose ball recoveries. Stop, and reread that with a bit of explanation, because this is crucial: all loose ball recoveries are steals (because any play classified as a loose ball that the defense retrieves is marked as a steal), but not every steal is a loose ball recovery (some steals are intercepted passes, others are occasions where a defender picks the ball handler’s pocket).

          What does that matter? Steals don’t mean squat, right? Well, let’s see … If we know that some steals are loose ball recoveries, but all loose ball recoveries are steals, then we have a good working estimate for how many steals actually multiplayer efforts which are credited to one player in spite of resulting from plays made by his teammates. A loose ball steal needs to be dislodged from the offense in order to become a loose ball to be recovered. If the player who disrupts the offense is the same one who recovers the ball, the play is never coded as a loose ball, but only as a steal.

          The steals that don’t mean squat are the steals that are credited to a player who doesn’t deserve the credit. If we subtract a player’s loose ball recoveries from his steals, we will effectively remove the steals that don’t reflect the player’s own ability. The steals which are not also loose ball recoveries are plays where the steal was entirely the work of one player. The steals that are loose ball recoveries (the ones we’ve removed) are plays that the player does not deserve all of the credit for. If our goal is to quantify individual contribution to the turnovers forced by a team, this is exactly what we want: a play that reflects the player’s own impact, rather than wrongly attributing credit based on the interaction of multiple players. I call these plays “solo steals.”

          We have two known factors: charges drawn and solo steals. There are a number of turnovers which fall outside of these two categories, and it would be nice if it were possible to determine how much each player contributes to them. Is it possible? To answer that question, we will have to move into the unknown.

The Unknown

          When we adjust both figures for pace, 4,300 of the offensive fouls which occurred in the 2018-19 season were not charges. If they were distributed evenly across teams (unlikely, but an acceptable supposition for the sake of argument), we would have 143 offensive fouls per team per season, or 1.75 per team per game. These fouls are disproportionately committed by big men. While I’m not aware of any source that has classified these fouls by type, the most commonly occurring types are both likely to involve bigger players: illegal screens and fouls occurring in the fracas under the basket.

          The problem with giving defensive players credit for these plays should be readily apparent: the matchup data only tells us whose designated opponent committed the foul, not who drew the foul. Illegal screens, though usually committed by a big man, are not committed against the opposing big men. The screener doesn’t screen his own defender, he screens the ballhandler’s defender. Since the publicly available data does not identify specifically which pair of offensive players and defenders is involved in every screen, it would be inaccurate and irresponsible to assign credit to the defender for offensive fouls committed by his assigned matchup. As such, I have elected to leave these plays out of my accounting.

          If the unknown offensive foul turnovers are not capable of being discriminated on the player level, we can only approach the unknown type of non-offensive foul turnovers with caution.

“Other” Turnovers

Let’s first carefully define which plays we are considering. We are presently focusing on plays that result in turnovers which are not solo steals (steals without a loose ball recovery) or offensive fouls. I can find no convincing argument that these plays do not reflect the defender’s talent. The turnovers which result from a bad pass are usually caused, at least in part, by defensive pressure. Steals that transpire after a deflection or loose ball scramble may not be credited to the defender who caused the initial disruption, but they are always debited against the offensive player who lost control of the ball.

This means that if we want to know who did the most toward causing the turnover, we should look at who was defending the player that lost the ball. The matchup data tells us who defended a certain player for the majority of the possession, meaning that it is not certain that this was the player who forced the turnover. In my judgment, however, it is sufficiently likely that the player who spent the most time during the possession guarding the ball handler is at least partly responsible for the turnover. In what follows, I will refer to these events as “other” turnovers for sake of shortening the verbiage wherever possible.

          If we accept the general reliability of opponent turnovers which are not solo steals or offensive fouls for evaluating defenders, can we simply take the “other turnovers” committed by a defender’s opponents as a direct reflection of the player’s aptitude for causing turnovers? I would caution against such an approach, due to the relative likelihood of each opponent committing a turnover. In order to deal with this variance, which should be expected based on what we know about the ball handler’s play style and offensive tendencies, I have employed the following workaround:

• Find player’s OOTO (opponent “other” turnovers) divided by expected OOTO
  • This answers the question, “Did this player’s opponent turn the ball over more often or less often than normal for him when he was guarded by this defender?”
• Divide the player’s OOTO  by the team’s total OOTO
  • This answers the question, “What percentage of the “other” turnovers forced by this team did this player cause?”
• Multiply the quotient of the first bullet point by the quotient of the second bullet point
  • This answers the question, “How many of the turnovers that the team forced did this player cause, relative to how many he should have forced, based on the players he was guarding?”
• Divide the resulting factor by the sum of these factors for all the players on a team (team total is usually close to, but slightly above or below, 1.0)
  • This answers the question, “How did this player’s contribution to forcing turnovers compare with the contributions of his teammates?
• Multiply the result by the total available pool (the team’s OOTO)
  • This answers the question, “How much credit does this player deserve for forcing turnovers?

Estimating the Turnovers Forced by Each Player

          We now have three tools for measuring a player’s performance on possessions that do not involve a shot: solo steals, charges drawn, and “other” turnovers. I have termed the result “Non-Shooting Defense” as a colloquialism in order to keep to the fore the point that this model inspects only plays that do not include a shot. The results appear in the table below, with low-minute “non-qualifying” players removed to allow for more profitable sorting of the table.

2018-19 Non-Shooting Defense

          How does this data contribute to the overall picture of individual defensive evaluation which we began last week in Matchup-Based Defense? In order to make the results accessible, I have juxtaposed Non-Shooting Defense (both total and per 100 possessions) with Shooting Defense + Non-Shooting Defense in the table below. The columns on the right express the player’s defensive performance on both plays that include a shot and plays that do not. The columns to their left express only the player’s Non-Shooting Defense. Comparing them should help to clarify the relationship between the two.

Two notes about the table:

• The table includes only players with 50 or more total points saved in order to allow for more effective sorting of the “Per 100” columns
• The Shooting Defense calculations from my previous work are multiplied by 0.744 before being added to Non-Shooting Defense in the table below. In a previous study, I found that defensive rebounders deserve 25.6% of the credit for missed field goals. This leaves 74.4% of the credit for the defender who forced the miss.

Shooting Defense Plus Non-Shooting Defense

          At this point I’d like to make a note on magnitude. As will be evident from the data, Non-Shooting Defense deals with far fewer plays than does Shooting Defense. 451 players last season fell between 4.5 and 1.0 Non-Shooting stops per 100 possessions, meaning that even the difference between an elite defender and an ineffective defender is only three to four plays over a full game.

          The upshot here is that forcing turnovers has a much smaller impact on defensive value than forcing misses. Just like steals don’t mean squat in the box score, forcing turnovers has much less explanatory power than getting stops on the many, many more plays which include a shot. If we only had Shooting Defense, we would still know a lot about how much individual defenders contribute to their teams. Adding Non-Shooting Defense provides us with a slightly more nuanced view of overall individual contribution by crediting players for the possessions which they disrupt before the offense can get a shot up.   

Evidence of the Hypothesis

As a concluding consideration, I’d like to deal with worries about falsely assigning credit for “other” turnovers to the wrong defender. How can we be sure that the player we are crediting with forcing the turnover is, in fact, responsible? One method I used to investigate this question was inspecting game logs from players who forced a lot of “other” turnovers.

The San Antonio Spurs had the fewest steals in the league last year, and only one high-load defender: Derrick White. In games that the Spurs notched 10+ steals, Derrick White’s opponents coughed up turnovers left and right. White did not get credit for most of the steals, but games where his teammates recorded a lot of steals were uniformly games in which White’s matchup turned the ball over. (Note: I also included the Spurs highest-steal playoff game, despite San Antonio getting fewer than 10 steals)

Did Derrick White Force Turnovers in Games Where the Spurs Got 10+ Steals?

I also pulled game logs for two other players who a) play a lot of minutes, b) are high-load defenders, c) play for teams that are below average in steals, and d) do not have teammates who are high-load defenders. These players should allow us to test our hypothesis, since there are fewer confounding variables. Focusing on players without high-load teammates also helps minimize collinearity, since having teammates who are strong defenders could inflate the value of a specific individual defender due to the team’s scheme or game plan rather than the player’s own performance.

Below are the game logs for Patrick Beverley and Frank Ntilikina, including regular season and postseason games in which their teams accumulated 10+ steals. Just as we saw with Derrick White, we find evidence for these players’ propensity for creating turnovers when we compare the number of turnovers their assigned opponent committed in games where their team garnered double-digit steals. In the vast majority of the cases, the team got more steals than normal when their high-load defender forced a lot of turnovers.

Patrick Beverley

Frank Ntilikina