Comparing Defensive Projections
There’s a couple of different defensive projections that are currently available for all to see. You’ve got mine linked over on the right sidebar, and Jeff Z’s available through this link. The beauty of the two sets of projections are that the respective methodologies are discussed in the articles presenting them, and the projections are fairly simple to compare (i.e. only one number really). A second positive is that the methodologies only differ by one element, the FSR, as Jeff includes 4 yrs UZR (when available) and I include 3 yrs + the fans. Since that’s the only difference, it makes drawing some insights/conclusions from analyzing the differences a little simpler, and that’s exactly what I’m going to step through here. Jeff is doing the same over at BtB, so go check out his piece as well.
First, I’d like to get a feel for just how different the two projections were. For that a simple distribution should do the trick. The absolute difference is across the x axis and the count is on the y.
Clearly the majority of the differences are less that 4 runs and over half has a difference of 0 or 1. Given that there are differences though, what positions are the differences coming from. In this chart absolute difference is again across the x, but now percent (by position) is on the y.
At first glance it seems like the outfield becomes more prevalent the farther right you go…
So now that we have a decent idea of the magnitude of the differences, it’s time to dig into where the actual differences are. Who is affected by adding in the FSR as a factor? I’ll answer that question by examining two parameters: 1)Experience of the player and 2)Position / FSR rank combination. This first table highlights the experience piece
|Game Bin||AVG ABS Diff||STD DEV||Count|
As one would expect the less experience the player has the bigger the difference between the two projections. The FSR number are a larger percentage of the puzzle for less experienced players as I weighted it at 125 games no matter what the experience level of the player was. [Update: I used my effective defensive games to bin the games, not actual games as Jeff did in his analysis]
Finally, which position / FSR rank combos gained the most by inclusion of the FSR
and lost the most
All told it appears that the FSR does make a difference, but it’s usually only on the order of a couple of runs, which is well within the margin of error for UZR. It has the potential to clear up the picture for players with limited major league experience, as it makes the “available data set” larger, so there is less regression to the mean.