I have a number of things in life that I’m passionate about. Two of those things are sports, and statistical analysis. Note that I said “statistical analysis” and not just statistics – I don’t think anybody actually LOVES to run numbers. What I love to do is use numbers to explain things or make arguments. In 2005, the Los Angeles Dodgers entered the season having either traded away or failed to re-sign quite a few of their big name players, instead bringing in players ranging from the well-known (J.D. Drew) to the bizarre (Norihiro Nakamura). Without having read Moneyball, or anything by Baseball Prospectus or any other sabermetric website, I did a statistical analysis to predict that the Dodgers would be a much worse team, and predicted them to win many fewer games than the 93 they won in 2004. I did this by running a regression analysis to determine which baseball statistics best correlated with run scoring (on-base percentage and slugging percentage, same as argued in Moneyball) and then used those statistics to make predictions. It was the ability to use statistics to make an argument (that the Dodgers would be much worse) that I loved so much.
It’s much harder to do that in MMA for a number of reasons. One is that the data is so limited. Sure, there are plenty of statistics available on Fight Metric for a fighter like Chuck Liddell, but what about a fighter like Antonio McKee? McKee has fought a lot of times, and is a very good fighter in my opinion, but there’s only one fight of data available for him. It’s impossible to judge how good McKee is (at anything) based on that lack of information.
To illustrate this, let’s talk about football. In football, there are three layers of potential analysis. They are:
- Evaluation based on games (wins/losses)
- Evaluation based on possessions or points scored/points against
- Evaluation based on individual plays
The biggest challenge with football analysis (and the reason I chose football as an analogy to MMA) is sample size. A sample of 16 games is just not enough to have a good idea about how good a football team is. Sure, you get a general idea. I’m a Minnesota Vikings fan. As delusional as I CAN be, there’s no getting around a 2-8 record. Good teams don’t go 2-8 in the NFL. Regardless, there’s not much precision in evaluating football teams that way. If somebody decided to predict all NFL games in 2011 based on record, with home-field advantage being the tiebreaker, that person would be 100-63 picking games so far, good for 61.3% accuracy. That’s not necessarily bad, but it could be better.
That’s a big part of why the Football Outsiders created their DVOA statistic. By going into NFL play-by-play data, they could evaluate teams on a much deeper level. Instead of dealing with a sample size of 16 or 10, they were able to greatly improve the sample size by looking at every play in a team’s season, and then adjusting for the situation and the quality of the opponent.
As great as Fight Metric is, it just doesn’t provide the kind of information needed to break MMA down past the first level of evaluating fighters by wins and losses (on an OBJECTIVE basis). Unlike in baseball, basketball, or football, there’s no such thing as a “play” in MMA. In a hypothetical world, it might be telling to take Fight Metric’s Effectiveness Algorithm and break up a fight into individual minutes, but even then, just as Football Outsiders does with DVOA, adjustments would need to be made based on the quality of the opponent. In any case, the Effectiveness Algorithm is proprietary and the results of it are only released for a select few fights.
That means that, for people like me, we’re relegated to the first tier of statistical analysis: evaluating fighters based on their wins and losses. That’s the entire concept behind SILVA in a nutshell: I make this information as good as I possibly can by adjusting for the quality of the opponent. That specific opponent adjustment (Victory Score) needs tweaking, and that will happen in the next version of SILVA, but there’s only so far I can go with such limited data. (Funny enough, SILVA 1.0’s prediction accuracy for UFC fights was almost exactly the same as the 61.3% cited above for NFL games.)
My hope is that, over time, there will be more widespread data collection for MMA fights, and it will become possible to conduct deeper, more meaningful statistical analysis in MMA, and I do have to applaud Rami Genauer and the Fight Metric team for getting the ball rolling on this data collection. For now, as flawed as it is, SILVA is the best I can do as far as objective analysis of MMA fighters is concerned. I’ll keep looking for ways to improve SILVA (and there IS room for improvement), but until I can go beyond the fights and truly evaluate the underlying performances, there’s only so far I can take statistical analysis in MMA.