What is in a Computer Ranking

Most college football fans know of the BCS, and may even know how it works, but I'm guessing most people do not really have much of an idea about the factors that go into the various computer rankings. While the powers that be have mandated that the computer rankings used in the BCS cannot take "margin of victory" into account, each ranking still has its unique kicharacteristics. To the extent that the authors of the rankings have made their methods publicly available, I'd like to offer a few insights into what makes each of these computer programs (and their creators) tick. Anderson - Hester: http://www.andersonsports.com/football/ACF_frnk.html Jeff Anderson is a professor of Political Science at the Air Force Academy, and Chris Hester is a Seattle-area broadcaster. The authors are very clear as to what they feel makes their rankings unique. When computing a team's strength of schedule (SOS), the Anderson-Hester rankings consider the strength of conference that each team is in, not just the individual won-loss records. They also make it clear that their rankings have no pre-season bias. Each team starts the year with a blank slate, and the rankings are not even initialized until after the 5th week of the season, so each team's rankings are always solely a reflection of their performance that season. Richard Billingsley: http://www.cfrc.com/ Richard is a former minister, a rose connoisseur, a management consultant, and perhaps most importantly to fans, a college football historian. He has been using mathematics and computers to rank teams for almost 40 years. Richard's method uses four "phases" to rank the teams. First of all, the Billingsley rankings start off the season with each team ranked as they were at the end of the previous season. Each team is separated by one rating point (#1 gets 270). While, in general, his calculations are designed to make the rankings more stable from week-to-week, he relaxes the rules for the first few games to allow teams to make larger moves, especially to account for head-to-head results. He feels that this allows teams to "earn" a jump in the rankings, rather than simply relying on preseason perceptions. Next, his system determines the strengths of the opponents, by looking at not only win-loss records, but also the rating and rank of the opponents. Teams move up and down the rankings based on the quality of their opponents, but their "forward progress" is limited by the number of losses they have. For example, an undefeated Team A will earn more points for a win over Team C than Team B with one loss would get for that same win over Team C. Richard says that this is designed to prevent a team with multiple losses from remaining highly ranked unless they have played a very tough schedule. Finally, some additional adjustments are made for playing on the road (with a bigger gain for winning as a road underdog), holding an opponent to six points or less, and for a team's overall record. The last adjustments are made so that if two teams somehow end up with the same rating otherwise, the tiebreakers go to teams with better records, that have played more on the road, and have the best defensive performance. Sounds simple enough, huh? Colley Matrix: http://www.colleyrankings.com/ Dr. Wes Colley is a research scientist in astrophysics at the University of Alabama. He has provided a description of his rankings in a 23-page paper on his website. In case you don't want to try and read it, here is a very short summary. Basically, he uses Laplace's Method (using a craps table analogy) to develop a modified winning percentage statistic: (1 + wins) / (2 + wins + losses). As time goes on and games are played, wins are added to the numerator. However, his method then starts adjusting a team's wins and losses into "effective" wins and losses, based on the ratings of the team's opponents. Thus, for example, you may only get credit for 5/6 of a win against a losing team, but also might only get 5/6 of a loss against a winning team. Of course, each team's rating depends on their opponents' ratings, which depend on everyone's ratings, including the original team. So, the method uses iterations until the ratings converge. In the end, the method has a matrix of 117 equations with 117 variables (one for each team's rating). The average ratings of all teams is 1/2, which is the same rating each team starts with. In other words, a team's rating can only improve at the expense of another team. The method does not require a pre-season ranking (or bias), does not adjust for conferences or home/away, and does not include any games where both teams are not Division 1-A. So, beating a 1-AA team does not affect your rating at all, but then again, neither would LOSING to a 1-AA team. A great feature of his website is that you can add / remove actual or hypothetical games and see how the rankings would change. Massey Ratings: http://www.mratings.com/rate.php?lg=cf Ken Massey is a math professor at Carson-Newman College. His method starts with a weighted average of the previous season rankings, but as the season is played out, the effect of the previous season's rankings decays to become negligible. Ken's system ranks 706 college football teams, from 1-A to NAIA, and rely on connections between common opponents. Massey's ratings depend only on the score, where the game is played, and the date the game is played. Ken believes that recent performance is more important than early games, so as the season progresses, the results of the early-season games mean less and less. The easiest way to explain how the Massey ratings work is that the method compares the probability that Team A should beat Team B with the actual result. Ratings are calculated by the strength of the schedule a team faces, and how they perform against that schedule. Home-field advantage is taken into account in both the strength of schedule and the probability of which team wins. The actual home-field advantage of a particular team is computed by comparing their home and away performances; it is not the same for all teams. While Ken's rating system used by the BCS does not include margin of victory (as required by the BCS for all six rankings), he also provides a rating system that does factor in the scoring margins. Sagarin's Elo-Chess: http://www.kiva.net/~jsagarin/sports/cfsend.htm Jeff Sagarin, a graduate of MIT and Indiana, is probably the most well-known sports computer rankings guru. He has rankings for football (NFL, college, and high school), basketball, baseball, NASCAR, golf, soccer, and even volleyball. Like Ken Massey, Jeff has one college football ranking that is used by the BCS that does not use margin of victory (Elo-Chess), and one that relies on scoring margin (Predictor). He also blends the two for an overall rating. The Elo-Chess system is based on a system originally designed to rank chess players, and has also been adapted for other two-player and multi-player games. Jeff's Elo-Chess system uses a preseason rank for the early part of the season, but once every team among the 241 1-A and 1-AA teams that are ranked is connected by common opponents, the preseason rankings mean nothing. Jeff's ratings also consider home-field advantage. In brief, the Elo system assumes that the performance of one competitor in a particular game is a normally distributed random variable, and that mean value of those performances changes slowly as more games are played. In a game, the expected score (or probability of winning) for each team is calculated based on the current ratings. The side that wins is assumed to have performed at a higher level than the side that loses. If one competitor outperforms their expectations, then their rating is assumed to be too low, and is consequently raised to account for that performance. In chess, a player's rating changes based on their performance in a tournament of players of various skills and ratings. In football, Jeff's system looks at a team's performance over the course of a season. Teams that perform well against competitors of high rating are rewarded with an increase in their own rating. Peter Wolfe: http://prwolfe.bol.ucla.edu/cfootball/ratings.htm Dr. Peter Wolfe is an AIDS researcher at UCLA, who uses his spare time to research football and basketball. The details of Peter's rating system are not readily available. However, the basic premise of his system is based on the Bradley - Terry model of maximum likelihood. In this model, the probability that Team A beats Team B is Pab = Ra / (Ra + Rb), where Ra and Rb are the ratings of teams A & B respectively. The overall probability (P) that every game had the result it did is the product of Pab from every game in the season played thus far. Similar to the Massey ratings, Wolfe's system compares all 700+ college teams that are connected by common opponents. The actual ratings are determined by finding the one set of ratings that maximizes the value P. In other words, he tries to find the ratings that would best explain the probability of the entire season unfolding exactly as it has up to that point. His system is known to account for home field advantage, but exactly how this and other factors fit in is not easily found. However, his website does provide one other valuable resource ... he lists the scores of every college football game played for the season on one page, noting which team is home and which is the visitor, or the location of a neutral site. Anyone who wants to try to create their own ranking system can start with this list. Good luck!