Bill Hancock stopped himself before saying never.
He still considers the Bowl Championship Series to be “a brilliant mathematical system,” even though it’s remembered today as another lukewarm attempt at improving college football’s postseason.
There were no meetings and no deliberation like the College Football Playoff or NCAA Tournament selection committees conduct now. The way the BCS worked at its most granular level as recently as a decade ago, according to Hancock, is he would receive emails each week containing the computer rankings that constituted one portion of the BCS.
The former BCS executive director would then send those figures to a neutral entity to be compiled along with the national polls and strength of schedule components that made up the remainder of the formula to determine the sport’s national championship game beginning in 1999 through the start of the CFP ahead of the 2014 season.
And Hancock, who retired last month after serving as the CFP executive director since its inception, never wants college football to use something like that again.
“It lost favor quickly, and what it was missing was the human element,” he said. “I was going to say I don’t ever see the loss of the human element, but I’ve been following what’s been happening with the Big Ten and SEC. I think it would only go back if people forget their history.”
Would March Madness metrics work for the College Football Playoff?
As Selection Sunday approaches and March Madness begins, there will be seven different metrics on team sheets to help the 12 members of the men’s basketball selection committee fill out the 2025 NCAA Tournament bracket. There are predictive ratings that reflect a team’s performance based on offensive and defensive efficiency and results-based ratings that judge a team’s resume.
There was nothing near that in place for the CFP selection committee when the college football’s playoff expanded to 12 teams last season and chose SMU over Alabama for the final at-large spot last December. Conference commissioners have discussed changes to the format since then, most notably an increase in the number of teams in the bracket and more automatic bids for the Big Ten and SEC, in particular.
Given the likelihood this will continue to yield more subjective decisions ripe for controversy, might college football benefit from relying more on objective metrics? At least one prominent member of the college basketball analytics community wonders why it hasn’t already gone down that path.
“There’s a huge contrast between college football and college basketball in my mind, which doesn’t make sense because there’s a ton of rating systems out there for college football as well,’ said Ken Pomeroy, who created kenpom.com in 2002 and is credited with ushering in the modern analytics movement in college basketball. “Obviously they went through the whole BCS thing and maybe people are scarred from that or something. But it just feels like there’s so little objective criteria in college football, which is extremely weird to me.”
Matt Morris, ESPN’s director of analytics, confirmed the CFP selection committee did have access to ESPN’s Football Power Index and Strength of Record metrics for its deliberations on selection day this year. But he is among those who believe the smaller sample size in college football – teams play 12 regular-season games as opposed to as many as 31 in college basketball – makes the endeavor much harder and more likely to produce anomalies or outliers on the gridiron.
Michigan State athletic administrator Kevin Pauga, who created the results-based Kevin Pauga Index used by the NCAA Tournament selection committee and has a separate business that helps conferences build schedules, noted whereas each game in college basketball counts for 3% of a team’s resume, that figure rises to 8.5% in college football.
‘You can come up with adjusted efficiency numbers … a lot easier in basketball just by the nature of how the sport is played,’ Pauga said. ‘The availability of the predictive data in basketball is just more prevalent. There’s some ways to do better in terms of football data, but it’s far less a perfect science.”
How to measure CFP’s ‘eye test’
Morris and his team at ESPN nonetheless formulated a “playoff predictor” for this past season, but found predicting what football committee members might do to be a lot trickier than their basketball counterparts.
Not only did the model need to project the winner of each game, but it also had to predict what a committee might do on selection day with a format being implemented for the first time.
“The nebulous eye test just becomes so much more paramount,” Morris said. “You had a little bit more opportunity to leverage the data with 12 (teams), and it seemed like they did that to some degree. But I think there were some thumbs on scales a little bit this year. I don’t mean that controversially. I don’t mean they wanted certain teams in and certain teams out. I just meant the narrative played a little bit of a bigger role in not having another SEC team, for example, when the metrics in the past might have indicated they would have picked (Alabama) for that 12th spot.”
The 16-team NCAA men’s hockey tournament adopted something akin to a metrics-based selection process in recent years, awarding its 10 at-large berths through a computer system that uses record against common opponents, head-to-head record and the Ratings Percentage Index (RPI) to measure a team’s season. There are only 40 Division I men’s ice hockey programs in the country, compared with 134 Football Bowl Subdivision teams and 364 Division I men’s basketball teams this year.
Hancock, who was also the first full-time director of the men’s basketball Final Four from 1989 to 2002, believes there will always be teams that feel unfairly left out no matter which system is put in place over college football and college basketball in the future.
‘It just really works best with a human committee that can use human judgment – informed by the data, of course,’ Hancock said. ‘Consulting the data, using the data, but I’m a big believer in that it’s the best the way it is, both in basketball and football.”
He just knows better at this point than to say that won’t change.
Follow Mark Giannotto on social media @mgiannotto and email him at mgiannotto@gannett.com.