New Method Offers Better Way To Rank Universities, Researchers Say

July 26, 2000

GAINESVILLE, Fla. — A group of University of Florida researchers has developed a new way to evaluate America’s top research universities that they say is more objective, reliable and useful than other popular annual college rankings.

The single largest improvement: Rather than using raw data from federal agencies and national organizations that often does not separate individual campuses from entire state university systems, the new method adjusts the numbers so institutions are compared one on one.

Not only will academics find the data useful for understanding the context of research universities, but the data also provide a clear sense of each institution’s performance characteristics, said John Lombardi, the study’s lead researcher.

“Universities are always interested in improving their performance, and these data offer a frame of reference for understanding the comparative performance of their peers,” said Lombardi, former UF president and director of The Center for Studies in the Humanities and Social Sciences, known as TheCenter, which created the new evaluation method.

The new system evolved from a performance-based plan for improving UF during the 1990s. The effectiveness of those techniques brought national attention and prompted TheCenter to translate the performance measurement process into a generally useful set of comparative data.

Don’t look for who’s No. 1, though.

Instead, the study displays the top universities in two tables: one for private institutions and the other for public universities. The tables group the institutions by the number of indicators for which each ranks in the top 25 nationally. The top group consists of the private or public universities that rank in the top 25 on all nine indicators included in the study. The second group includes those with eight indicators in the top 25, etc.

Those nine indicators are:

  • total research and development expenditures,
  • federally sponsored research and development expenditures,
  • number of faculty who are members of The National Academies,
  • number of significant faculty awards,
  • number of doctoral degrees awarded,
  • number of postdoctoral appointments,
  • average SAT scores,
  • endowment size,
    and
  • amount of annual private contributions.
  • Within groups, the list is alphabetical, so no single institution appears at the top of the heap because the authors believe such micro-distinctions have little meaning.
    “Although everyone wants to define No. 1,” said Diane Craig, one of the principal authors of the study, “universities do not keep score precisely enough to produce such a distinction in any useful way. Instead, the best universities are very good at almost everything; the next best are good at most things; and so on. What matters is the overall strength of the institution, not an artificially constructed No. 1 score.”

    In addition to the published data, TheCenter’s Web site (http://thecenter.ufl.edu) offers a data set with other indicators that help put the rankings in context, such as enrollment and student body composition. TheCenter’s site also includes data for all universities with more than $20 million in federal research, and data on the top 100 universities in each of the nine measures. TheCenter plans to update the study annually.

    The researchers believe that by providing all the data that underlies their analysis, other people can use the data in their own work to help measure and support university improvement.

    That said, 10 private universities had all nine measures within the top 25 schools in each category. They are: the California Institute of Technology, Columbia University, Duke University, Harvard University, Johns Hopkins University, the Massachusetts Institute of Technology, Northwestern University, Stanford University, the University of Pennsylvania and Yale University.

    In the public sector, four schools had all nine indicators within the top 25 schools in each category: the University of California-Berkeley, the University of California-Los Angeles, the University of Michigan-Ann Arbor and the University of North Carolina-Chapel Hill.

    Among Florida’s private universities, the University of Miami appears in the seventh group with three measures in the top 25. Among the public universities, the University of Florida appears in the second tier with eight measures within the top 25, the study shows.

    “In the end,” Lombardi said, “the only thing that counts is whether the university improves, and the competition for the people and resources needed to improve at this level is intense.”