Quite a few magazines and institutions publish college and university rankings. Different criteria used by rating agencies have generated different ranking orders. Universities ascending in the ratings boast with pride about their independently documented quality. They use ranking to attract higher caliber students and faculty as well as to boost research grants and donations. Those descending quibble over the validity of the metrics employed. Parents and students regularly refer to the ranking reports when deciding their university preferences. Their decisions obviously carry significant financial burdens and life-changing outcomes. The stakes are high for all constituencies. Students and parents should read college rankings with critical eyes.

 

Magazines use ranking to boost circulation and thus to generate revenue. The releases of survey results appear to attract increased readership. It is said that US News and World Report that news stand sales jump 50% with the issue of its college surveys. Website hits climb to 10 million within three days compared to the 500,000 mouthy averages. One can safely assume that the bulk of this added attention comes from the one constituency, parents and students, that has the greatest interest in rankings’ relevance and efficacy comparing specific institutions. 

 

It is important to probe beneath the media hype to gauge the utility of these increasingly popular surveys to parents and students. Two questions are in order: What data supports these assessments of relative superiority? More important than institutional bragging rights aside, what is their utility in helping parents and students in making their university choice? The following are the variables used by several popular college ranking institutions. 

 

U.S. News and World Report

US. News uses the following metrics and weights to determing its annual college ranking. 

1. Academic Peer Review, a composite globally balanced survey of nearly 10,000 academics, is the centerpiece of the scoring scheme weighted at 40%, and is characterized as an input measure.

2. Employer Review, a composite globally balanced survey, includes employer appraisals of hard and soft skills of bachelor and master graduates, weighted at 10% of the scoring scheme and is characterized as a proxy output measure. I say proxy because it is a composite of what is essentially subjective consumer after purchase opinion.

3. Faculty Student Ratio, universally available metric, weighted at 20%. Low faculty to student ratios have long been a pedagogical ideal. I characterized as an input measure.

4. Citations per Faculty, based on the Scopus abstract and citation database of research literature, includes latest five complete years of data with the total citation count factored against the number of faculty, weighted at 20%, and is as an input measure.

5. International Faculty, said to reflect the attraction of quality faculty to world class institution, weighted at 5%, and is characterized as an input measure.

6. International Students, said to reflect the attraction of quality faculty to world class institution, weighted at 5%, and is characterized as an input measure.

 

Times Higher Education – QS World University Rankings

Times Higher Education – QS World University Rankings collaboration focused on a nearly identical blend of quality indicators

1. Academic peer review, weighted at 40%, and is characterized as an input measure.

2. Employer review, weighted at 10% and again is characterized as a proxy output measure

3. International faculty score, weighted at 5%, and again is characterized as an input measure with the same concern as noted above.

4. International student score, weighted at 5%, and again is characterized as an input measure with the same concern as noted above.

5. Faculty/student score, weighted at 20%, and again is characterized as an input measure with the same concern as noted above.

6. Citation/Faculty score, weighted at 20%, and again is characterized as an input measure.

 

Shanghai Jiao Tong University’s Academic Ranking of World Universities

The Shanghai Jiao Tong University’s Academic Ranking of World Universities places the greatest emphasis on emphasis on aggregate faculty research productivity.

1. Alumni of an institution winning Nobel Prizes and Fields Medals, weighted at 10% and is characterized as a proxy output measure.

2. Staff of an institution winning Nobel Prizes and Fields Medals, weighted at 20% and is characterized as an input measure

3. Highly cited researchers in 21 broad subject categories, weighted at 20% and is characterized as an input measure.

4. Articles published in Nature and Science, weighted at 20% and is characterized as an input measure.

5. Articles Indexed in Science Citation Index-Expanded and Social Science Citation Index, weighted at 20% and is characterized as an input measure.

6. Academic performance with respect to the size of an institution, weighted at 10% and is characterized as a leveling measure.

 

When reading the ranking, students and parents should ask the following questions:

1. Do the criteria used in ranking make sense? Do they actually measure the quality of education?

2. Are the weights used in the ranking appropriate?

3. Are there any missing variables in the ranking?

4. Has the data used in ranking been audited by an independent auditing agency? Did universities misrepresent the data?