Critical analysis of universities and research institutes’ ranking systems

Document Type : Research paper

Author

Associate Professor Psychology and Education Faculty, Allameh Tabatabaei’ University, Tehran, Iran

Abstract

Purpose: Using the evaluation standards and procedures used by the global ranking systems, the study's primary goal is to offer recommendations for improving the effectiveness of the ranking systems for universities and research institutions.
Methodology: A comparative study technique was utilized to examine four international ranking systems and the ISC in order to determine the evaluation metrics and how universities and research institutions are ranked. Then, a critical analysis was applied and recommendations were given in the area of evaluation indicators and ranking processes of universities and research institutes based on the indicators and ranking processes of the ISC. The chosen indicators were validated by 10 specialists in the domains of scientometrics, webometrics, information science, and epistemology.
Findings: Based on the indicators of the country, primary organization, affiliation, starting point of the activity, primary criteria, data sources, evaluation criteria, extent of coverage, and field of activity, the rating systems were examined. The study's findings revealed that most systems' primary objectives are to assist students in selecting a university and offer recommendations to institutions on how to enhance their operations. The standards of research and education are given particular priority in these systems. Most systems use registration systems and surveys, self-reports, and legitimate databases as their data sources.
Conclusion: The ISC should continue its activities in a private and revenue-generating manner, according to the study's findings, and its goals should be amended. Based on the goals, the primary procedure and criteria should also be in place. It has to be substantially changed.

Keywords

Main Subjects

References

Aguillo, I., Bar-Ilan, J., Levene, M., & Ortega, J. (2010). Comparing university rankings. Scientometrics, 85 (1), 243-56. DOI:10.1007/s11192-010-0190-z
Arasteh, H., & Fazeli Masuleh, S. T. (2012). Rankings and characteristics of world-class universities. Iranian Journal of Engineering Education, 13 (52), 87-101. (Persian)
Benito, M., & Romera, R. (2011). Improving quality assessment of composite indicators in university rankings: A case study of French and German universities of excellence. Scientometrics, 89 (1), 153–176. doi:10.1007/s11192-011-0419-5
Boulton, G. (2011). University rankings: Diversity, excellence and the European initiative. Procedia-Social and Behavioral Sciences, 13, 74-82. DOI: https://doi.org/10.1016/j.sbspro.2011.03.006
Buela-Casal, G., Gutiérrez-Martínez, O., Bermúdez-Sánchez, M., & Vadillo-Muñoz, O. (2007). Comparative study of international academic rankings of universities. Scientometrics, 71 (3), 349-65. DOI:10.1007_s11192-007-1653-8
Chen, K. H., & Liao, P. Y. (2012). A comparative study on world university rankings: a bibliometric survey. Scientometrics, 92 (1), 89-103. DOI: 10.1007/s11192-012-0724-7
Chirikov, I. (2021). Does conflict of interest global university rankings? Berkeley research and occasional paper series. Retrieved August 15, 2022, from: https://escholarship.org/uc/item/8hk672nh
Dill, D., & Soo, M. (2005). Academic quality, league tables, and public policy: A cross-national analysis of university rankings. Higher Education, (49), 495-533.
Ehsani, V. (2019). The most effective indicators of university ranking systems in determining the ranking of Iran universities. Rahyaft, 29 (1), 27-41. (Persian)
Elken, M., Hovdhaugen, E., & Stensaker, B. (2016). Global rankings in the Nordic region: challenging the identity of research-intensive universities? Higher Education, 72 (6), 781-95.
Erfanmanesh, M., & Keshavarzian, S. (2017). Publication delay of scholarly journals: A case for journals accredited by the Ministry of Science, Research & Technology of Iran. Iranian Journal of Information Processing and Management, 33 (1), 1-24. (Persian)
Erkkilä, T. (2014). Global university rankings, transnational policy discourse and higher education in Europe. European Journal of Education Research, Development and Policy, 49 (1), 91-101. DOI:10.1111/ejed.12063
Faizpour, M. Al., Khanalizadeh, R., & Dehmobad, B. (2011). A comparative study of university ranking indicators in the world and Iran [Paper presentation]. In The First National Education Conference in Iran in 1404, Tehran, Iran. (Persian)
Farasatkhah, M. (2008). A study of the existing and the desired state of the “Scientific Evaluation System” in Iran; Emphasizing the Higher Education Sector. Journal of Science and Technology Policy, 1 (2), 43-58. (Persian)
Ghaderi Sheykhi Abadi, M., & Tale Pasand, S. (2013). Determining the Weight of Performance Evaluation Indicators and Ranking Nongovernmental-Nonprofit Higher Education Institutions in Western Provinces of Iran. Quarterly of Educational Measurement, 4 (13), 69-96. (Persian) DOI: 10.22054/jem.2013.5671
Gheitasi, Sh., Ghaderi, M., & Bolandhematan, K. (2021). Investigating the viewpoints of Kurdistan University professors on the ranking system of Iranian universities. Higher Education Letter, 53, 25-49. (Persian)
Hägg, I., & Wedlin, L. (2013). Standards for quality? A critical appraisal of the Berlin principles for international rankings of universities. Quality in Higher Education, 19 (3), 326-42. DOI:10.1080/13538322.2013.852708
Hazelkorn, E. (2013). How rankings are reshaping higher education. In Climent, V., Michavila, F., & Ripolles, M. (Eds.). Los Rankings Univeritarios, Mitos y Realidade (pp. 1-8). Dublin: Technological University Dublin.
Hesabi, S., Sotudeh, H., & Yousefi, Z. (2022). A Correlation Study of bibliometric-based and informed-peer-review university rankings: The case of UK Research Excellence Framework (REF) and the world's prestigious university ranking systems. Scientometrics Research Journal, 8(2), 75-98. (Persian) DOI: 10.22070/rsci.2020.5836.1437
Huang, M. H. (2011). A comparison of three major academic rankings for world universities: From a research evaluation perspective. Journal of Library & Information Studies, 9 (1), 1-25. DOI:10.6182/jlis.2011.9(1).001
Ishikawa, M. (2009). University rankings, global models, and emerging hegemony: Critical analysis from Japan. Journal of Studies in International Education, 13 (2), 159-73. DOI:10.1177/1028315308330853
Kehm, B. (2014). Global university rankings- impacts and unintended side effects. European Journal of Education, 49 (1), 102-12. DOI:10.1111/ejed.12064
Khanizad, R., & Montazer, G. (2017). A comparative evaluation of the world university rankings systems. Journal of Science and Technology Policy, 10 (3), 31-43. (Persian)
Khorsandi Taskoh, A. (2016). International higher education: Strategies and conditions of possibility. Tehran: Research Institute of Cultural and Social Studies. (Persian)
Khorsandi Taskoh, A., & Panahi, M. (2015). Critical analysis of international university ranking systems: Policy suggestions for higher education in Iran. Iranian Higher Education, 8 (3), 111-36. (Persian)
Khosrowjerdi, M., & Zeraatkar, N. (2022). A review of outcomes of seven world university ranking systems. Iranian Journal of Information Processing and Management, 28 (1), 71-84. (Persian)
Kroth, A., & Daniel, H.-D. (2013). International university rankings- A critical review of the methodology. Zeitschrift fur Erziehungswissenschaft, 11 (4), 542-58.
Liu, N., Cheng, Y., & Liu, L. (2005). Academic ranking of world universities using scientometrics. A comment to the fatal attraction. Scientometrics, 64 (1), 101-9. DOI:10.1007/s11192-005-0241-z
Lukman, R., Krajnc, D., & Glavič, P. (2010). University ranking using research, educational and environmental indicators. Journal of Cleaner Production, 18 (7), 619-28. DOI:10.1016/j.jclepro.2009.09.015
Mahdavi Mazdeh, M., Bank, M., Zahedi, M. R., & Poormesgari, M. (2013). Recognition of state universities' entrepreneurship indicators and ranking the universities inthe aspect of entrepreneurship. Journal of, 6 (1), 81-98. (Persian)
Mohammadi, A., Mojtahedzadeh, R., Karimi, A. (2007). Ranking of Iran's medical sciences fields and universities and introduction of the country's top models in 2014. Journal of Medicine Spiritual Cultivation, 14 (3), 65-73. (Persian)
Molinari, J.-F., & Molinari, A. (2008). A new methodology for ranking scientific institutions. Scientometrics, 75 (1), 163-74. DOI:10.1007/s11192-007-1853-2
Niazi, J., & Abu Nouri, I. (2009). Ranking of webometric indicators in selected universities. Library and Information, 13 (4), 143-71. (Persian)
Niyazi, E., & Abounori, A. (2020). Ranking of selected universities based on infrastructures of knowledge management. Commercial Strategies, 9 (50), 263-86. (Persian)
Nourmohammadi, H. A., & Safari, F. (2013). Introductionof  the global rankings of universities and review criteria of this system. Science and Technology Policy Letters, 3 (2), 71-86. (Persian)
Ordorika, I., & Lloyd, M. (2013). A decade of international university rankings: A critical perspective from Latin America. In Marope, P. T. M., Wells, P. J., & Hazelkorn, E. (Eds.), Rankings and accountability in higher education: Uses and misuses (pp. 128-138). Paris: UNESCO Publication.
Pakzad, M., Ghaempanah, A., & Jahan, M. (2013). Studying the status of Iranian mother universities in international ranking Universities and Higher Education Centers. Rahyaft, 22 (53), 33-45. (Persian)
Pusser, B., & Marginson, S. (2013). University rankings in critical perspective. The Journal of Higher Education, 84 (4), 544-68. DOI:10.2307/23486805
Rauhvargers, A. (2011). Global university rankings and their impact. Belgium: European University Association.
Rauhvargers, A. (2013). Global university rankings and their impact: Report II. Brussels: European University Association.
Saka, Y., & Yaman, S. (2011). URSs; criteria and critiques. Journal of Higher Education and Science, 1 (2), 72-9. DOI:10.5961/jhes.2011.012
Shin, J. C., Toutkoushian, R. K., & Teichler, U. (2011). University rankings: Theoretical basis, methodology and impacts on global higher education. Berlin: Springer.
Stolz, I., Hendel, D., & Horn, A. (2010). Ranking of rankings: Benchmarking twenty-five higher education ranking systems in Europe. Higher Education, 60 (5), 507-28. DOI:10.1007/s10734-010-9312-z
Taylor, P., & Braddock, R. (2007). International URSs and the idea of university excellence. Journal of Higher Education Policy and Management, 29 (3), 245-60. DOI:10.1080/13600800701457855
Usher, A., & Savino, M. (2007). A global survey of university ranking and league tables. Higher Education in Europe, 32 (1), 5-15. DOI:10.1080/03797720701618831
Van Raan, A. F. (2005). Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods. Scientometrics, 62, 133-43. DOI: 10.1007/s11192-005-0008-6
Van Raan, A. F., Van Leeuwen, T. N., & Visser, M. S. (2011). Severe language effect in university rankings: Particularly Germany and France are wronged in citation-based rankings. Scientometrics, 88 (2), 495-98. DOI: 10.1007/s11192-011-0382-1
Vatiainen, P. (2002). On the principles of comparative evaluation. Evaluation, 8 (3), 359-71. DOI:10.1177/135638902401462484
Waltman, L., Calero‐Medina, C., Kosten, J., Noyons, E. C., Tijssen, R. J., van Eck, N. J., ... & Wouters, P. (2012). The Leiden Ranking 2011/2012: Data collection, indicators, and interpretation. Journal of the American Society for Information Science and Technology, 63 (12), 2419-32. DOI: 10.1002/asi.22708
Wilbers, S., & Brankovic, J. (2023). The emergence of university rankings: a historical‑sociological account. Higher Education, 86 (4), 733-50. DOI: 10.1007/s10734-021-00776-7
Wu, H. Y., Chen, J. K., Chen, I. S., & Zhuo, H. H. (2012). Ranking universities based on performance evaluation by a Hybrid MCDM Model. Measurement, 45 (5), 856-80. DOI: 10.1016/j.measurement.2012.02.00
Yousefi, M., Moradi, G., & Azimi, A. (2011). Rating of Iranian medical sciences universities in education, based on efficiency index. Iranian Journal of Medical Education, 11 (4), 408-17. (Persian)
Zare Banadkuki, M., Vahdatzad, M. A., Owlia, M. S., & Lotfi, M. M. (2017). Pathology of rankings systems of universities from the perspective of upstream documents of Iranian Higher Education. Library and Information Sciences, 20 (1), 52-87. (Persian)
Zitt, M., & Filliatreau, G. (2007). Big is (made) beautiful: Some comments about the Shanghai-Ranking of World-Class Universities. In S. J. UNESCO European Centre for Higher Education (Ed.), The world class university and ranking: aiming beyond status (pp. 147-165). Paris: UNESCO.
 
 

Files

Share

How to cite

Statistics