Challenges and Requirements for Monitoring the Objectives of Science and Technology in the Seventh Development Plan

Document Type : Research paper

Authors

1 Faculty Member, Management of Technology, Shahid Beheshti University, Tehran, Iran

2 PhD Candidate, Science and Technology Policymaking, Tarbiat Modares University, Tehran, Iran

3 Assistant Professor, Science and Technology Policymaking, National Research Institute for Science Policy, Tehran, Iran

Abstract

Monitoring and evaluation, by providing feedback on the outputs, outcomes, and impacts of policies, not only contribute to understanding of the factors responsible for the success or failure of policies, but also play a crucial role in enhancing learning within the policy process. Given the challenges pertinent to monitoring and evaluating science and technology (S&T) objectives in Iran—and consequently, the difficulties in monitoring the objectives stipulated in national policies, particularly development plans— this research aims to explore and analyze the performance objectives of science and technology in the Seventh Development Plan and to propose policy recommendations to overcome them. Following review of the literature on challenges of S&T monitoring, given the context of Iran, and semi-structured interviews with three experts, a conceptual framework of S&T monitoring challenges was developed. Then, a panel of 12 experts from relevant institutions, including the Ministry of Science, Research and Technology, Statistical Center of Iran, Ministry of Health and Medical Education, National Research Institute for Science Policy, and the Vice-Presidency for Science and Technology was convened to gather insights regarding the challenges and imperatives for monitoring S&T objectives of the seventh development plan. Findings of this research reflect that the main barrier to S&T objectives monitoring include idealistic goal-setting, insufficient pragmatic consideration of existing conditions and capabilities, not properly engaging experts in drafting S&T policies and their objectives, lack of standard definitions for S&T indicators, lack of attention to innovation-related indicators, and inadequate resource allocation. Effective resolve to aforementioned challenges includes the development of a national S&T monitoring and evaluation manual, setting STI targets by benefiting from experts' engagement, and measurability considerations throughout the policy and goal setting stage. Furthermore, lack of a S&T data governance system and the difficulty of collecting data through surveys are key shortcomings in data collection dimension. These challenges can be addressed by reinforcing data governance, establishing a statistics and registry-based information system, and enhancing the legal enforceability of data collection and publication among agencies responsible for S&T statistics. Lastly, lack of a national division of labor among institutions responsible for S&T monitoring and evaluation, and the absence of legal mechanisms to enforce data provision, are major institutional challenges. Our recommendations to overcome institutional challenges entail enhancing financial, human, and infrastructural resources allocated to stakeholder organizations in the monitoring of S&T objectives.

Keywords

Main Subjects

References

Abbaszadeh, M. (2012). Validity and reliability in qualitative researches. Journal of Applied Sociology, 23(1), 19-34. (Persian)
Alizadeh, P., & Ghazinoori, S. S. (2020). Requirements, Challenges, and Policy Recommendations for Measuring R&D Expenditure in Iran. Rahyaft, 30(1), 107-124. (Persian). DOI: 10.22034/rahyaft.2020.13822
Allas, T., Bravo-Biosca, A., Phipp, J., Hart, M., Laatsit, M., & Roper, S. (2018). Evaluation framework: How we assess our impact on business and the economy. Retrieved from: https://research.cbs.dk/en/publications/evaluation-framework-how-we-assess-our-impact-on-business-and-the
Al-Mubaraki, H. M., Muhammad, A. H., & Busler, M. (2015). Measuring innovation: The use of indicators in developed countries. World Journal of Entrepreneurship, Management and Sustainable Development, 11(3), 220-230. DOI: 10.1108/WJEMSD-02-2015-0007.
Azadi, G. (2020). Analyze the role of the country's science, technology and innovation assessment bodies and propose a comprehensive monitoring and evaluation system. Rahyaft, 30(1), 125-140. (Persian). DOI: 10.22034/rahyaft.2020.13823
Baghjanati, M., CheshmehSohrabi, M. and Jamali, H. (2024). Analysis of the documents of research evaluation in Iran: Towards the National Research Evaluation System (NRES). Librarianship and Information Organization Studies, 35(3), 175-220. (Persian). DOI: 10.30484/NASTINFO.2024.3584.2272
Bundi, P., & Trein, P. (2022). Evaluation use and learning in public policy. Policy Sciences, 55(2), 283-309. DOI:10.1007/s11077-022-09462-6 ‏
Cresweel, J. (2012). Qualitative inquiry and research design: Choosing among five approaches. Thousand Oaks: Sage Publications.
Esparaein, F., & Noormohammadi, H. (2014). Evaluation of science and technology (challenges and solutions). Congress of Progressive Pioneers, 5, 609-618. (Persian)
Farazkish, M. (2024). Analyzing the success of Iran's Sixth Development Plan from the policy evaluation perspective (Exploration in the field of research and technology). Iranian Journal of Public Policy, 10(4), 9-36. (Persian) DOI: 10.22059/jppolicy.2024.99824.
Ghaffarzadegan, M., & Ghazinoory, S. (2025). Research method in understandable language.  Tehran: Chapar. (Persian)
Ghazanfari, R., & AliAhmadi, A. (2019). National innovation systems in Iran: Challenges and approaches. International Journal of Industrial Engineering and Management Science, 6(1), 1-23.‏
Ghazinoory, S. and Farazkish, M. (2018). A modal for STI national evaluation based efficiency, effectiveness and utility index. Strategic Studies of Public Policy, 8(27), 205-229. (Persian)
Ghazinoory, S., Farazkish, M., Nasri, S., & Mardani, A. (2023). Designing a Science, Technology, and Innovation (STI) evaluation dashboard: A comprehensive and multidimensional approach. Technology Analysis & Strategic Management, 35(8), 1005-1023.‏ DOI: 10.1080/09537325.2021.1990877.
Hall, B. H., & Jaffe, A. B. (2018). Measuring science, technology, and innovation: A review. Annals of Science and Technology Policy, 2(1), 1-74. DOI:10.1561/110.00000005.
Janavi, E. (2020). Analysis of indicators for measuring and evaluating the fields of technology and innovation in the country policy documents. Rahyaft, 30(1), 25-44. (Persian). DOI: 10.22034/rahyaft.2020.13818.
Kamali, Y. (2023). Identifying and ranking the challenges of evaluating development policies in Iran (five-year development plans). Majlis and Rahbord, 30(115), 91-125. (Persian) DOI: 10.22034/mr.2022.5393.5150.
‏Khayyatian, M. S., Fartash, K., & Pourasgari, P. (2020). Development of a framework for monitoring and evaluation of Iran’s National System of Science, Technology and Innovation. Strategy for Culture, 13(49), 119-154. (Persian) DOI:10.22034/jsfc.2020.109868.
Kusek, J. Z., & Rist, R. C. (2004). Ten steps to a results-based monitoring and evaluation system: A handbook for development practitioners. Washington, D.C.: World Bank Publications.‏
Lengrand, L., & Associés, S. I. (2006). A practical guide to evaluating innovation programmes. European Communitoes, Brussels-Luxembourg.‏ Retrieved from: chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/http://www.magmaconsulting.be/IMG/pdf/sar1_smartinnovation_master2.pdf
Liu, X., & Liu, J. (2011). Science and technology and innovation policy in China. Londresp: Anthem Press.‏ DOI:10.7135/UPO9781843318149.008
Mehrotra, S. (2013). Monitoring, evaluation and performance management in South Asia: The challenge of building capacity. Evaluation, 19(1), 74-84.‏ DOI:10.1177/1356389012471257.
Montes, R. (2019). Evaluation in science and technology in Argentina. State of situation and proposals. Ciencia, Tecnología y Política, 6(1-2), e024. DOI: 10.24215/26183188e024
Muzari, T., Shava, G. N., & Shonhiwa, S. (2022). Qualitative research paradigm, a key research design for educational researchers, processes and procedures: A theoretical overview. Indiana Journal of Humanities and Social Sciences, 3(1), 14-20.
Namdarian, L. (2017). Evaluation of Science, Technology, and Innovation (STI) in Iran. Collnet Journal of Scientometrics and Information Management, 11(2), 271-235.  Doi:10.1080/09737766.2017.1321728
Namdarian, L., Kalantari, N., & Alidoosti, S. (2017). Assessment of science, technology and innovation. Tehran: Chapar. (Persian)
Noroozi Chakoli, A., & Abdi, S. (2020). Challenges, issues and structural require elements for the implementation of the national system for evaluating the effectiveness of science, technology and innovation. Rahyaft, 30(1), 89-106. (Persian) DOI: 10.22034/rahyaft.2020.13821
Ortiz-Núñez, R., Novo-Castro, S., & Casate-Fernández, R. (2023). Indicators for the evaluation of science, technology and innovation activities: A systematized review. Journal of Scientometric Research, 12(2), 448-458.‏ DOI:10.5530/jscires.12.2.041
 Fteval. (2005). Evaluation standards in research and technology policy. Plattform Forschungs- und Technologieevaluierung. Retrieved from https://wbc-rti.info/object/document/7019/attach/EvaluationStandardsinRTDfteval.pdf
Sanderson, I. (2002). Evaluation, policy learning and evidence-based policy making. Public Administration, 80(1), 1-22. DOI:10.1111/1467-9299.00292
Shahmirzadi, T., Hariri, N., Fahimnia, F., Babalhavaeji, F., & Matlabi, D. (2019). Investigation of evaluating indicators for science, technology and innovation in the agricultural research, education and extension organization. Scientometrics Research Journal, 5(9), 47-66. (Persian) DOI:10.22070/rsci.2018.639
Soltani, A. M., & Tabatabaeian, S. H. (2019). Science, Technology, and Innovation Policy Evaluation. Journal of Science and Technology Policy, 12(2), 561-578. (Persian)
Tabatabaee, A., Hasani, P., Mortazavi, H., & Tabatabaeichehr, M. (2013). Strategies to enhance rigor in qualitative research. Journal of North Khorasan University of Medical Sciences, 5(3), 663-670. (Persian) DOI: 10.29252/jnkums.5.3.663
The National Advisory Council on Innovation. (2020). Monitoring and evaluating framework for the south African science, technology and innovation system. Retrieved from: https://www.naci.org.za/index.php/monitoring-and-evaluation-framework-for-the-south-african-science-technology-and-innovation-system
UK (United Kingdom), HM Treasury. (2020). The Magenta Book: Central government guidance on evaluation. London: HM Treasury.
United Nations Institute for Training and Research. (2012). Monitoring and evaluation policy framework. Retrieved from: https://unitar.org/sites/default/files/uploads/pprs/unitar_monitoring_and_evaluation_policy_framework.pdf
World Bank . (2024). Innovation policy learning from Korea: The case of Monitoring and Evaluation (M&E). Ministry of Economy and Finance. Retrieved from chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://documents1.worldbank.org/curated/en/099435005132470189/pdf/IDU-08b01532-579b-4079-83af-de18cf7459ba.pdf
 

Files

Share

How to cite

Statistics