The global transition toward responsible research assessment (RRA), formalized by the Agreement on Reforming Research Assessment (ARRA), emphasizes qualitative evaluation over quantitative metrics . Despite broad international support, a significant gap persists between high-level policy commitments and their practical implementation . This study examines the operationalization of ARRA Commitment 2—balancing qualitative judgment with the responsible use of indicators—within career progression procedures at the National Research Council of Italy (CNR) . Following its 2022 signature of the ARRA, CNR launched four competitive calls in 2023 that introduced Narrative CVs and a framework focused on qualitative merit . Through a qualitative content analysis of 4,090 criteria defined by 90 evaluation committees, the research reveals a partial and uneven implementation of these principles . While research outputs are increasingly subject to qualitative assessment, mechanical scoring remains dominant for other contributions, such as teaching and evaluation . The findings highlight heterogeneous interpretations of qualitative assessment across disciplinary clusters and varying levels of guidance provided to committees . Ultimately, the study underscores the challenges of translating reform principles into consistent practice and encourages research organizations to "Evaluate your Evaluation" to better align assessment processes with career development goals .
From Commitment to Practice: Implementing ARRA Commitment 2 in Career Progression Procedures at the National Research Council of Italy
Provost Lottie
;Di Donato Francesca;Mannocci Andrea;Pecoraro Fabrizio
2026
Abstract
The global transition toward responsible research assessment (RRA), formalized by the Agreement on Reforming Research Assessment (ARRA), emphasizes qualitative evaluation over quantitative metrics . Despite broad international support, a significant gap persists between high-level policy commitments and their practical implementation . This study examines the operationalization of ARRA Commitment 2—balancing qualitative judgment with the responsible use of indicators—within career progression procedures at the National Research Council of Italy (CNR) . Following its 2022 signature of the ARRA, CNR launched four competitive calls in 2023 that introduced Narrative CVs and a framework focused on qualitative merit . Through a qualitative content analysis of 4,090 criteria defined by 90 evaluation committees, the research reveals a partial and uneven implementation of these principles . While research outputs are increasingly subject to qualitative assessment, mechanical scoring remains dominant for other contributions, such as teaching and evaluation . The findings highlight heterogeneous interpretations of qualitative assessment across disciplinary clusters and varying levels of guidance provided to committees . Ultimately, the study underscores the challenges of translating reform principles into consistent practice and encourages research organizations to "Evaluate your Evaluation" to better align assessment processes with career development goals .I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
