What we talk about when we talk about research quality. A discussion on responsible research assessment and Open Science

The crisis of the current research assessment system has been at the centre of academic debate for many years but it is only recently that the issue has been brought to the forefront of the agendas of public institutions financing research. In fact, the existing evaluation criteria are believed to place excessive emphasis on quantity and on productivity, leaving aside research quality, collaborative open research methods, and the wider impact of research on society. The article starts from this consideration (section 1) and focuses on the ongoing reform coordinated by the Coalition for Advancing Research Assessment (CoARA), analysing in particular the Agreement on Reforming Research Assessment (ARRA) (section 2), which proposes a vision that relies on qualitative judgement based on peer reviewing, and places research quality at the centre of the reform. The definition of quality which emerges in the ARRA is closely linked to that of Open Science, and in particular that given by the UNESCO recommendation of 2021, to which section 3 is dedicated. Sections 4, 5 and 6 aim to provide the philosophical foundations of the definitions of quality and of Open Science which emerge in the previous sections. In particular, section 4 focuses on the definition of quality starting from the reflection of Robert Maynard Pirsig who, like in the Agreement, identifies quality with a procedural definition, namely the integrity of the scientific method. This connection leads us to the nexus between science and truth, which is addressed in Immanuel Kant's reflection on the purpose of science and on the conditions that make research possible in The Conflict of the Faculties (section 5). The analysis of Kant's considerations leads to defining what the philosopher means by the term Streit (conflict), i.e. a scientific debate between peers, a category that Kant understands in a broad sense. The space for such debate is based on collaboration, which is an essential character of the scientific method itself, as shown in section 6. In the conclusions, the definition of quality and qualitative judgement proposed by the ARRA is analysed again, and an enriched interpretation is proposed.