Knowledge production has no socioeconomic value until it is used in the further advancement of knowledge itself (scholarly impact), or to improve practices, goods, or services through incorporation in product or process technologies (social impact). For diffusion of their research results, scholars encode them in written forms, mostly intended for publication in scientific journals. For assessment of the manuscripts, journal editors recur to peer reviewers, who are asked to provide recommendations based on the quality of the manuscripts (e.g. originality, significance, rigour). These assessments are inevitably subjective, as evidenced by frequent divergences among peers considering a single paper. While quality is the main determinant of future impact, non-scientific factors concur. But, if it is impact that is the ultimate aim of research, then the selection of manuscripts for publication should also be based on those potentially influential non-scientific factors. We analyse nearly two million 2017 publications and their impact, measured by normalized citations accrued to 2022. Based on theory and previous literature, we extrapolate the publication traits of text, byline, and bibliographic references that are expected to relate to future citations. We then fit a regression model with the outcome variable as the scholarly impact of the publication, and the independent variables as the above non-scientific traits, controlling for fixed effects at journal level. Overall, such variables explain more than 20% of the paper's impact, with little variation across disciplines. The remaining 80% is evidently explained by the inherent quality of the manuscript, but also the marketing activities by the authors, and some non-scientific factors not included in the statistical model. Our bibliometric system could assist editors, complementing the recommendations from peer review. In particular, in case of divergent recommendations, the bibliometric system would suggest not rejecting a paper with expected non-negligible future impact, or conversely, rejecting one that would not receive adequate scholarly attention. This avoids the need for recourse to an additional reviewer, with evident advantages in expenses (opportunity cost of additional reviews), and time (delay in diffusing new knowledge to potential users).
Predicting the scholarly impact of manuscripts before their publication
Abramo G;D'Angelo CA;
2023
Abstract
Knowledge production has no socioeconomic value until it is used in the further advancement of knowledge itself (scholarly impact), or to improve practices, goods, or services through incorporation in product or process technologies (social impact). For diffusion of their research results, scholars encode them in written forms, mostly intended for publication in scientific journals. For assessment of the manuscripts, journal editors recur to peer reviewers, who are asked to provide recommendations based on the quality of the manuscripts (e.g. originality, significance, rigour). These assessments are inevitably subjective, as evidenced by frequent divergences among peers considering a single paper. While quality is the main determinant of future impact, non-scientific factors concur. But, if it is impact that is the ultimate aim of research, then the selection of manuscripts for publication should also be based on those potentially influential non-scientific factors. We analyse nearly two million 2017 publications and their impact, measured by normalized citations accrued to 2022. Based on theory and previous literature, we extrapolate the publication traits of text, byline, and bibliographic references that are expected to relate to future citations. We then fit a regression model with the outcome variable as the scholarly impact of the publication, and the independent variables as the above non-scientific traits, controlling for fixed effects at journal level. Overall, such variables explain more than 20% of the paper's impact, with little variation across disciplines. The remaining 80% is evidently explained by the inherent quality of the manuscript, but also the marketing activities by the authors, and some non-scientific factors not included in the statistical model. Our bibliometric system could assist editors, complementing the recommendations from peer review. In particular, in case of divergent recommendations, the bibliometric system would suggest not rejecting a paper with expected non-negligible future impact, or conversely, rejecting one that would not receive adequate scholarly attention. This avoids the need for recourse to an additional reviewer, with evident advantages in expenses (opportunity cost of additional reviews), and time (delay in diffusing new knowledge to potential users).I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
