The lack of efficiency in neural information retrieval remains one of the primary obstacles to deploying neural retrieval models as a first-stage retriever at scale. While recent tools have improved the standardized measurement of model efficiency, substantial progress is still needed to enable systematic comparative evaluation, for example, in terms of standards for systems and hardware configurations, cloud-based evaluation, benchmarks, and reproducibility. Beyond measurement, the IR community also needs stronger incentives to move in this direction, such as cost-efficiency as a review criterion or as efficiency and/or effectiveness measures in shared tasks, related teaching materials, efficiency-oriented user studies, and specialized awards for efficiency achievements. In particular, developing more efficient variants of highly effective retrieval algorithms should become an admissible research goal for PhD students if cost-efficiency is to become a first-class design objective in∼IR. With ReNeuIR, we have established a recurring forum where these questions and new ideas are discussed and where the community comes together to collaboratively evaluate and improve efficiency benchmarking frameworks - -most notably through the organization of a shared task focused on efficiency and reproducibility.

ReNeuIR at SIGIR 2026: the Fifth Workshop on reaching efficiency in neural information retrieval

Nardini Franco Maria;
2026

Abstract

The lack of efficiency in neural information retrieval remains one of the primary obstacles to deploying neural retrieval models as a first-stage retriever at scale. While recent tools have improved the standardized measurement of model efficiency, substantial progress is still needed to enable systematic comparative evaluation, for example, in terms of standards for systems and hardware configurations, cloud-based evaluation, benchmarks, and reproducibility. Beyond measurement, the IR community also needs stronger incentives to move in this direction, such as cost-efficiency as a review criterion or as efficiency and/or effectiveness measures in shared tasks, related teaching materials, efficiency-oriented user studies, and specialized awards for efficiency achievements. In particular, developing more efficient variants of highly effective retrieval algorithms should become an admissible research goal for PhD students if cost-efficiency is to become a first-class design objective in∼IR. With ReNeuIR, we have established a recurring forum where these questions and new ideas are discussed and where the community comes together to collaboratively evaluate and improve efficiency benchmarking frameworks - -most notably through the organization of a shared task focused on efficiency and reproducibility.
2026
Istituto di Scienza e Tecnologie dell'Informazione "Alessandro Faedo" - ISTI
979-8-4007-2599-9
Algorithms
Efficiency
Neural IR
Ranking
Retrieval
Sustainable IR
File in questo prodotto:
File Dimensione Formato  
Nardini et al_ReNeuIR_SIGIR_VoR.pdf

accesso aperto

Descrizione: eNeuIR at SIGIR 2026: The Fifth Workshop on ReachingEfficiency in Neural Information Retrieval
Tipologia: Versione Editoriale (PDF)
Licenza: Creative commons
Dimensione 897.37 kB
Formato Adobe PDF
897.37 kB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/20.500.14243/598123
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 0
  • ???jsp.display-item.citation.isi??? ND
social impact