Streamlining Evaluation with ir-measures

pdf bibtex code 35 citations demonstration paper

Authors: Sean MacAvaney, Craig Macdonald, Iadh Ounis

Appeared in: Proceedings of the 44th European Conference on Information Retrieval Research (ECIR 2022)

Links/IDs:

DOI 10.1007/978-3-030-99739-7_38 DBLP conf/ecir/MacAvaneyMO22a arXiv 2111.13466 Google Scholar 7wWfoDgAAAAJ:hFOr9nPyWt4C Semantic Scholar e586b41aab5b96b79410d134ab5efba0d35d4856 Enlighten 259597 smac.pub ecir2022-irm

Abstract:

We present ir-measures, a new tool that makes it convenient to calculate a diverse set of evaluation measures used in information retrieval. Rather than implementing its own measure calculations, ir-measures provides a common interface to a handful of evaluation tools. The necessary tools are automatically invoked (potentially multiple times) to calculate all the desired metrics, simplifying the evaluation process for the user. The tool also makes it easier for researchers to use recently-proposed measures (such as those from the C/W/L framework) alongside traditional measures, potentially encouraging their adoption.

BibTeX @inproceedings{macavaney:ecir2022-irm, author = {MacAvaney, Sean and Macdonald, Craig and Ounis, Iadh}, title = {Streamlining Evaluation with ir-measures}, booktitle = {Proceedings of the 44th European Conference on Information Retrieval Research}, year = {2022}, url = {https://arxiv.org/abs/2111.13466}, doi = {10.1007/978-3-030-99739-7_38} }