bibtex short conference paper to appear
Appearing in: 47th European Conference on Information Retrieval (ECIR 2025)
Abstract:
State-of-the-art cross-encoders can be fine-tuned to be highly effective in passage re-ranking. The typical fine-tuning process of cross-encoders as re-rankers requires large amounts of manually labelled data, a contrastive learning objective, and a set of heuristically sampled negatives. An alternative recent approach for fine-tuning instead involves teaching the model to mimic the rankings of a highly effective large language model using a distillation objective. These fine-tuning strategies can be applied either individually, or in sequence. In this work, we systematically investigate the effectiveness of point-wise cross-encoders when fine-tuned independently in a single stage, or sequentially in two stages.Our experiments show that the effectiveness of point-wise cross-encoders fine-tuned using contrastive learning is indeed on par with that of models fine-tuned with multi-stage approaches.
BibTeX @inproceedings{pezzuti:ecir2025-onestageft, author = {Pezzuti, Francesca and MacAvaney, Sean and Tonellotto, Nicola}, title = {Exploring the Effectiveness of Multi-stage Fine-tuning for Cross-encoder Re-rankers}, booktitle = {47th European Conference on Information Retrieval}, year = {2025} }