pdf bibtex reproducibility paper
Appeared in: 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2026)
Abstract:
Deep research has emerged as an important task that aims to address hard queries that need extensive open-web exploration. To tackle it, most prior work equips large language model (LLM)-based agents with opaque web search APIs, enabling agents to iteratively issue search queries, retrieve external evidence, and reason over it. Despite search's essential role in deep research, black-box web search APIs leave the behaviour of established text ranking methods in deep research largely unclear. To fill this gap, we reproduce key findings and best practices for text ranking methods in deep research. We examine their effectiveness from three perspectives: (i) retrieval units (documents vs. passages), (ii) pipeline configurations (different retrievers, re-rankers, and re-ranking depths), and (iii) query characteristics (the mismatch between agent-issued queries and the training queries of text rankers). We perform experiments on BrowseComp-Plus, a deep research dataset with a fixed corpus, evaluating 2 open-source agents, 5 retrievers, and 3 re-rankers. We find that agent-issued queries typically follow web-search-style syntax (e.g., quoted exact matches), favouring lexical, learned sparse, and multi-vector retrievers; passage-level units are more efficient under limited context windows, and avoid the difficulties of document length normalisation in lexical retrieval; re-ranking is highly effective. We further propose a query-to-question (Q2Q) method that translates agent-issued queries into natural language questions, significantly reducing the query mismatch.
BibTeX @inproceedings{meng:sigir2026-textranking, author = {Meng, Chuan and Ou, Litu and MacAvaney, Sean and Dalton, Jeff}, title = {Revisiting Text Ranking in Deep Research}, booktitle = {49th International ACM SIGIR Conference on Research and Development in Information Retrieval}, year = {2026}, url = {https://arxiv.org/abs/2602.21456}, doi = {10.1145/3805712.3808557} }