link bibtex long conference paper
Appeared in: Proceedings of the 2026 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval (ICTIR 2026)
Abstract:
Retrieval axioms are formalized constraints that capture aspects of document relevance typically implemented as pairwise document preferences. Such axiomatic preferences serve diverse downstream tasks like explainability and model diagnostics. However, almost all axioms are based on lexical matching heuristics, which do not allow for term-to-term interactions. For example, the REG~axiom that states to prefer documents that cover more query aspects only uses the number of occurrences of query terms in a document. This limitation to simplistic heuristics makes it difficult to align neural models with axioms because the axioms can not capture soft term-matching. We address this shortcoming with large language models~(LLMs) to relax the constraints of the existing retrieval axioms: \Ni replacing lexical matching with LLM-based ``soft'' term matching, and \Nii using informal formulations of the axioms' intuitions as prompts for an instruction-tuned neural ranker. We find that the new soft matching-based axiom variants better capture relevance than the traditional lexical ones, improving by up to~35\%. Furthermore, they reduce the number of failed comparisons (i.e., those without a preference) by up to~86\%, thereby enabling more detailed axiomatic analyses. Finally, we apply the new axiom variants to diagnose several state-of-the-art ranking architectures. We find that our new axioms allow us to re-validate prior axiomatic findings and measure behaviors that were previously only quantified using other (non-axiomatic) interpretability methods.
BibTeX @inproceedings{parry:ictir2026-aixollm, author = {Parry, Andrew and Fröbe, Maik and Potthast, Martin and MacAvaney, Sean and Stein, Benno and Hagen, Matthias and Ganguly, Debasis}, title = {Relaxed Term Matching for Neural Axiomatic Diagnostics}, booktitle = {Proceedings of the 2026 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval}, year = {2026}, doi = {10.1145/3805713.3820405}, pages = {274--284} }