← smac.pub home

IntenT5: Search Result Diversification using Causal Language Models

pdf bibtex 13 citations non-refereed

Authors: Sean MacAvaney, Craig Macdonald, Roderick Murray-Smith, Iadh Ounis

Appeared in: arXiv

DBLP journals/corr/abs-2108-04026 arXiv 2108.04026 Google Scholar 7wWfoDgAAAAJ:ZeXyd9-uunAC Semantic Scholar 53220193decd8615c255bd71bd63d44efafd5313 smac.pub arxiv2021-intent5


Search result diversification is a beneficial approach to overcome under-specified queries, such as those that are ambiguous or multi-faceted. Existing approaches often rely on massive query logs and interaction data to generate a variety of possible query intents, which then can be used to re-rank documents. However, relying on user interaction data is problematic because one first needs a massive user base to build a sufficient log; public query logs are insufficient on their own. Given the recent success of causal language models (such as the Text-To-Text Transformer (T5) model) at text generation tasks, we explore the capacity of these models to generate potential query intents. We find that to encourage diversity in the generated queries, it is beneficial to adapt the model by including a new Distributional Causal Language Modeling (DCLM) objective during fine-tuning and a representation replacement during inference. Across six standard evaluation benchmarks, we find that our method (which we call IntenT5) improves search result diversity and attains (and sometimes exceeds) the diversity obtained when using query suggestions based on a proprietary query log. Our analysis shows that our approach is most effective for multi-faceted queries and is able to generalize effectively to queries that were unseen in training data.

BibTeX @article{macavaney:arxiv2021-intent5, author = {MacAvaney, Sean and Macdonald, Craig and Murray-Smith, Roderick and Ounis, Iadh}, title = {IntenT5: Search Result Diversification using Causal Language Models}, year = {2021}, url = {https://arxiv.org/abs/2108.04026}, journal = {arXiv}, volume = {abs/2108.04026} }