← smac.pub home

Efficient Document Re-Ranking for Transformers by Precomputing Term Representations

pdf bibtex slides long conference paper

Authors: Sean MacAvaney, Franco Maria Nardini, Raffaele Perego, Nicola Tonellotto, Nazli Goharian, Ophir Frieder

Appeared in: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2020)

Links/IDs:
DOI 10.1145/3397271.3401093 DBLP conf/sigir/MacAvaneyN0TGF20b arXiv 2004.14255 Google Scholar 7wWfoDgAAAAJ:MXK_kJrjxJIC Semantic Scholar 0c3bdbad193ec8a5b1f4005dc1496e341a2025b4 smac.pub sigir2020-eff

Abstract:

Deep pretrained transformer networks are effective at various ranking tasks, such as question answering and ad-hoc document ranking. However, their computational expenses deem them cost-prohibitive in practice. Our proposed approach, called PreTTR (Precomputing Transformer Term Representations), considerably reduces the query-time latency of deep transformer networks (up to a 42x speedup on web document ranking) making these networks more practical to use in a real-time ranking scenario. Specifically, we precompute part of the document term representations at indexing time (without a query), and merge them with the query representation at query time to compute the final ranking score. Due to the large size of the token representations, we also propose an effective approach to reduce the storage requirement by training a compression layer to match attention scores. Our compression technique reduces the storage required up to 97.5%, and it can be applied without a substantial degradation in ranking performance.

BibTeX @inproceedings{macavaney:sigir2020-eff, author = {MacAvaney, Sean and Nardini, Franco Maria and Perego, Raffaele and Tonellotto, Nicola and Goharian, Nazli and Frieder, Ophir}, title = {Efficient Document Re-Ranking for Transformers by Precomputing Term Representations}, booktitle = {Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval}, year = {2020}, url = {https://arxiv.org/abs/2004.14255}, doi = {10.1145/3397271.3401093}, pages = {49--58} }