Traditional information retrieval is based on sparse bag-of-words vector representations of documents and queries. More recent deep-learning approaches have used dense embeddings learned using a transformer-based large language model. We show that on a classic benchmark on scientific document retrieval in the medical domain of cystic fibrosis, that both of these models perform roughly equivalently. Notably, dense vectors from the state-of-the-art SPECTER2 model do not significantly enhance performance. However, a hybrid model that we propose combining these methods yields significantly better results, underscoring the merits of integrating classical and contemporary deep learning techniques in information retrieval in the domain of specialized scientific documents.

PDF, Arxiv

The 4th Workshop on Scientific Document Understanding, AAAI (2024).

@article{mandikal:aaai-sdu23, title={Sparse Meets Dense: A Hybrid Approach to Enhance Scientific Document Retrieval}, author={Priyanka Mandikal and Raymond Mooney}, booktitle={The 4th Workshop on Scientific Document Understanding, AAAI}, month={February}, url="http://www.cs.utexas.edu/users/ai-labpub-view.php?PubID=128046", year={2024} }

Slides (PPT)