用混合检索与重排,从社交媒体隐含提及中精准找到相关论文。
Deep Retrieval at CheckThat! 2025: Identifying Scientific Papers from Implicit Social Media Mentions via Hybrid Retrieval and Re-Ranking
- 结合关键词匹配与语义向量搜索,融合多模态信息。
- 在开发集上达76.46% MRR@5,测试集66.43%,排名前三。
- 仅用开源模型本地运行,无需外部训练数据,高效可复现。
我们介绍了Deep Retrieval团队在CLEF CheckThat! 2025竞赛子任务4b中的方法与结果,该任务旨在为给定的社交媒体帖子检索相关科学文献。为此,我们提出一种混合检索流水线,结合词法精确性、语义泛化和深度上下文重排,以应对非正式语言到正式学术语言的鸿沟。具体而言,我们采用基于BM25的关键词匹配与使用微调后的INF-Retriever-v1模型构建的FAISS向量库进行密集语义检索,前者返回前30个候选,后者生成100个候选,随后通过大型语言模型(LLM)交叉编码器对合并结果进行重排。该方法在开发集上取得76.46%的MRR@5,在隐藏测试集上达到66.43%,在开发集排行榜上位列第一,在测试集排行榜上位列第三(共31支队伍),与第一名相对差距仅2个百分点。性能优势源于本地运行开源模型且不依赖外部训练数据,凸显了精心设计与微调检索流水线的有效性。
原文摘要 · Abstract (English)
We present the methodology and results of the Deep Retrieval team for subtask 4b of the CLEF CheckThat! 2025 competition, which focuses on retrieving relevant scientific literature for given social media posts. To address this task, we propose a hybrid retrieval pipeline that combines lexical precision, semantic generalization, and deep contextual re-ranking, enabling robust retrieval that bridges the informal-to-formal language gap. Specifically, we combine BM25-based keyword matching with a FAISS vector store using a fine-tuned INF-Retriever-v1 model for dense semantic retrieval. BM25 returns the top 30 candidates, and semantic search yields 100 candidates, which are then merged and re-ranked via a large language model (LLM)-based cross-encoder. Our approach achieves a mean reciprocal rank at 5 (MRR@5) of 76.46% on the development set and 66.43% on the hidden test set, securing the 1st position on the development leaderboard and ranking 3rd on the test leaderboard (out of 31 teams), with a relative performance gap of only 2 percentage points compared to the top-ranked system. We achieve this strong performance by running open-source models locally and without external training data, highlighting the effectiveness of a carefully designed and fine-tuned retrieval pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。