用小型模型探针替代大模型提示，更准更稳地过滤多源检索元数据。
Probe, Don't Prompt: A Hidden-State Probe for Metadata Filtering in Multi-Meta-RAG
- 用小模型隐藏状态探针代替GPT提取元数据，避免漂移风险。
- 在2556个查询上达到90.9%的精确匹配率，显著优于GPT-3.5的80.9%。
- 仅需135M参数，推理成本低，适合部署在本地环境。
Multi-Meta-RAG通过提示GPT-3.5-turbo从查询中提取元数据（新闻来源）来改进多跳问答的检索效果。本文证明，该专有、自由格式的提取器可被一个基于小型开源语言模型隐藏状态训练的本地、确定性探针取代。在全部2556个MultiHop-RAG查询上，该探针达到90.9%的集合精确匹配率，优于无模型子串基线的88.0%和GPT-3.5的80.9%；这一优势完全来自对空查询的处理——GPT-3.5从不拒绝空查询，而探针能准确识别并过滤。由于探针输出空间严格限定于49个源的固定词表，不会偏离允许列表。三项设计使其有效：选择浅层表示、均值池化，以及针对长尾源的类别不平衡多标签训练。135M参数模型性能接近1.5B模型，仅需前几层的部分前向传播加一个线性头，无需调用API。代码已开源：https://github.com/mxpoliakov/Multi-Meta-RAG。
原文摘要 · Abstract (English)
Multi-Meta-RAG improves retrieval for multi-hop question answering by filtering a vector store on metadata (the news source) that it extracts from each query by prompting gpt-3.5-turbo. We show this proprietary, free-form extractor can be replaced by a local, deterministic probe trained on the hidden states of a small open-source language model. On all 2556 MultiHop-RAG queries the probe reaches 90.9% set-exact accuracy against 88.0% for a model-free substring baseline and 80.9% for GPT-3.5, a margin that comes entirely from null queries, on which GPT-3.5 never abstains; on non-null queries all three stay within about a point. Because the probe's output space is exactly the fixed 49-source vocabulary, it cannot drift outside the allow-list as the prompted model does. Three design choices make it work: selecting a shallow layer, mean pooling, and class-imbalance-aware multi-label training over the long tail of sources. A 135M-parameter model lands within ~1.5 points of a 1.5B one, so the filter is cheap to output: a partial forward pass through the first few layers plus one linear head, with no API. The code is available at https://github.com/mxpoliakov/Multi-Meta-RAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。