针对电信领域低资源问答,通过优化检索器提升生成效果。
ARMOR: Adaptive Retriever Optimization for Low-Resource Telecom Question Answering
- 用查询编码器自适应优化检索器,避免生成器过拟合。
- 在电信领域多个基准上,检索与生成性能均显著提升。
- 适合资源有限但需高精度技术问答的场景。
电信问答(QA)是检索增强生成(RAG)中的挑战性任务:证据分散在标准文档、论文、百科和网页中,答案常依赖技术表格、公式和专业协议语言。在低资源子领域,微调生成器易过度专业化并损害通用能力,因此查询端检索器适配成为更优选择。我们探讨固定生成器下,查询适配的RAG系统能否优于生成器侧适配,并识别最佳检索目标。基于容量对比分析,在参数受限和软检索假设下,当有效维度较小时,查询编码器微调的估计误差小于监督微调。我们提出两个关键目标:潜在文档RAG似然(优化生成效用)和InfoNCE对比目标(改善语义检索结构),并通过联合优化实现。我们提出ARMOR(自适应正则化混合优化器),学习独立温度控制检索分布与InfoNCE softmax,并正则化适配后的查询编码器以贴近预训练基线。在多个电信领域专用检索与生成问答基准上,ARMOR在域内多种设置下均提升了证据检索与答案生成表现。代码已开源:https://github.com/heshandevaka/ARMOR.git。
原文摘要 · Abstract (English)
Telecom question answering (QA) is a challenging setting for retrieval-augmented generation (RAG): evidence is fragmented across standards, papers, encyclopedic resources, and web documents, and answers often hinge on technical tables, equations, and specialized protocol language. In low-resource subdomains, generator fine-tuning can over-specialize and degrade general capability, making query-side retriever adaptation an attractive alternative. To this end, we ask whether a fixed-generator, query-adapted RAG system can outperform generator-side adaptation, and which retriever objectives best support that setting. We motivate retrieval, rather than generator fine-tuning, as the adaptation target through a capacity comparison: under bounded-parameter and soft-retrieval assumptions, query-encoder tuning can have a smaller estimation term than supervised fine-tuning when its effective dimension is smaller. We identify two particularly relevant objectives -- the latent-document RAG likelihood, which optimizes generation utility, and the InfoNCE contrastive objective, which improves semantic retrieval geometry -- and leverage them jointly through a retriever optimization method targeting downstream QA performance in the telecom domain. Specifically, we introduce ARMOR, Adaptive Regularized Mixture Optimization for Retrievers, which learns separate temperatures for the RAG retrieval distribution and InfoNCE softmax and regularizes the adapted query encoder toward the frozen base query encoder. Across telecom-specific retrieval and generative QA benchmarks, we show that ARMOR improves evidence retrieval and answer generation in several in-domain settings. Code is available at https://github.com/heshandevaka/ARMOR.git.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。