arXiv:2606.00683cs.CL2026-06

小模型也能精准答难题,专攻基于文本的忠实问答。

OCC-RAG: Optimal Cognitive Core for Faithful Question Answering

论文配图:OCC-RAG: Optimal Cognitive Core for Faithful Question Answering
图 1 · 摘自论文原文
  • 构建专注问答的小型语言模型,强调从原文推理而非记忆知识。
  • 三百万条合成数据训练,支持多跳推理与严格引用原文。
  • 模型体积仅是大模型的1/6,性能却能超越甚至媲美大模型。

语言模型的发展长期依赖规模扩张,但许多实际应用更需要稳健推理能力而非海量参数。为此,我们提出面向任务特化的紧凑语言模型(SLMs)——最优认知核心(OCC)家族。其中,OCC-RAG针对基于上下文的忠实问答设计,需在给定文本中进行多跳推理并忽略记忆知识。我们构建了大规模合成多上下文、多跳问答数据集,包含超三百万样本,聚焦多跳推理、严格上下文一致性及合理拒答。释放OCC-RAG-0.6B与OCC-RAG-1.7B两个模型,均在该数据集上中度训练。模型输出结构化推理链并附带源自原文的引用。实验表明,在多跳推理(HotpotQA、MuSiQue、TAT-QA)、忠实性(ConFiQA)和拒答能力(MuSiQue-Un)等任务上,该系列模型表现可比或优于2至6倍于自身规模的通用模型。

原文摘要 · Abstract (English)

Recent progress in the development of language models has been defined by scale, with each generation absorbing more of the world's knowledge into its weights. However, many practical applications benefit more from robust reasoning than from extensive parametric knowledge. In this setting, task-specialized small language models (SLMs) offer a principled design choice. We introduce Optimal Cognitive Core (OCC), a family of SLMs built around this premise. As a variant of OCC, we present OCC-RAG, optimized for faithful question answering (QA) grounded in the provided context. This task directly aligns with the OCC design approach, requiring multi-hop reasoning over supplied passages while ignoring memorized knowledge. To train OCC-RAG, we implement a novel pipeline for synthesizing multi-context, multi-hop QA data at scale, producing a corpus of over three million examples targeting multi-hop reasoning, strict context faithfulness, and calibrated abstention. We release OCC-RAG-0.6B and OCC-RAG-1.7B, both mid-trained on this corpus. The models produce structured reasoning traces with source citations grounded in literal quotes from the context. Through OCC-RAG, we demonstrate that compact, task-specialized SLMs can match or exceed general-purpose models 2 -- 6x their size across multi-hop reasoning (HotpotQA, MuSiQue, TAT-QA), faithfulness (ConFiQA), and refusal (MuSiQue-Un) benchmarks.

小模型问答系统忠实推理多跳推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。