arXiv:2606.29863cs.CL2026-06被引 1

让AI更懂何时该信自己、何时该查资料,减少胡说八道。

KbSD: Knowledge Boundary aware Self-Distillation for Behavioral Calibration in Agentic Search

论文配图:KbSD: Knowledge Boundary aware Self-Distillation for Behavioral Calibration in Agentic Search
图 1 · 摘自论文原文
  • 用教师模型生成带边界提示的推理示范,实现细粒度监督
  • 在难判别的知识边界区域,准确率提升12.3%,幻觉减少28%
  • 适合需要可靠决策的智能搜索系统研发者

代理式搜索赋予大语言模型动态检索能力,但现有强化学习方法在知识边界校准上受限于奖励稀疏性——即难以判断何时信任参数记忆、何时依赖检索证据、何时应放弃回答。二元奖励虽能惩罚错误结果,却无法提供跨不同知识状态作出校准决策所需的推理指导。为此,我们提出KbSD(知识边界感知自蒸馏)框架,通过密集的令牌级监督、结果级稀疏奖励以及四象限自适应优化来解决此问题。KbSD构建了一个与学生模型结构相同的提示增强型教师模型,接收参数确定性、检索质量及真实答案等明确的知识边界信号,生成校准后的推理示范。这种信息不对称的自蒸馏机制在无需更大外部模型的前提下实现了密集监督。为进一步应对不同知识状态下推理分布的异质性,引入四象限自适应蒸馏目标:对集中整合采用反向KL,对多样化拒绝采用正向KL,对需兼顾精度与覆盖的非对称象限则采用帕累托最优双向KL。多基准测试结果表明,KbSD在任务准确率和幻觉抑制方面均持续优于强基线,尤其在稀疏奖励最不具信息量的挑战性象限中表现最佳。

原文摘要 · Abstract (English)

Agentic search equips large language models with dynamic retrieval abilities, but existing reinforcement learning methods remain limited by reward sparsity in knowledge boundary calibration -- deciding when to trust parametric memory, when to rely on retrieved evidence, and when to abstain. Binary rewards can penalize undesirable outcomes, but provide little guidance on the reasoning process required to make calibrated decisions across different knowledge states. To address this, we propose KbSD (Knowledge boundary Self-Distillation), a framework that tackles this limitation through dense token-level supervision, outcome-level sparse rewards, and quadrant-adaptive optimization. KbSD constructs a hint-augmented teacher, architecturally identical to the student, that receives explicit knowledge boundary signals -- including parametric certainty, retrieval quality, and ground-truth answers -- to generate calibrated reasoning demonstrations. This information-asymmetric self-distillation enables dense supervision without requiring a larger external model. To further account for the heterogeneous reasoning distributions across knowledge states, we introduce a quadrant-adaptive distillation objective: reverse KL for concentrated integration, forward KL for diverse refusal, and Pareto-optimal bidirectional KL for asymmetric quadrants requiring both precision and coverage. Experiments on multiple benchmarks show that KbSD consistently improves both task accuracy and hallucination mitigation over strong baselines, with the largest gains appearing in the challenging quadrants where sparse rewards are least informative.

知识边界自蒸馏幻觉抑制智能搜索

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。