通过风险感知的注意力机制，减少大模型生成中的幻觉问题。
PRISM: Probability Reallocation with In-Span Masking for Knowledge-Sensitive Alignment
- 在关键句段引入事实风险标记，动态调整训练时的预测概率分布。
- 在多个事实敏感数据集上，事实性指标显著提升，且通用能力保持稳定。
- 适合需要高可信度生成的场景，如医疗、法律等专业领域应用。
监督微调（SFT）使用逐标记硬标签会放大对无事实支持目标的过度自信模仿，导致多句生成中幻觉传播。我们研究一种增强型SFT设置，其中训练样本包含粗粒度的句子级事实风险标签和句间依赖标注，提供关于事实支撑薄弱位置的结构化信号。我们提出 extbf{PRISM}，一种可微的风险门控框架，仅在事实关键位置修改学习过程。PRISM在标准SFT基础上加入轻量级、模型感知的概率重分配目标，惩罚高置信度的高风险目标标记，其作用范围由跨度级风险权重和模型感知门控控制。在幻觉敏感的事实基准和通用评估中，PRISM在多种骨干模型上均提升了事实性表现，同时保持了良好的整体性能。消融实验表明，辅助信号在保守使用时最有效，知识掩码与模型感知重分配在平衡事实修正与能力保留方面起互补作用。
原文摘要 · Abstract (English)
Supervised fine-tuning (SFT) with token-level hard labels can amplify overconfident imitation of factually unsupported targets, causing hallucinations that propagate in multi-sentence generation. We study an augmented SFT setting in which training instances include coarse sentence-level factuality risk labels and inter-sentence dependency annotations, providing structured signals about where factual commitments are weakly supported. We propose \textbf{PRISM}, a differentiable risk-gated framework that modifies learning only at fact-critical positions. PRISM augments standard SFT with a lightweight, model-aware probability reallocation objective that penalizes high-confidence predictions on risky target tokens, with its scope controlled by span-level risk weights and model-aware gating. Experiments on hallucination-sensitive factual benchmarks and general evaluations show that PRISM improves factual aggregates across backbones while maintaining a competitive overall capability profile. Ablations further show that the auxiliary signal is most effective when used conservatively, and that knowledge masking and model-aware reallocation play complementary roles in balancing factual correction and capability preservation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。