用新模型提前发现生成文本中的事实矛盾,提升大模型输出可靠性。
PrefixNLI: Detecting Factual Inconsistencies as Soon as They Arise
- 针对生成过程中的每段前缀进行事实一致性判断,改进传统NLI方法
- 在前缀级别上比基线模型高5-14个F1点,显著提升检测精度
- 适合需要高可信度生成的场景,如摘要、对话系统等
自然语言推理(NLI)模型常被用于提升大语言模型输出的事实性。通常通过NLI模型判断生成内容是否由给定证据蕴含,并触发纠正动作,如解码时重排或训练时强化学习奖励。然而,标准NLI模型针对完整句子训练,而自回归生成中决策基于逐步演化的文本前缀。为此,本文将蕴含检测任务推广至任意文本前缀,提出其在提升生成忠实性方面的价值。构建了相应评估与训练数据集,训练出专用模型MiniTruePrefixes,该模型在前缀级蕴含检测上相比基线模型提升5-14 F1点。进一步实验表明,将其集成到受控解码框架中可显著提升抽象摘要的事实一致性。使用MiniTruePrefixes引导后,LLaMA-3.2-3B-Instruct在忠实性和运行时间上达到同家族8B模型水平,仅需一半内存。
原文摘要 · Abstract (English)
Natural Language Inference (NLI) models have been used in various ways to improve the factuality of LLM outputs. This is typically done by applying an NLI model to judge whether the model output is entailed from the supposed evidence, triggering some corrective actions, such as beam reranking at inference time or RL rewards during training. While NLI models are trained to detect factual inconsistencies over complete sentences, decisions in the common autoregressive generation architecture are made for each evolving text prefix, during decoding. Addressing this setting, we generalize the entailment detection task to apply over arbitrary text prefixes, and suggest its utility for improving generation faithfulness. Providing suitable evaluation and training datasets for this task, we train MiniTruePrefixes, a novel specialized model that better detects factual inconsistencies over text prefixes, outperforming comparable baseline NLI models by 5-14 F1 points in prefix-level entailment. We further demonstrate that integrating MiniTruePrefixes into a controlled decoding framework substantially improves factual consistency in abstractive summarization. When guided by MiniTruePrefixes, LLaMA-3.2-3B-Instruct matches the faithfulness and runtime of the 8B model from the same model family, while using only half the memory.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。