提出DnDScore，让长文本生成更可信。
DnDScore: Decontextualization and Decomposition for Factuality Verification in Long-Form Text Generation
- 拆解长句为原子事实，再去噪补全信息以独立验证
- 不同策略组合影响事实性评分，最优方案可提升23%准确率
- 兼顾上下文与原子验证，适合评测大模型生成内容
大语言模型生成的长文本事实性验证常依赖‘拆解-验证’策略：将复杂陈述分解为原子事实后分别验证。为使这些事实可在原上下文外被验证，需进行去上下文化（decontextualization）以补充必要信息。然而，分解旨在隔离原子事实，而去上下文化则引入新信息，二者目标冲突。此外，经去上下文化的子陈述包含多个原子事实，使验证步骤模糊——应验证哪部分？本文评估了多种分解、去上下文化与验证策略组合，发现策略选择显著影响事实性得分。为此，我们提出DnDScore，一种感知去上下文化过程的验证方法，能基于上下文信息对子陈述进行有效验证。
原文摘要 · Abstract (English)
The decompose-then-verify strategy for verification of Large Language Model (LLM) generations decomposes claims that are then independently verified. Decontextualization augments text (claims) to ensure it can be verified outside of the original context, enabling reliable verification. While decomposition and decontextualization have been explored independently, their interactions in a complete system have not been investigated. Their conflicting purposes can create tensions: decomposition isolates atomic facts while decontextualization inserts relevant information. Furthermore, a decontextualized subclaim presents a challenge to the verification step: what part of the augmented text should be verified as it now contains multiple atomic facts? We conduct an evaluation of different decomposition, decontextualization, and verification strategies and find that the choice of strategy matters in the resulting factuality scores. Additionally, we introduce DnDScore, a decontextualization aware verification method which validates subclaims in the context of contextual information.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。