用语义相似度提升事实核查的断句质量，还证明了修复流程的收敛性。
CREDENCE: Claim Reduction for Decomposition & Enhanced Credibility -- Semantic Metrics and Convergence Analysis
- 用BGE-large语义相似度替代传统重叠率，避免对同义表达的误判。
- 在多个数据集上，新指标比旧指标高15-32个百分点，修复后原子性错误减少47%-100%。
- 适合研究自动断句、事实核查或模型可解释性的研究人员使用。
将复合句分解为可验证的原子陈述是可靠自动化事实核查的前提。以往工作依赖于词元重叠（如Jaccard）度量，会系统性低估同义表达的分解质量，且缺乏对修复循环的正式终止分析。本文提出Credence框架，解决上述问题：(1) 引入语义-F1，采用BGE-large余弦相似度作为保真度度量，克服Jaccard对同义句的惩罚，提升下游事实核查准确率；(2) 建立收敛定理，形式化证明规则修复流程在理想解析器假设下单调且有限终止；基于LLM的自修复为非单调，需设置早停机制；(3) 构建涵盖社交媒体、百科和新闻领域的三个评估基准，用于跨域泛化能力测试；(4) 在四个规模3.8B-12B的分解模型及一个闭源API模型上进行多模型对比。实验在SocialClaimSplit、WikiSplitBench和ClaimDecompBench上显示，语义-F1较Jaccard-F1提升15-32个百分点；在SocialClaimSplit和WikiSplitBench上EPR达0.94至1.00，而ClaimDecompBench因新闻文本更复杂，基础EPR低至0.824；规则修复使原子性违规率（AVR）相对基线降低47%-100%，且不损害保真度。
原文摘要 · Abstract (English)
Decomposing compound sentences into atomic, verifiable claims is a prerequisite for reliable automated fact-checking. Prior work has relied on token-overlap (Jaccard) metrics that systematically underestimate decomposition quality for paraphrastic claims, and has lacked formal termination analysis for the repair loop. We present Credence, a revised claim decomposition and evaluation framework addressing both shortcomings. Our contributions are: (1) Semantic-F1: we use BGE-large cosine similarity fidelity metric that resolves Jaccard's penalisation and improves downstream fact-checking accuracy; (2) Convergence theorems: we formally characterise four properties of the repair pipeline, establishing that rule-based repair is monotone and finitely terminating under an oracle parser assumption; LLM-based self-repair is provably non-monotone and requires an early-exit guard; (3) Three evaluation benchmarks spanning social-media, encyclopaedic, and news domains for cross-domain generalisation measurement; (4) Multi-model benchmarking across four decomposer models (3.8B-12B) and a closed API model. Experiments on SocialClaimSplit, WikiSplitBench, and ClaimDecompBench show that Semantic-F1 outperforms Jaccard-F1 by +15-32pp. EPR ranges from 0.94 to 1.00 on SocialClaimSplit and WikiSplitBench, while ClaimDecompBench includes lower base EPR cases (down to 0.824) due to harder news-domain constructions, and rule-repair reduces the Atomicity Violation Rate (AVR) by 47-100% relative to the base model without degrading fidelity.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。