arXiv:2602.11908cs.AIcs.CL2026-02被引 1

让大模型在不确定时自动简化细节,提升长文本生成的可靠性。

When Should LLMs Be Less Specific? Selective Abstraction for Reliable Long-Form Text Generation

  • 将回答拆成独立事实单元,对不确定部分用更笼统但可信的表述替代。
  • 在多个数据集上测试,比直接删除错误内容提升27.73%的可靠性与信息保留率。
  • 适合需要高准确率的长文本生成场景,如医疗、法律等高风险领域。

大语言模型虽广泛应用,但仍易产生事实性错误,削弱用户信任,限制其在高风险场景的应用。现有方法通过置信度低时完全回避来降低风险,但在长文本生成中过于严苛,常丢弃有用信息。本文提出选择性抽象(Selective Abstraction, SA)框架,允许模型在不确定时主动降低细节程度以换取可靠性。我们从选择性风险与覆盖率角度形式化该框架,并提出原子级选择性抽象(Atom-wise Selective Abstraction),将回答分解为原子事实(atomic claims),对不确定项替换为更高置信度但更模糊的抽象表达。为评估该框架,我们构建端到端生成评估管道,以事实正确性衡量风险,以信息论指标衡量信息保留量。在六个开源模型上,基于FactScore和LongFact-Objects基准的实验表明,原子级SA显著优于现有基线,使风险-覆盖率曲线下面积(AURC)提升最高达27.73%,证明适度减少细节可同时提升准确率与可靠性,并保留原意核心。

原文摘要 · Abstract (English)

LLMs are widely used, yet they remain prone to factual errors that erode user trust and limit adoption in high-risk settings. One approach to mitigate this risk is to equip models with uncertainty estimation mechanisms that abstain when confidence is low. However, this binary "all-or-nothing" approach is excessively restrictive in long-form settings, often discarding valuable information. We introduce Selective Abstraction (SA), a framework that enables LLMs to trade specificity for reliability by selectively reducing the detail of uncertain content. We first formalize SA through the lenses of selective risk and coverage. We then propose Atom-wise Selective Abstraction, a claim-level instantiation that decomposes responses into atomic claims (short, self-contained statements each expressing a single fact) and replaces uncertain atoms with higher confidence, less specific abstractions. To evaluate this framework, we develop a novel end-to-end pipeline for open-ended generation that instantiates risk as factual correctness and measures coverage using an information-theoretic measure of retained information. Across six open-source models on the FactScore and LongFact-Objects benchmarks, atom-wise SA consistently outperforms existing baselines, improving the area under the risk-coverage curve (AURC) by up to 27.73% over claim removal, demonstrating that reducing specificity can boost accuracy and reliability while preserving most of their original meaning.

大模型可靠性长文本生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。