让摘要生成有可靠保障，关键信息不遗漏。
Document Summarization with Conformal Importance Guarantees
- 用置信区间方法校准句子重要性阈值，保证关键内容覆盖
- 在多个数据集上实现理论承诺的覆盖率和召回率
- 无需修改模型，适配现有大模型，适合医疗法律等高风险场景
自动摘要系统随着大语言模型迅速发展，但在医疗、法律、金融等高风险领域仍缺乏关键内容包含的可靠保障。本文提出首个兼顾重要性保留与严格覆盖保证的摘要生成框架——共形重要性摘要（Conformal Importance Summarization）。该方法通过共形预测校准句子级重要性评分阈值，实现用户指定的覆盖率和召回率，且不依赖特定分布假设。方法具备模型无关性，仅需少量校准数据，可无缝集成现有黑盒LLM。在主流摘要基准上的实验表明，该方法实现了理论保证的信息覆盖水平。研究显示，该框架可与现有技术结合，推动可信可控的自动摘要在关键应用中的安全部署。代码已开源：https://github.com/layer6ai-labs/conformal-importance-summarization。
原文摘要 · Abstract (English)
Automatic summarization systems have advanced rapidly with large language models (LLMs), yet they still lack reliable guarantees on inclusion of critical content in high-stakes domains like healthcare, law, and finance. In this work, we introduce Conformal Importance Summarization, the first framework for importance-preserving summary generation which uses conformal prediction to provide rigorous, distribution-free coverage guarantees. By calibrating thresholds on sentence-level importance scores, we enable extractive document summarization with user-specified coverage and recall rates over critical content. Our method is model-agnostic, requires only a small calibration set, and seamlessly integrates with existing black-box LLMs. Experiments on established summarization benchmarks demonstrate that Conformal Importance Summarization achieves the theoretically assured information coverage rate. Our work suggests that Conformal Importance Summarization can be combined with existing techniques to achieve reliable, controllable automatic summarization, paving the way for safer deployment of AI summarization tools in critical applications. Code is available at https://github.com/layer6ai-labs/conformal-importance-summarization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。