为医疗摘要生成添加可校准的安全层,自动标记遗漏与虚构内容。
CARE: A Conformal Safety Layer for Medical Summarization

- 基于共形风险控制,不重训练即可为任意大模型输出加安全标签。
- 在100次分割测试中,95%情况下满足15%风险上限,仅需约100标注样本。
- 临床医生实验显示,漏检率平均降低28.6个百分点,可调风险与审查负担平衡。
大型语言模型(LLM)在医疗摘要生成中应用日益广泛,但其输出可能遗漏重要医学信息或引入无依据陈述。现有错误检测方法生成启发式或未校准的评分,无法对遗漏错误提供形式化控制,也缺乏在安全性与临床医生审查负担之间权衡的合理机制。我们提出共形风险评估(CARE),一种无需重训练的后处理、模型无关安全层,利用共形风险控制,在任何LLM生成的摘要上叠加校准后的遗漏与幻觉标记。CARE通过两个控制器提供有限样本、分布无关的保证:幻觉控制器限制文档中存在未标记幻觉句子的概率;遗漏控制器限制未被揭示的重要遗漏的期望比例。与幻觉检测不同,遗漏取决于源句是否重要且是否被摘要覆盖。我们发现,仅校准单一维度会违反目标风险边界,而边缘分解虽有效但过于保守。通过联合校准完整(τ,γ)阈值空间,CARE在保持形式保证的同时,比其他校准基线减少最多5倍的待审句子。在五个医疗摘要任务中,使用每领域约100个标注文档,CARE在α=0.15下以95%置信度满足目标风险边界,经100次校准/测试分割验证。初步临床医生研究(75份文档评审)表明,校准标记使遗漏检测平均提升28.6个百分点。结果表明,针对LLM辅助医疗摘要的句子级安全保证是可行的,并提供可调节的风险与审查成本平衡机制。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used for medical summarization, but their outputs can omit medically important information and introduce unsupported claims. Existing error-detection methods produce heuristic or uncalibrated scores, providing no formal control over missed errors and no principled way to trade off safety against clinician review burden. We introduce Conformal Assessment for Risk Evaluation (CARE), a post-hoc, model-agnostic safety layer that uses conformal risk control to overlay calibrated omission and hallucination flags onto summaries from any LLM without retraining. CARE provides finite-sample, distribution-free guarantees through two controllers: a hallucination controller that bounds the probability of a document containing any unflagged hallucinated sentence, and an omission controller that bounds the expected fraction of important omissions not surfaced for review. Unlike hallucination detection, omissions depend jointly on whether a source sentence is important and whether it is covered by the summary. We show that calibrating only one dimension can violate the target risk bound, while marginal decompositions remain valid but overly conservative. By jointly calibrating over the full $(τ,γ)$ threshold space, CARE preserves formal guarantees while surfacing up to 5$\times$ fewer sentences than alternative calibrated baselines. Across five medical summarization tasks, CARE satisfies the target risk bound at $α= 0.15$ with 95% confidence across 100 calibration/test resplits, using only ~100 labeled documents per domain. In a preliminary clinician study (75 document reviews), calibrated flags improved omission detection by 28.6 percentage points on average. These results show that sentence-level safety guarantees are feasible for LLM-assisted medical summarization and offer a tunable mechanism for balancing residual risk and review effort.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。