用结构化证据传输评估放射科报告生成质量，更懂临床错误类型。
RadOT-Eval: Auditable Structured-Evidence Transport for Radiology Report Evaluation
- 将报告拆解为临床证据单元，用最优传输对齐并检测偏差
- 在独立数据集上与人工标注误差相关性达0.715，优于传统方法
- 适合医疗文本生成评估，尤其关注遗漏、幻觉等高风险错误
自动评估对高风险文本生成至关重要，因错误常表现为遗漏发现、虚构内容、极性反转、位置变更、不确定性错配及时间对比错误，而不仅仅是表面相似度低。放射科报告生成是严峻挑战，因生成报告需保留跨源的结构化临床证据。我们提出RadOT-Eval，一种可审计的结构化证据最优传输框架，用于离线审计放射科报告生成。RadOT-Eval将参考报告与候选报告分解为属性-结构化的临床证据单元，利用熵正则最优传输对齐对应证据，并通过单调风险模型结合临床有意义的侧通道差异预测错误负担。所有运输、特征和读出选择均基于ReXVal数据集选定，冻结系统在独立的RadEvalX数据集上评估。RadOT-Eval在总错误负担、临床显著错误负担和临床不显著错误负担上的斯皮尔曼相关系数分别为0.715、0.548和0.399，点估计高于标准评估指标及开源大模型评估器GREEN-radllama2-7B。在ReXErr-v1的冻结辅助破坏敏感性压力测试中，获得0.768 AUROC和0.990的受损>清洁配对胜率。结果表明，结构化证据传输提供了一种可审计、以排序为导向的评估工具，适用于仅使用ReXVal进行模型选择、并在冻结状态下于RadEvalX上测试的高风险生成临床文本。
原文摘要 · Abstract (English)
Automatic evaluation is critical for high-stakes text generation, where errors often involve omitted findings, hallucinated content, polarity reversals, location changes, uncertainty mismatches, and temporal-comparison errors rather than low surface similarity alone. Radiology report generation provides a challenging test case because generated reports must preserve structured clinical evidence across sources. We present RadOT-Eval, an interpretable structured-evidence optimal transport framework for offline auditing of radiology report generation. RadOT-Eval decomposes reference and candidate reports into attribute-structured clinical evidence units, aligns corresponding evidence using entropy-regularized optimal transport, and uses clinically meaningful side-channel discrepancies in a monotone risk model to predict error burden. All transport, feature, and readout choices are selected using the ReXVal dataset, and the frozen system is evaluated on the independent RadEvalX dataset. RadOT-Eval achieves Spearman correlations of 0.715, 0.548, and 0.399 with total, clinically significant, and clinically insignificant annotated error burden, respectively, yielding higher point estimates than standard evaluation metrics and the open-source large language model (LLM)-based evaluator GREEN-radllama2-7B. In a frozen auxiliary corruption-sensitivity stress test on ReXErr-v1, RadOT-Eval achieves 0.768 AUROC and a 0.990 corrupted-greater-than-clean paired win rate. These results show that structured evidence transport provides an auditable, rank-oriented evaluation tool for high-stakes generated clinical text under ReXVal-only model selection and frozen RadEvalX testing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。