新指标能精准识别放射报告中的临床矛盾，避免表面相似却含义相反的误判。
RadSEM: A Finding-by-Finding Metric for Clinical Consistency in Radiology Reports
- 将报告拆解为原子级发现句，按部位-发现关系结构化
- 通过矛盾约束匹配，对相反描述（如‘有积液’与‘无积液’）不给分
- 基于异常发现加权计算F1，提升对临床错误的敏感性，适合医学评估场景
放射报告评估需区分临床一致性与表面相似性，因否定、侧别或正常/异常极性反转会改变语义。我们提出RadSEM（放射学句子级评估指标），一种基于参考文本的约束型大模型辅助评估方法。该方法将参考与生成报告重写为有序的原子发现句，每句表达一个部位-发现命题。随后执行矛盾约束的多对多匹配：如‘积液’与‘无积液’等不兼容对不予评分，而可接受粒度差异的部分匹配。通过确定性阶段依据整体-部分及异常-细节关系加权配对，统计未匹配发现，最终生成以异常为中心的加权F1分数。大模型仅用于结构化重写与局部对齐，而非黑箱评判。在包含2,448份去标识化报告、扩展至五级渐进式污染的SSREE测试集上，RadSEM达到肯德尔tau_b为0.957，所有配对一致性97.8%，相邻一致性95.0%，严格五级排序达81.9%，优于专用于放射学及通用文本指标，且避免了极性反转报告因词法重叠而被误评的问题。在599组同义/反义三元组中，81.9%的报告实现严格五级排序，且对同义词偏好率达99.67%（597/599）。结果表明，显式发现单元、矛盾感知匹配和异常导向评分使评估更可解释、更敏感于临床错误。代码开源：https://github.com/jdh-algo/RadSEM。
原文摘要 · Abstract (English)
Radiology report evaluation must distinguish clinical compatibility from surface similarity, because negation, laterality, or normal-abnormal polarity can reverse a finding. We propose RadSEM (Radiology Sentence-Level Evaluation Metric), a constrained LLM-assisted metric for reference-based evaluation of radiology Findings. RadSEM rewrites reference and generated reports into ordered atomic finding sentences, each expressing one site-finding proposition. It then performs contradiction-constrained many-to-many matching: incompatible pairs such as "effusion" and "no effusion" receive no credit, while compatible granularity differences can receive partial credit. A deterministic stage weights pairs by part-whole and abnormal-detail relationships, counts unmatched findings, and produces an abnormal-focused weighted F1 score. Thus, the LLM supports structured rewriting and local alignment rather than acting as an opaque judge. We evaluate RadSEM with SSREE, a controlled monotonicity stress test built from 2,448 de-identified reports expanded into five graded corruption levels. RadSEM achieves Kendall tau_b of 0.957, all-pairs concordance of 97.8%, adjacent concordance of 95.0%, and strict five-level ordering for 81.9% of reports, outperforming radiology-specific and general text metrics while avoiding the failure in which polarity-inverted reports regain lexical overlap. On the same SSREE set, RadSEM outperforms the Ref-anchored RadSEM-Alt policy, improving adjacent concordance from 90.7% to 95.0% and strict ordering from 67.2% to 81.9%. On a 599-triplet synonym/antonym subset, RadSEM prefers synonyms in 597 cases (99.67%). These results suggest that explicit finding units, contradiction-aware matching, and abnormal-focused deterministic scoring make report scoring more interpretable and sensitive to clinically meaningful errors. Code is available at https://github.com/jdh-algo/RadSEM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。