arXiv:2605.03103cs.CLcs.AI2026-05中稿 · KSEM 2026被引 1

构建医疗报告结构化提取基准,评估未知字段与识别噪声下的模型性能。

MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports

论文配图:MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
图 1 · 摘自论文原文
  • 设计新基准,模拟真实医疗文本中未知字段和光学字符错误。
  • 小模型在关键条件问答任务中表现更优,大模型整体效果更强。
  • 适合研究医疗信息抽取、临床数据处理的开发者参考使用。

从光学字符识别(OCR)生成的临床报告中进行半结构化信息抽取,对高效重建患者长期病史至关重要。实际场景常涉及三项任务:(i) 字段标题(键)发现,(ii) 关键词条件问答(QA),(iii) 端到端键值对抽取。然而现有评估往往忽略两个关键因素:键的异构性与不完全已知性,以及OCR引入的噪声,导致难以评估模型在真实场景下的鲁棒性。本文提出MedStruct-S基准,专为在未知键和OCR噪声条件下评估上述任务而设计。该基准包含3,582页真实临床报告,涵盖4个编码器-仅序列标注模型和5个解码器-仅结构化生成模型,参数规模从0.11B至103B不等。结果表明,尽管体积较小,编码器-仅模型在非空值关键词问答任务中表现最佳;在相近规模下,编码器-仅模型整体表现仍更优。未控制模型规模时,微调后的解码器-仅模型取得最强综合性能。这说明该基准能可靠支持不同半结构化信息抽取场景中的模型选型与比较。

原文摘要 · Abstract (English)

Semi-structured information extraction (IE) from OCR-derived clinical reports is crucial for efficiently reconstructing patients' longitudinal medical histories. In practice, this scenario commonly involves three tasks: (i) field-header (key) discovery, (ii) key-conditioned question answering (QA), and (iii) end-to-end key-value pair extraction. However, existing evaluations often under-model two factors: heterogeneous and incompletely known key representations, and OCR-induced noise. This makes it difficult to assess model robustness in real-world settings. We present MedStruct-S, a benchmark specifically designed to evaluate these tasks under unknown keys and OCR noise. MedStruct-S contains 3,582 annotated real-world clinical report pages. Using MedStruct-S, we benchmark two representative paradigms: encoder-only sequence labeling with post-processing and decoder-only structured generation, covering four encoder-only and five decoder-only models spanning 0.11B to 103B parameters. Our results show that encoder-only models achieve the best performance for non-null-value key-conditioned QA despite being substantially smaller than decoder-only models. When comparing models of similar order of magnitude, encoder-only models still perform better overall. Without controlling for model scale, fine-tuned decoder-only models deliver the strongest overall results. These findings show that the benchmark provides a reliable and practical basis for selecting and comparing models across different semi-structured IE settings.

信息抽取医疗文本基准测试

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。