用闭环验证修复框架，让医疗大模型输出符合标准编码规范。
Closed-Loop Validation-Repair for Healthcare Interoperability: A Multi-Model Study of Schema Compliance in Clinical LLMs
- 设计闭环验证修复机制，自动检测并修正模型输出的格式错误。
- 修复后整体合规率提升至99.0%，误差多在1-2次迭代内解决。
- 适合关注医疗AI系统落地与电子病历对接的研究者和开发者。
医疗互操作性要求AI系统生成符合标准结构化数据，如用于诊断编码的ICD-10、用于手术计费的CPT以及用于数据交换的HL7 FHIR。尽管大语言模型具备临床推理能力，但其集成到电子健康记录系统时面临关键障碍：结构合规性不足。本研究通过本地部署评估了三个开源模型（Qwen2.5 7B、Llama 3.1 8B、Gemma2 9B），在涵盖十大学科领域的320个临床场景中生成960个模型-场景组合，在配对基线与验证修复条件下进行测试。结果显示，三类模型基线合规率在85.9%至91.6%之间，表明医疗训练语料存在共性缺陷而非模型特异性问题；96%的验证失败源于表示层面的格式错误，如替代缩写和代码前缀，说明模型遵循临床书写习惯但缺乏对医疗信息标准的认知；验证修复框架实现总体99.0%的合规率，各模型间为98.4%至99.4%，多数错误在一到两次迭代内修复。精确McNemar检验p值均低于0.001，绝对改进幅度达7.8至12.5个百分点，证实统计显著性。结果支持闭环验证修复作为保障医疗互操作性的有效系统级策略，显著提升下游临床系统集成的结构化准备度。
原文摘要 · Abstract (English)
Healthcare interoperability requires AI systems to produce structured outputs conforming to standardized schemas including ICD-10 for diagnostic coding, CPT for procedure billing, and HL7 FHIR for data exchange. While large language models demonstrate clinical reasoning capabilities, their integration into electronic health record systems faces a critical barrier: schema noncompliance. We evaluate three open-source models, Qwen2.5 7B, Llama 3.1 8B, and Gemma2 9B, via local deployment across 320 clinical scenarios spanning ten medical specialties, yielding 960 model-scenario pairs assessed under paired baseline and validation-repair conditions. First, schema noncompliance is consistent across the three model families, with baseline compliance rates ranging from 85.9 to 91.6 percent despite varying architectures and training data, suggesting shared gaps in medical training corpora rather than model-specific limitations. Second, 96 percent of validator-detected failures are representation-level format violations such as alternative medical abbreviations and code prefixes, indicating models follow clinical writing conventions but lack awareness of healthcare IT standards. Third, the validation-repair framework achieves 99.0 percent overall compliance, ranging from 98.4 to 99.4 percent across models, with most errors resolving within one or two iterations. Exact McNemar p-values below 0.001 and absolute improvements of 7.8 to 12.5 percentage points across model sizes confirm statistical significance. These results support closed-loop validation-repair as an effective system-level safeguard for healthcare interoperability, improving schema-level readiness for downstream clinical system integration.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。