用知识图谱一致性奖励,让肺部X光模型更准确、可验证地推理。
CheXPO-v2: Preference Optimization for Chest X-ray VLMs with Knowledge Graph Consistency
- 通过结构化三元组解析推理过程,实现对逻辑错误的细粒度惩罚
- 仅用5000样本即达到新最好性能,比传统方法更高效
- 适合医疗AI安全性和可解释性要求高的场景
医学视觉-语言模型易产生幻觉,影响临床可靠性。尽管强化学习方法如分组相对策略优化(GRPO)提供低成本对齐方案,但其依赖稀疏的结果奖励,无意中鼓励模型‘过度思考’——生成冗长、复杂且不可验证的思维链来支撑答案。这种对结果的关注掩盖了事实错误,带来重大安全风险。为此,我们提出CheXPO-v2,一种从结果监督转向过程监督的新对齐框架。核心创新是基于实体-关系匹配的知识图谱一致性奖励机制。通过将推理步骤显式解析为‘疾病-关系-解剖’三元组,实现原子级别的逻辑不一致与幻觉惩罚。结合硬样本挖掘策略,该方法在MIMIC-CXR-VQA等基准上显著优于GRPO及现有先进模型。关键的是,仅使用5000个样本即达到新最佳准确率,展现出卓越的数据效率,同时生成临床可信且可验证的推理。项目代码已公开:https://github.com/ecoxial2007/CheX-Phi4MM。
原文摘要 · Abstract (English)
Medical Vision-Language Models (VLMs) are prone to hallucinations, compromising clinical reliability. While reinforcement learning methods like Group Relative Policy Optimization (GRPO) offer a low-cost alignment solution, their reliance on sparse, outcome-based rewards inadvertently encourages models to "overthink" -- generating verbose, convoluted, and unverifiable Chain-of-Thought reasoning to justify answers. This focus on outcomes obscures factual errors and poses significant safety risks. To address this, we propose CheXPO-v2, a novel alignment framework that shifts from outcome to process supervision. Our core innovation is a Knowledge Graph Consistency Reward mechanism driven by Entity-Relation Matching. By explicitly parsing reasoning steps into structured "Disease, Relation, Anatomy" triplets, we provide fine-grained supervision that penalizes incoherent logic and hallucinations at the atomic level. Integrating this with a hard-example mining strategy, our approach significantly outperforms GRPO and state-of-the-art models on benchmarks like MIMIC-CXR-VQA. Crucially, CheXPO-v2 achieves new state-of-the-art accuracy using only 5k samples, demonstrating exceptional data efficiency while producing clinically sound and verifiable reasoning. The project source code is publicly available at: https://github.com/ecoxial2007/CheX-Phi4MM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。