用可解释的原型网络生成真实医疗报告,防止大模型胡编乱造。
ProtoMedAgent: Multimodal Clinical Interpretability via Privacy-Aware Agentic Workflows

- 通过符号化瓶颈迭代优化,让视觉和表格特征变成离散语义记忆。
- 在4160名患者数据上,报告忠实度达91.2%,远超传统RAG的46.2%。
- 引入隐私保护机制,降低9.8%的成员推理风险,适合临床可信生成。
尽管可解释的原型网络为临床诊断提供了基于案例的推理能力,但其原始连续输出缺乏医疗记录所需的语义结构。通过标准检索增强生成(RAG)填补这一空白时,常出现‘检索奉承’现象,即大语言模型(LLM)为迎合视觉预测而虚构事后解释。我们提出ProtoMedAgent,将多模态临床报告生成建模为在严格神经符号瓶颈上的无梯度、测试时优化问题。在冻结的原型主干基础上,将隐含的视觉与表格特征提炼为离散语义记忆。在线生成过程受精确集合论差异与反思性‘记录员-批评家’循环严格约束,数学上杜绝未经支持的叙述性声明。为安全控制数据泄露,引入由k-匿名性和ℓ-多样性驱动的语义隐私门控。在4,160名患者的临床队列上评估,ProtoMedAgent实现91.2%的对比集忠实度,显著优于标准RAG的46.2%。此外,通过绑定的ℓ-多样性相变机制,将逐片段的成员推理风险系统性降低9.8个百分点。
原文摘要 · Abstract (English)
While interpretable prototype networks offer compelling case-based reasoning for clinical diagnostics, their raw continuous outputs lack the semantic structure required for medical documentation. Bridging this gap via standard Retrieval-Augmented Generation (RAG) routinely triggers ``retrieval sycophancy,'' where Large Language Models (LLMs) hallucinate post-hoc rationalizations to align with visual predictions. We introduce ProtoMedAgent, a framework that formalizes multimodal clinical reporting as an iterative, zero-gradient test-time optimization problem over a strict neuro-symbolic bottleneck. Operating on a frozen prototype backbone, we distill latent visual and tabular features into a discrete semantic memory. Online generation is strictly constrained by exact set-theoretic differentials and a reflective Scribe-Critic loop, mathematically precluding unsupported narrative claims. To safely bound data disclosure, we introduce a semantic privacy gate governed by $k$-anonymity and $\ell$-diversity. Evaluated on a 4,160-patient clinical cohort, ProtoMedAgent achieves 91.2% Comparison Set Faithfulness where it fundamentally outperforms standard RAG (46.2%). ProtoMedAgent additionally leverages a binding $\ell$-diversity phase transition to systematically reduce artifact-level membership inference risks by an absolute 9.8%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。