不更新模型权重，通过稀疏自编码器在生成时修正医学影像报告的幻觉问题。
Universal Boosts, Specific Suppressors: Sparse Autoencoder Steering of Medical Vision-Language Models
- 基于令牌级稀疏自编码器，在推理阶段进行残差导向干预。
- 在MIMIC-CXR上提升3个模型的临床综合指标，最高增益17.0%。
- 发现可迁移的增强方向，但抑制方向需针对模型单独设计。
医学视觉语言模型（VLMs）在生成胸部X光报告时常出现幻觉：虚构图像中不存在的病灶、遗漏重要发现或定位错误。本文提出一种无需权重更新的解码时残差引导方法，基于每令牌稀疏自编码器（SAE），在深层特征层选取Top-K SAE，采用因果引导抑制临床错误，推理时联合实施抑制与增强干预。在MIMIC-CXR测试集上，该无训练方法使三个放射科VLM（RadVLM、LLaVA-Rad、CheXOne）的报告质量显著提升，临床综合指标相对提高5.4%、7.2%和17.0%，且所有骨干模型均获得统计显著的GREEN增益。跨模型特征对齐显示，质量提升方向在不同架构间高度重合，而引发幻觉的方向则具有模型特异性。因此，可迁移的引导策略必须为各模型定制抑制项，而非共享统一抑制列表。相同方法零样本迁移至IU-Xray，GREEN指标提升7.7%，验证所识别特征属模型自身属性，非训练数据所致。代码与可视化工具已公开：https://cxr-sparse-feature-dashboard.netlify.app/。
原文摘要 · Abstract (English)
Medical vision-language models (VLMs) often hallucinate findings when generating chest X-ray reports: they fabricate findings that are not present in the image, miss important ones, or locate them incorrectly. We mitigate this without weight updates by decoding-time residual steering on a per-token sparse autoencoder (SAE) basis: Top-$K$ SAEs on late layers, causal steering against clinical errors, then combined suppress/boost intervention at inference time. On the MIMIC-CXR test split, our inference-only method improves the quality of generated reports for three radiology VLMs (RadVLM, LLaVA-Rad, and CheXOne), with relative improvements of +5.4%, +7.2%, and +17.0% in the clinical composite metric, and statistically significant GREEN gains on all backbones. A cross-model feature alignment shows that the quality-promoting (boost) directions overlap strongly across architectures, whereas hallucination-linked (suppress) directions are model-specific. Therefore, transferable steering must treat suppression per-backbone, rather than sharing a universal suppress list. The same recipe transfers zero-shot to IU-Xray (Green $+7.7\%$ rel.) without retraining, confirming that the identified features are properties of the model, not of the training corpus. We release causal feature sets and an interactive feature dashboard: https://cxr-sparse-feature-dashboard.netlify.app/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。