无需训练即可生成内镜图像反事实样本，保持结构一致。
PolypSteer: Counterfactual Endoscopic Synthesis via Training-Free Activation Steering
- 通过激活引导在扩散模型中直接操控病理概念。
- 反事实生成翻转率最高达0.950，结构保留更优。
- 适合医学图像增强与可解释性研究者使用。
生成式扩散模型被广泛用于医学影像数据增强，但文本提示无法生成因果训练数据。重提示会重新生成整个图像轨迹，改变解剖结构、纹理和背景。基于重构的编辑方法引入重建误差，导致结构漂移。我们提出PolypSteer，一种无需训练的激活引导框架，用于内镜合成。PolypSteer在扩散变换器的交叉注意力层中识别每对对比提示的病理向量。推理时，沿该向量引导图像激活，从头生成仅差异于引导概念的反事实图像对，其余结构均保持不变。我们在Kvasir v3和HyperKvasir上开展三项实验。在三组临床概念对的反事实生成中，PolypSteer实现0.800、0.925、0.950的翻转率，优于最佳重构基线，在概念翻转率与结构保留上均更优。在染色分离任务中，实现75%染色去除，远超PnP（20%）与h-Edit（10%）。在下游息肉检测任务中，使用PolypSteer生成的反事实对进行数据增强，使ViT AUC达0.9755，显著高于数量匹配的重提示方法（0.9083），验证了反事实结构带来的性能提升。代码见https://github.com/UARK-AICV/PolypSteer。
原文摘要 · Abstract (English)
Generative diffusion models are increasingly used for medical imaging data augmentation, but text prompting cannot produce causal training data. Re-prompting rerolls the entire generation trajectory, altering anatomy, texture, and background. Inversion-based editing methods introduce reconstruction error that causes structural drift. We propose PolypSteer, a training-free activation-steering framework for endoscopic synthesis. PolypSteer identifies a pathology vector for each contrastive prompt pair in the cross-attention layers of a diffusion transformer. At inference time, it steers image activations along this vector, generating counterfactual pairs from scratch where the only difference is the steered concept. All other structure is preserved by construction. We evaluate PolypSteer across three experiments on Kvasir v3 and HyperKvasir. On counterfactual generation across three clinical concept pairs, PolypSteer achieves flip rates of 0.800, 0.925, and 0.950, outperforming the best inversion-based baseline in both concept flip rate and structural preservation. On dye disentanglement, PolypSteer achieves 75% dye removal against 20% (PnP) and 10% (h-Edit). On downstream polyp detection, augmenting with PolypSteer counterfactual pairs achieves ViT AUC of 0.9755 versus 0.9083 for quantity-matched re-prompting, confirming that counterfactual structure drives the gain. Code is at link https://github.com/UARK-AICV/PolypSteer
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。