用视觉变压器实现保结构风格迁移，提升医学图像泛化能力。
Stylizing ViT: Anatomy-Preserving Instance Style Transfer for Domain Generalization
- 共享注意力块同时保持解剖结构与风格转移
- 在病理和皮肤科图像上最高提升13%准确率
- 训练后测试时仍能提升17%性能，适合医疗图像应用
医学图像分析中的深度学习模型常因数据异质性和稀缺性，在不同领域和人群间泛化能力差。传统增强方法在显著域偏移下失效。近期风格增强虽提升域泛化，但风格多样性不足或引入伪影。为此，我们提出Stylizing ViT，一种新型视觉变换器编码器，通过共享权重的注意力块实现自注意力（保持解剖一致性）与交叉注意力（执行风格迁移）。我们在组织病理学和皮肤病学三个图像分类任务中，以该方法进行数据增强。结果表明，相比最先进方法，准确率最高提升13%，生成图像无伪影且视觉逼真。此外，该方法在推理阶段亦有效，用于测试时增强可带来17%的性能提升。
原文摘要 · Abstract (English)
Deep learning models in medical image analysis often struggle with generalizability across domains and demographic groups due to data heterogeneity and scarcity. Traditional augmentation improves robustness, but fails under substantial domain shifts. Recent advances in stylistic augmentation enhance domain generalization by varying image styles but fall short in terms of style diversity or by introducing artifacts into the generated images. To address these limitations, we propose Stylizing ViT, a novel Vision Transformer encoder that utilizes weight-shared attention blocks for both self- and cross-attention. This design allows the same attention block to maintain anatomical consistency through self-attention while performing style transfer via cross-attention. We assess the effectiveness of our method for domain generalization by employing it for data augmentation on three distinct image classification tasks in the context of histopathology and dermatology. Results demonstrate an improved robustness (up to +13% accuracy) over the state of the art while generating perceptually convincing images without artifacts. Additionally, we show that Stylizing ViT is effective beyond training, achieving a 17% performance improvement during inference when used for test-time augmentation. The source code is available at https://github.com/sdoerrich97/stylizing-vit .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。