基于偏度剪枝的多模态Transformer，让皮肤病变分类模型更小更快
Skewness-Guided Pruning of Multimodal Swin Transformers for Federated Skin Lesion Classification on Edge Devices
- 根据输出分布偏度动态剪枝注意力与全连接层
- 模型体积减少36%且精度不变
- 适合边缘设备上隐私保护的医疗图像分析
近年来，高性能计算机视觉模型在医学影像中取得显著进展，部分皮肤病变分类系统诊断准确率甚至超过皮肤科专家。然而，这些模型计算量大、体积庞大，难以部署在边缘设备上。同时，严格的隐私限制阻碍了数据集中管理，推动了联邦学习（FL）的应用。为此，本文提出一种基于偏度引导的剪枝方法，根据多模态Swin Transformer中多头自注意力和多层感知机层输出分布的统计偏度，选择性地进行剪枝。该方法在水平联邦学习环境中验证，有效降低了模型复杂度且保持性能。在紧凑型Swin Transformer上的实验显示，模型大小减少约36%，精度无损失。结果表明，在边缘设备上实现高效模型压缩与隐私保护的分布式医疗AI是可行的。
原文摘要 · Abstract (English)
In recent years, high-performance computer vision models have achieved remarkable success in medical imaging, with some skin lesion classification systems even surpassing dermatology specialists in diagnostic accuracy. However, such models are computationally intensive and large in size, making them unsuitable for deployment on edge devices. In addition, strict privacy constraints hinder centralized data management, motivating the adoption of Federated Learning (FL). To address these challenges, this study proposes a skewness-guided pruning method that selectively prunes the Multi-Head Self-Attention and Multi-Layer Perceptron layers of a multimodal Swin Transformer based on the statistical skewness of their output distributions. The proposed method was validated in a horizontal FL environment and shown to maintain performance while substantially reducing model complexity. Experiments on the compact Swin Transformer demonstrate approximately 36\% model size reduction with no loss in accuracy. These findings highlight the feasibility of achieving efficient model compression and privacy-preserving distributed learning for multimodal medical AI on edge devices.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。