动态自适应采样,让模型自动选关键帧,省算力还提效果
Deep Attention-guided Adaptive Subsampling
- 用注意力机制动态决定哪些帧该保留
- 在医学影像和超声视频上准确率提升且计算量减少
- 适合需要实时推理的医疗视觉任务
尽管深度神经网络带来了显著性能提升,但通常伴随计算复杂度和成本增加。在3D体数据或视频分类等任务中,由于固有冗余,并非所有切片或帧都必要。为此,我们提出一种可融入任意神经网络架构的可学习采样框架。由于采样是不可微操作,直接整合至深度学习模型存在挑战。尽管已有研究使用Gumbel-max技巧应对非可微性,但它们仅具备任务自适应性,不随输入变化——训练后采样策略固定,难以应用于真实场景。为此,我们提出一种注意力引导的采样模块,在推理时也能根据输入动态调整。这种动态适应带来性能提升并降低模型复杂度。我们在MedMNIST3D医学影像数据集及两个超声视频数据集(包括一个在真实临床条件下采集的挑战性内部数据集)上验证了方法的有效性。
原文摘要 · Abstract (English)
Although deep neural networks have provided impressive gains in performance, these improvements often come at the cost of increased computational complexity and expense. In many cases, such as 3D volume or video classification tasks, not all slices or frames are necessary due to inherent redundancies. To address this issue, we propose a novel learnable subsampling framework that can be integrated into any neural network architecture. Subsampling, being a nondifferentiable operation, poses significant challenges for direct adaptation into deep learning models. While some works, have proposed solutions using the Gumbel-max trick to overcome the problem of non-differentiability, they fall short in a crucial aspect: they are only task-adaptive and not inputadaptive. Once the sampling mechanism is learned, it remains static and does not adjust to different inputs, making it unsuitable for real-world applications. To this end, we propose an attention-guided sampling module that adapts to inputs even during inference. This dynamic adaptation results in performance gains and reduces complexity in deep neural network models. We demonstrate the effectiveness of our method on 3D medical imaging datasets from MedMNIST3D as well as two ultrasound video datasets for classification tasks, one of them being a challenging in-house dataset collected under real-world clinical conditions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。