提出新方法解耦3D卷积核的纹理与运动偏好，让动态特征更可解释
Feature Visualization in 3D Convolutional Neural Networks
- 通过数据驱动分解最大激活输入，分离纹理与运动成分
- 两阶段优化使3D卷积核的运动模式可视化清晰可辨
- 适用于理解视频模型深层特征，适合视觉算法研究者
理解卷积神经网络的计算过程需要有效的卷积核可视化。尽管最大激活法在2D卷积核中表现良好，但直接应用于3D卷积常因高维性和复杂性导致结果不可读。为此，我们提出一种新型3D卷积核可视化方法，可解耦纹理与运动偏好。该方法首先对最大化激活特定卷积核的输入进行数据驱动分解，再采用两阶段优化策略从中提取独立的纹理和运动成分。在多个预训练模型不同深度的卷积核上应用该方法，结果表明所生成的可视化，特别是运动部分，能清晰揭示3D卷积核编码的动态模式。这证明了该方法在提供可解释性洞察方面的有效性。代码已开源：https://github.com/YatangLiLab/3DKernelVisualizer。
原文摘要 · Abstract (English)
Understanding the computations of convolutional neural networks requires effective visualization of their kernels. While maximal activation methods have proven successful in highlighting the preferred features of 2D convolutional kernels, directly applying these techniques to 3D convolutions often leads to uninterpretable results due to the higher dimensionality and complexity of 3D features. To address this challenge, we propose a novel visualization approach for 3D convolutional kernels that disentangles their texture and motion preferences. Our method begins with a data-driven decomposition of the optimal input that maximally activates a given kernel. We then introduce a two-stage optimization strategy to extract distinct texture and motion components from this input. Applying our approach to visualize kernels at various depths of several pre-trained models, we find that the resulting visualizations--particularly those capturing motion--clearly reveal the preferred dynamic patterns encoded by 3D kernels. These results demonstrate the effectiveness of our method in providing interpretable insights into 3D convolutional operations. Code is available at https://github.com/YatangLiLab/3DKernelVisualizer.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。