让视觉Mamba模型具备旋转不变性,提升鲁棒性和效率
Rotation Equivariant Mamba for Vision Tasks
- 设计旋转等变的跨扫描策略与群Mamba模块
- 在多个任务上实现更强旋转鲁棒性,参数减少50%
- 适合追求高效高鲁棒性的视觉模型研究者
旋转等变性是视觉数据中最普遍且关键的结构先验,但当前基于Mamba的视觉架构中仍缺失这一特性。尽管Mamba在自然语言处理中表现优异,并逐步应用于计算机视觉,现有视觉Mamba模型未考虑图像旋转对称性,导致其对图像旋转敏感,限制了鲁棒性与跨任务泛化能力。为此,本文提出EQ-VMamba,首个用于视觉任务的旋转等变视觉Mamba架构。核心包括精心设计的旋转等变跨扫描策略与群Mamba块。我们还提供了内在等变误差的严格理论分析,证明该架构可端到端实现旋转等变性。在多个基准测试(包括高层图像分类、中层语义分割和低层图像超分辨率)上,EQ-VMamba consistently 提升了旋转鲁棒性,性能优于或媲美非等变基线,同时参数量减少约50%。结果表明,嵌入旋转等变性不仅能有效增强视觉Mamba模型对旋转变换的鲁棒性,还能在显著提升参数效率的同时提升整体性能。
原文摘要 · Abstract (English)
Rotation equivariance constitutes one of the most general and crucial structural priors for visual data, yet it remains notably absent from current Mamba-based vision architectures. Despite the success of Mamba in natural language processing and its growing adoption in computer vision, existing visual Mamba models fail to account for rotational symmetry in their design. This omission renders them inherently sensitive to image rotations, thereby constraining their robustness and cross-task generalization. To address this limitation, we incorporate rotation symmetry, a universal and fundamental geometric prior in images, into Mamba-based architectures. Specifically, we introduce EQ-VMamba, the first rotation equivariant visual Mamba architecture for vision tasks. The core components of EQ-VMamba include a carefully designed rotation equivariant cross-scan strategy and group Mamba blocks. Moreover, we provide a rigorous theoretical analysis of the intrinsic equivariance error, demonstrating that the proposed architecture enforces end-to-end rotation equivariance throughout the network. Extensive experiments across multiple benchmarks -- including high-level image classification, mid-level semantic segmentation, and low-level image super-resolution -- demonstrate that EQ-VMamba consistently improves rotation robustness and achieves superior or competitive performance compared to non-equivariant baselines, while requiring approximately 50\% fewer parameters. These results indicate that embedding rotation equivariance not only effectively bolsters the robustness of visual Mamba models against rotation transformations, but also enhances overall performance with significantly improved parameter efficiency. Code is available at https://github.com/zhongchenzhao/EQ-VMamba.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。