提出动态稀疏注意力机制，让3D重建Transformer更快更省资源。
SAF3R: Dynamic Sparse Attention for Feed-Forward 3D Reconstruction Transformers
- 通过分析发现全局注意力具有高度稀疏且动态变化的特性，据此设计动态稀疏策略。
- 在不损失重建精度前提下，实现高达90%以上的注意力稀疏度，推理速度提升3倍以上。
- 无需重新训练，适配性强，适合部署在长序列3D重建任务中。
前馈3D重建（F3R）Transformer近期取得显著进展，但扩展到长图像序列时仍面临挑战，因其跨视图全局注意力的二次复杂度迅速成为主要计算瓶颈。现有方法虽尝试通过压缩或稀疏注意力提升效率，却未能充分挖掘全局注意力固有的稀疏性与动态行为。本文对多个F3R Transformer中的全局注意力进行系统分析，发现其在不同层与注意力头间表现出高度异质、动态且极稀疏的特性。基于此，我们提出SAF3R——一种面向F3R Transformer的免训练动态稀疏注意力框架。该框架结合定制化稀疏机制、离线头级分析与高效在线适应策略，精准匹配输入依赖的注意力行为。大量实验表明，SAF3R在保持相机位姿估计与3D重建质量的前提下，实现极高稀疏率，相比现有方法显著提升端到端推理速度。代码已开源。
原文摘要 · Abstract (English)
Feed-forward 3D reconstruction (F3R) transformers have recently achieved remarkable success. However, scaling them to long image sequences remains challenging, as the quadratic complexity of cross-view global attention quickly becomes the dominant computational bottleneck. While recent efforts attempt to improve efficiency through compressed or sparse attention, they fail to fully exploit the inherent sparsity and dynamic behavior of global attention. In this work, we present a comprehensive analysis of global attention across multiple F3R transformers and reveal that attention patterns are highly heterogeneous, dynamic, and extremely sparse across layers and attention heads. Motivated by these findings, we propose SAF3R, a training-free dynamic sparse attention framework tailored to F3R transformers. SAF3R integrates tailored sparse attention mechanisms with offline head profiling and an efficient online adaptation strategy to match input-dependent attention behaviors. Extensive experiments demonstrate that SAF3R achieves high sparsity ratios while preserving camera pose estimation and 3D reconstruction quality, translating into substantial end-to-end speedup on F3R transformers compared to existing methods. Code is available at https://github.com/jndeng/SAF3R
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。