用哈达玛变换加速立体深度估计,提升效率不降精度。
WHTMix: Efficient Stereo Depth Estimation via Walsh-Hadamard Token Mixing

- 用哈达玛变换替代全局自注意力,实现对数线性复杂度混合。
- 在合成数据上误差与基线相当,计算量降低2.46倍,推理延迟降2.65倍。
- 适用于高分辨率立体匹配,尤其对远距离物体误差更小。
面向自动驾驶、机器人和增强现实的立体深度估计需在高分辨率下满足严格延迟要求。基于Transformer的匹配器中,全局自注意力因随像素数呈二次增长而成为主要耗时环节。本文表明,立体Transformer中负责跨视图传播上下文的联合自注意力阶段,可被一种不依赖数据的哈达玛令牌混合器取代,该混合器在变换域中以对数线性成本完成全局令牌混合,而保留依赖数据的交叉注意力以执行左右对应。在合成驾驶数据上,该方法在端点误差上与注意力基线相当,模型计算量减少2.46倍,单图像推理延迟降低2.65倍。复杂度分析显示收益取决于序列长度与通道宽度之比,这解释了为何高分辨率立体匹配尤为受益,而分类任务则不然;我们也在非立体长序列基准上验证了这一令牌-通道缩放规律。此外,提出一种混合对数位移损失函数,重点提升远距离物体的小位移像素权重,显著降低远距离误差,且无额外计算开销。
原文摘要 · Abstract (English)
Stereo depth estimation for driving, robotics and augmented reality must run at high resolution under tight latency budgets, yet in transformer-based matchers the global self-attention that aggregates scene context grows quadratically with the number of pixels and comes to dominate runtime. We show that the joint self-attention stage of a stereo transformer, whose role is to spread context across both views, can be replaced by a data-independent Walsh-Hadamard token mixer that mixes tokens globally in the transform domain at log-linear cost, while the data-dependent cross-attention that performs left-right correspondence is retained. On synthetic driving data the mixer matches the attention baseline in end-point error while reducing model compute by a factor of 2.46 and single-image inference latency by a factor of 2.65. A complexity analysis shows the benefit is governed by the ratio of sequence length to channel width, which explains why high-resolution stereo matching is a particularly favorable setting and why classification transformers are not; we confirm this token-to-channel scaling on non-stereo long-sequence benchmarks. Furthermore, we introduce a hybrid log-disparity loss function designed to up-weight small-disparity pixels corresponding to long-range objects. This approach reduces the error on distant objects without incurring any additional computational overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。