arXiv:2604.05414cs.LGcs.CV2026-04

提出旋转表示的梯度理论,解释为何训练不用SVD而推理用。

Training Without Orthogonalization, Inference With SVD: A Gradient Analysis of Rotation Representations

  • 分析SVD反向传播的梯度特性,发现其会引入可量化的梯度畸变。
  • 证明早期训练时因奇异值接近零,梯度畸变最严重,导致性能下降。
  • 揭示9D参数化优于6D,因后者梯度信号不均,适合快速推理场景。

近期研究表明,训练时不进行正交化、仅在推理时使用SVD投影能提升旋转估计性能,实证支持9D表示搭配SVD投影。然而,对为何SVD正交化在训练中造成损害,以及为何优于Gram-Schmidt在推理中被采用的理论理解仍不完整。本文针对3×3矩阵与SO(3)投影,提供了SVD正交化的详细梯度分析。核心结果为:精确推导出SVD反向传播雅可比矩阵的谱结构——秩为3(匹配SO(3)维度),非零奇异值为2/(s_i + s_j),条件数κ = (s_1 + s_2)/(s_2 + s_3),导致可量化的梯度畸变,尤其在预测矩阵远离SO(3)时最严重(如训练初期当s_3 ≈ 0)。进一步表明,即使稳定化后的SVD梯度仍存在方向误差,而移除训练中的SVD可彻底避免该权衡。同时证明6D Gram-Schmidt雅可比矩阵具有不对称谱结构,参数接收不等梯度信号,解释为何9D参数化更优。这些结果为直接9D回归训练并仅在推理时应用SVD投影提供了理论基础。

原文摘要 · Abstract (English)

Recent work has shown that removing orthogonalization during training and applying it only at inference improves rotation estimation in deep learning, with empirical evidence favoring 9D representations with SVD projection. However, the theoretical understanding of why SVD orthogonalization specifically harms training, and why it should be preferred over Gram-Schmidt at inference, remains incomplete. We provide a detailed gradient analysis of SVD orthogonalization specialized to $3 \times 3$ matrices and $SO(3)$ projection. Our central result derives the exact spectrum of the SVD backward pass Jacobian: it has rank $3$ (matching the dimension of $SO(3)$) with nonzero singular values $2/(s_i + s_j)$ and condition number $κ= (s_1 + s_2)/(s_2 + s_3)$, creating quantifiable gradient distortion that is most severe when the predicted matrix is far from $SO(3)$ (e.g., early in training when $s_3 \approx 0$). We further show that even stabilized SVD gradients introduce gradient direction error, whereas removing SVD from the training loop avoids this tradeoff entirely. We also prove that the 6D Gram-Schmidt Jacobian has an asymmetric spectrum: its parameters receive unequal gradient signal, explaining why 9D parameterization is preferable. Together, these results provide the theoretical foundation for training with direct 9D regression and applying SVD projection only at inference.

旋转估计SVD梯度分析SO(3)

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。