动态分配梯度提升多任务自监督学习效果
Rebalancing gradient to improve self-supervised co-training of depth, odometry and optical flow predictions
- 根据重建误差差异动态调整梯度分配，平衡多任务学习进度
- 在KITTI和CityScapes上同时提升深度、位姿和光流预测精度
- 适合自监督视觉感知系统研发者参考
我们提出CoopNet，通过动态调整梯度分配来改善协同训练网络的协作效率，确保各任务学习进度均衡。该方法应用于运动感知的自监督深度图预测，引入一种新型混合损失函数，基于深度+位姿联合网络与光流网络在像素级图像重建误差上的分布模型。该模型假设：运动物体对应的像素点（需从深度和位姿训练中剔除）是两个重建结果强烈不一致的位置。我们通过理论分析和实验验证了这一假设的有效性。在KITTI和CityScapes数据集上的对比评估表明，CoopNet在深度、位姿和光流预测方面均达到或超过当前最先进水平。
原文摘要 · Abstract (English)
We present CoopNet, an approach that improves the cooperation of co-trained networks by dynamically adapting the apportionment of gradient, to ensure equitable learning progress. It is applied to motion-aware self-supervised prediction of depth maps, by introducing a new hybrid loss, based on a distribution model of photo-metric reconstruction errors made by, on the one hand the depth + odometry paired networks, and on the other hand the optical flow network. This model essentially assumes that the pixels from moving objects (that must be discarded for training depth and odometry), correspond to those where the two reconstructions strongly disagree. We justify this model by theoretical considerations and experimental evidences. A comparative evaluation on KITTI and CityScapes datasets shows that CoopNet improves or is comparable to the state-of-the-art in depth, odometry and optical flow predictions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。