智能决定何时在线训练单目深度网络，省算力还保精度
DecTrain: Deciding When to Train a Monocular Depth DNN Online
- 基于自监督机制，动态判断是否需要在线训练
- 平均仅训练44%时间，精度接近全程训练
- 适合资源受限场景下高效部署深度模型
深度神经网络在部署数据与训练数据分布不一致时会精度下降。虽然全程在线训练可提升精度，但计算开销大。本文提出DecTrain算法，利用自监督方法以低开销决定单目深度DNN的在线训练时机。通过比较训练成本与预测精度增益，DecTrain在分布外数据上表现优异：平均仅训练44%的时间，仍保持与全程在线训练相当的精度。与高推理成本的通用模型相比，DecTrain恢复了97%的精度增益，而后者仅恢复66%；使用更小的模型时，仍实现89%精度恢复，计算量减少56%。DecTrain使小型模型在低计算成本下达到大型通用模型的竞争力。
原文摘要 · Abstract (English)
Deep neural networks (DNNs) can deteriorate in accuracy when deployment data differs from training data. While performing online training at all timesteps can improve accuracy, it is computationally expensive. We propose DecTrain, a new algorithm that decides when to train a monocular depth DNN online using self-supervision with low overhead. To make the decision at each timestep, DecTrain compares the cost of training with the predicted accuracy gain. We evaluate DecTrain on out-of-distribution data, and find DecTrain maintains accuracy compared to online training at all timesteps, while training only 44% of the time on average. We also compare the recovery of a low inference cost DNN using DecTrain and a more generalizable high inference cost DNN on various sequences. DecTrain recovers the majority (97%) of the accuracy gain of online training at all timesteps while reducing computation compared to the high inference cost DNN which recovers only 66%. With an even smaller DNN, we achieve 89% recovery while reducing computation by 56%. DecTrain enables low-cost online training for a smaller DNN to have competitive accuracy with a larger, more generalizable DNN at a lower overall computational cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。