arXiv:2410.08256cs.LGcs.AI2024-10中稿 · SenSys 2024被引 10

AdaShadow让手机模型在动态环境中快速自适应,提升响应速度与精度。

AdaShadow: Responsive Test-time Model Adaptation in Non-stationary Mobile Environments

  • 只更新关键网络层,减少计算量以降低延迟。
  • 实测比顶尖TTA方法快2至3.5倍,延迟在毫秒级。
  • 适合资源受限的实时移动应用,如自动驾驶和增强现实。

在持续、不可预测的领域偏移下,移动端应用(如自动驾驶、增强现实)需要在设备端实时适应环境变化以保障流畅体验。测试时自适应(TTA)通过在推理前利用无标签实时数据微调模型参数,成为有前景的解决方案。然而,TTA特有的前向-反向-再前向流程显著增加了延迟,削弱了对时间敏感应用的响应性。本文提出AdaShadow,一种针对非平稳移动数据分布与资源动态的响应式测试时自适应框架,通过选择性更新适应关键层实现高效性。尽管该策略在通用设备训练中已知,但TTA的无监督在线特性带来了独特挑战:如何快速评估层重要性、准确估计延迟并规划最优更新方案。AdaShadow采用无需反向传播的评估器快速识别关键层,基于单元的运行时预测器考虑资源动态进行延迟估算,并设计在线调度器实现及时层更新规划。此外,还引入内存I/O感知的计算复用机制,进一步降低再前向阶段的延迟。实验表明,AdaShadow在持续域偏移下实现了最佳的精度-延迟平衡:在低内存与能耗下,相较现有最先进TTA方法提速2至3.5倍(毫秒级),且精度相当;相比高效监督方法,在相似延迟下精度提升14.8%至25.4%。

原文摘要 · Abstract (English)

On-device adapting to continual, unpredictable domain shifts is essential for mobile applications like autonomous driving and augmented reality to deliver seamless user experiences in evolving environments. Test-time adaptation (TTA) emerges as a promising solution by tuning model parameters with unlabeled live data immediately before prediction. However, TTA's unique forward-backward-reforward pipeline notably increases the latency over standard inference, undermining the responsiveness in time-sensitive mobile applications. This paper presents AdaShadow, a responsive test-time adaptation framework for non-stationary mobile data distribution and resource dynamics via selective updates of adaptation-critical layers. Although the tactic is recognized in generic on-device training, TTA's unsupervised and online context presents unique challenges in estimating layer importance and latency, as well as scheduling the optimal layer update plan. AdaShadow addresses these challenges with a backpropagation-free assessor to rapidly identify critical layers, a unit-based runtime predictor to account for resource dynamics in latency estimation, and an online scheduler for prompt layer update planning. Also, AdaShadow incorporates a memory I/O-aware computation reuse scheme to further reduce latency in the reforward pass. Results show that AdaShadow achieves the best accuracy-latency balance under continual shifts. At low memory and energy costs, Adashadow provides a 2x to 3.5x speedup (ms-level) over state-of-the-art TTA methods with comparable accuracy and a 14.8% to 25.4% accuracy boost over efficient supervised methods with similar latency.

测试时自适应移动端优化延迟优化动态环境

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。