arXiv:2602.16160cs.CV2026-02被引 3

动态调整视觉追踪模型深度,提升效率且不损失精度

Uncertainty-Guided Inference-Time Depth Adaptation for Transformer-Based Visual Tracking

  • 基于置信度自适应选择编码器与解码器深度
  • 实测节省12%计算量,延迟降8.9%,能耗减10.8%
  • 适合长视频追踪,兼顾速度与精度

基于Transformer的单目标追踪器虽精度领先,但固定深度推理导致每帧都执行完整编码器-解码器结构,尤其在时间连贯性强的长视频中造成冗余计算。本文提出UncL-STARK,一种保持架构不变的动态深度自适应方法,无需修改原网络或增加辅助头。通过随机深度训练与知识蒸馏,使模型在多层中间深度仍具备鲁棒预测能力,支持运行时安全截断。推理阶段,直接从角点定位热图生成轻量级不确定性估计,结合视频的时间连贯性,以反馈策略决定下一帧的深度。在GOT-10k和LaSOT数据集上,实验显示最多可降低12% GFLOPs、8.9%延迟和10.8%能耗,同时短时与长时序列跟踪精度仅比全深度基线低0.2%。

原文摘要 · Abstract (English)

Transformer-based single-object trackers achieve state-of-the-art accuracy but rely on fixed-depth inference, executing the full encoder--decoder stack for every frame regardless of visual complexity, thereby incurring unnecessary computational cost in long video sequences dominated by temporally coherent frames. We propose UncL-STARK, an architecture-preserving approach that enables dynamic, uncertainty-aware depth adaptation in transformer-based trackers without modifying the underlying network or adding auxiliary heads. The model is fine-tuned to retain predictive robustness at multiple intermediate depths using random-depth training with knowledge distillation, thus enabling safe inference-time truncation. At runtime, we derive a lightweight uncertainty estimate directly from the model's corner localization heatmaps and use it in a feedback-driven policy that selects the encoder and decoder depth for the next frame based on the prediction confidence by exploiting temporal coherence in video. Extensive experiments on GOT-10k and LaSOT demonstrate up to 12% GFLOPs reduction, 8.9% latency reduction, and 10.8% energy savings while maintaining tracking accuracy within 0.2% of the full-depth baseline across both short-term and long-term sequences.

视觉追踪Transformer动态推理节能

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。