arXiv:2609.05239cs.CV2026-09

用视觉语言模型自动调参,无标签实现跨域跟踪适应。

Cross-Domain Tracker Adaptation Without Target-Domain Labels via Vision-Language Agents

论文配图:Cross-Domain Tracker Adaptation Without Target-Domain Labels via Vision-Language Agents
图 1 · 摘自论文原文
  • 用VLM分析跟踪结果,诊断失败原因并迭代优化参数。
  • 在MOT17→MOT20上恢复67.8%的性能损失,接近目标天花板。
  • 仅在需要时调整参数,避免对已良好配置的场景扰动。

我们提出一种系统,利用视觉语言模型(VLM)作为诊断代理,在无目标域标注的情况下,适应检测-跟踪流水线至新域。不同于依赖标注指标的优化,VLM直接检查生成的跟踪输出,识别视觉失败模式,并通过迭代调优循环推荐参数更新。我们首先证明,基于真实标注的超参数迁移可能脆弱:在MOT17→MOT20上,使用源域最优配置使平均HOTA从0.357降至0.267,下降0.090。在不使用任何目标域标签的前提下,我们的VLM调参器恢复了67.8%的性能损失,最终仅落后目标天花板0.029 HOTA;在最高密度序列中,恢复率达86.7%。此外,我们发现基于手工代理目标的无标签贝叶斯优化在大域偏移下表现不佳,甚至会恶化已有良好配置。相比之下,VLM调参器具有选择性:当视觉诊断未发现明显失败模式时,拒绝修改配置,从而在简单迁移中保持性能,同时提升困难迁移的表现。最后,我们界定该方法的有效条件:当域偏移表现为可暴露的检测级参数变化时有效;而在MOT17→DanceTrack这类源域配置已接近最优的场景下效果有限。

原文摘要 · Abstract (English)

We present a system that uses a Vision-Language Model (VLM) as a diagnostic agent for adapting a detect-to-track pipeline to a new target domain without access to target-domain labels. Rather than optimizing against annotated metrics, the VLM directly inspects rendered tracking outputs, identifies visual failure modes, and recommends parameter updates through an iterative tuning loop. We first demonstrate that ground-truth-supervised hyperparameter transfer can be brittle. On MOT17->MOT20, applying a source-derived oracle configuration reduces mean HOTA by 0.090, from a target-domain ceiling of 0.357, to 0.267. Without using any target-domain labels, our VLM-based tuner recovers 67.8% of this lost headroom, finishing within 0.029 HOTA of the target ceiling; on the highest-density target sequence, it recovers up to 86.7%. We further show that label-free Bayesian optimization with handcrafted proxy objectives struggles under large domain shifts and can degrade configurations that are already strong. In contrast, the VLM tuner acts selectively: when its visual diagnosis reveals no clear failure mode, it declines to modify the configuration, preserving performance on easy transfers while improving hard ones. Finally, we characterize the conditions under which this approach succeeds, namely, when domain shift manifests through exposed detection-level parameters, versus where it is less effective, such as MOT17->DanceTrack, where the source oracle is already near-optimal.

目标跟踪视觉语言模型无监督适应参数调优

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。