用生成模型给每个人涂持久颜色，实现无需追踪器的多目标跟踪
The generator is the tracker: Multi-object tracking by painting persistent identity colours
- 用文本到视频扩散模型为每个人分配唯一持久颜色，身份信息存于像素中
- 在DanceTrack上达40.3 HOTA，关联准确率达44.1，显著优于传统方法
- 适合关注生成式视觉推理与身份连续性建模的研究者
多目标跟踪通常分为检测与关联两步，身份信息依赖外部状态（如轨迹缓冲、运动模型、外观嵌入）。本文探索视频生成器能否将身份状态内化于像素。我们微调一个220亿参数的文本到视频扩散模型（LTX-2.3），通过轻量级上下文内LoRA，将RGB视频片段转为身份图片段——每人被涂上固定且唯一的颜色，随时间持续存在。长视频通过窗口链式生成，每个窗口基于前一窗口的清理尾部条件生成。简短续训后，模型可延续已有着色，使身份信息在无追踪器、无运动模型、无重识别模块下自然传递。在DanceTrack测试服务器上，本系统（首个在此评估的生成式跟踪器，唯一无检测器和追踪堆栈的方案）达到40.3 HOTA。虽低于顶尖水平（≥70 HOTA），但具独特反向误差特征：关联得分（AssA 44.1）超越原基准所有追踪器，仅检测性能拖累整体表现。对照实验表明机制关键：相同生成窗口若用传统后处理关联，性能差两倍（18.2 HOTA）；逐帧IoU关联会打断生成颜色保持的完整轨迹。在383个挖掘出的遮挡事件中，生成器在42%条件下成功恢复断开身份，而外观嵌入基线为零，甚至可处理超过其上下文长度的遮挡，证明颜色分配已成为涌现的重识别信号。代码、检查点及完整注册实验日志已公开。
原文摘要 · Abstract (English)
Multi-object tracking (MOT) is conventionally decomposed into detection followed by association, with object identity maintained as external state: track buffers, motion models, and appearance embeddings. We ask whether a video generator can maintain that state in pixels. We fine-tune a 22B text-to-video diffusion model (LTX-2.3) with a lightweight in-context LoRA to translate an RGB clip into an ID-map clip, a video in which every person is painted a flat, distinct color that persists over time: same color, same identity. Long videos are generated as chained windows, where each window is conditioned on the cleaned tail of the previous one. A brief continuation fine-tune teaches the model to extend a given coloring, after which identity flows through the chain with no tracker, no motion model, and no re-identification module. On the DanceTrack test server, our system, to our knowledge the first generative tracker evaluated there and the only entry with no detector and no tracking stack, reaches 40.3 HOTA. This is well below today's specialist state of the art (>=70 HOTA), but with a unique, inverted error profile: its association score (AssA 44.1) exceeds every tracker of the original benchmark suite while detection remains the sole deficit. Controlled comparisons show the mechanism matters: the same generated windows linked by classical post-hoc association score 2x worse (18.2 HOTA), and frame-to-frame IoU association fragments tracks that the generator's colors keep whole. On 383 mined occlusion events, the generator re-acquires identities after gaps at a 42% conditional rate where appearance-embedding baselines score zero, including gaps longer than its temporal context, evidence that the generator's color assignment functions as an emergent re-identification signal. We release code, checkpoints, and the full pre-registered experimental log.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。