提出FluxGraph，让物体状态追踪更快更准。
Efficient Tracking and Understanding Object Transformations
- 用SAM2的多掩码不一致做触发，只在可能变化时才计算
- 速度比TubeletGraph快3.3倍，且跟踪更准
- 适合需要实时追踪物体变化的场景
追踪物体在状态变换中的行为对理解真实世界动态至关重要。然而现有方法计算开销大。TubeletGraph虽表现优异，但单帧每物体推理耗时约4.4秒（在VOST上），难以实现实时部署。我们发现其高成本源于对输入视频进行密集时空分区：(1) 每帧都密集计算实体分割，无论是否发生变换；(2) 跟踪场景中所有实体，开销随场景复杂度增长，而非关注变换数量。为此，我们提出FluxGraph，一种响应式方法：利用SAM2内部多掩码不一致作为轻量级变换检测触发器，并避免跟踪视频中所有实体。FluxGraph在VOST上比TubeletGraph快约3.3倍，同时提升跟踪性能并保持状态图质量。此外，在VSCOS、M^3-VOS和DAVIS17上也实现了3.7–10.7倍的稳定加速，且性能不变。代码已公开于https://github.com/YihongSun/FluxGraph。
原文摘要 · Abstract (English)
Tracking objects through state transformations is essential for understanding real-world dynamics. However, existing methods are computationally expensive. TubeletGraph recently showed impressive capabilities, but its inference cost (~$4.4$ seconds per object-frame on VOST) precludes any real-time deployment possibilities. We observe that TubeletGraph's overhead arises from building a spatiotemporal partition of the input video: (1) entity segmentation is computed densely for every frame regardless of whether a transformation occurs, and (2) every entity in the scene is tracked, scaling cost with scene complexity rather than the number of transformations of interest. To address both, we propose FluxGraph, a reactive variant that uses SAM2's internal multi-mask disagreement as a lightweight trigger for transformation detection, and removes the need for tracking all entities in the given video. FluxGraph is ~$3.3\times$ faster than TubeletGraph on VOST while improving tracking performance and preserving state graph quality. Furthermore, we also observe consistent speedups of $3.7-10.7\times$ across VSCOS, M$^3$-VOS, and DAVIS17 while maintaining performance. Code is publicly available at https://github.com/YihongSun/FluxGraph.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。