arXiv:2510.13802cs.CV2025-10被引 32

用轨迹场统一表示视频中任意物体运动,一次前向传播完成预测。

Trace Anything: Representing Any Video in 4D via Trajectory Fields

  • 将每个像素的运动建模为3D贝塞尔曲线,用控制点参数化轨迹
  • 在新基准上达到顶尖性能,单次前向传播效率远超传统方法
  • 支持运动预测、目标导向操作等新能力,适合动态视频建模任务

有效的时空表征是建模、理解与预测视频动态的基础。视频的基本单元——像素,在时间上形成连续的3D轨迹,构成动态的原始元素。基于此,我们提出将任意视频表示为轨迹场:对每一帧中的每个像素,映射出一个随时间变化的连续3D轨迹函数。在此框架下,我们引入Trace Anything模型,通过一次前向传播即可预测整个轨迹场。具体而言,对每帧中每个像素,模型输出一组控制点以参数化轨迹(即贝塞尔曲线),从而在任意查询时间点获得其3D位置。我们在大规模4D数据集上训练该模型,包括自建平台的数据;实验表明:(i) Trace Anything在新提出的轨迹场估计基准上达到当前最优性能,并在已有点追踪基准上表现竞争力;(ii) 由于采用单次前向传播范式,显著提升效率,无需迭代优化或辅助估计器;(iii) 展现出涌现能力,如目标条件操控、运动预测和时空融合。项目主页:https://trace-anything.github.io/。

原文摘要 · Abstract (English)

Effective spatio-temporal representation is fundamental to modeling, understanding, and predicting dynamics in videos. The atomic unit of a video, the pixel, traces a continuous 3D trajectory over time, serving as the primitive element of dynamics. Based on this principle, we propose representing any video as a Trajectory Field: a dense mapping that assigns a continuous 3D trajectory function of time to each pixel in every frame. With this representation, we introduce Trace Anything, a neural network that predicts the entire trajectory field in a single feed-forward pass. Specifically, for each pixel in each frame, our model predicts a set of control points that parameterizes a trajectory (i.e., a B-spline), yielding its 3D position at arbitrary query time instants. We trained the Trace Anything model on large-scale 4D data, including data from our new platform, and our experiments demonstrate that: (i) Trace Anything achieves state-of-the-art performance on our new benchmark for trajectory field estimation and performs competitively on established point-tracking benchmarks; (ii) it offers significant efficiency gains thanks to its one-pass paradigm, without requiring iterative optimization or auxiliary estimators; and (iii) it exhibits emergent abilities, including goal-conditioned manipulation, motion forecasting, and spatio-temporal fusion. Project page: https://trace-anything.github.io/.

视频建模轨迹预测神经网络

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。