从单人视角视频预测多人并行执行任务，提升效率同时避免冲突
The N-Body Problem: Parallel Execution from Single-Person Egocentric Video
- 用结构化提示引导视觉语言模型推理3D环境与时间依赖
- 在100段视频上使动作覆盖率提升45%，碰撞率下降51%
- 适合研究多智能体协作与人机交互的学者参考
人类能直观地并行处理复杂活动，但模型能否仅通过观察单人视角视频就做出这种判断？本文提出N-Body问题：基于一段第一人称视频，预测N名个体假设性地并行完成相同任务的方案。目标是最大化加速比，但直接分配视频片段易违反现实约束，导致两人同时使用同一物体或占据同一空间等物理上不可能的情况。为此，本文形式化该问题，并提出一套评估指标，涵盖性能（加速比、任务覆盖）与可行性（空间碰撞、物体冲突、因果约束）。作为概念验证，引入一种结构化提示策略，引导视觉语言模型（VLM）推理3D环境、物体使用和时间依赖关系，生成可行的并行执行方案。在EPIC-Kitchens和HD-EPIC共100段视频上，当N=2时，该方法相较基线提示使Gemini 2.5 Pro的动作覆盖率提升45%，同时将碰撞、物体冲突和因果冲突分别降低51%、52%和55%。
原文摘要 · Abstract (English)
Humans can intuitively parallelise complex activities, but can a model predict this from observing a single person? Given one egocentric video, we introduce the N-Body Problem: predicting how N individuals, can hypothetically perform the same set of tasks. The goal is to maximise speed-up, but naive assignment of video segments to individuals often violates real-world constraints, leading to physically impossible scenarios like two people using the same object or occupying the same space. To quantify this, we formalise the N-Body Problem and propose a suite of metrics to evaluate both performance (speed-up, task coverage) and feasibility (spatial collisions, object conflicts and causal constraints). As a proof of concept, we introduce a structured prompting strategy that guides a Vision-Language Model (VLM) to reason about the 3D environment, object usage, and temporal dependencies, producing a viable parallel execution. On 100 videos from EPIC-Kitchens and HD-EPIC, for $N = 2$, our structured prompt improves action coverage by 45% over a baseline prompt for Gemini 2.5 Pro, while simultaneously slashing collision rates, object and causal conflicts by 51%, 52% and 55% respectively.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。