用自然语言生成可编辑的视角辅助场景,支持安全与日常任务测试。
VISTA: A Controllable Platform for Generating and Auditing Egocentric Assistance Scenarios

- 输入自然语言描述,自动构建包含六阶段的可编辑生成流程。
- 生成视频与审查轨迹,人类评估显示更贴近原始意图。
- 适合需要可控、可审计辅助场景的AI评估研究者使用。
评估人工智能代理在日常活动(从常规家务到紧急安全事件)中是否能主动协助人类,需要多样化的视觉数据。然而,真实世界中收集此类场景往往困难、昂贵或存在风险;而仿真环境又常缺乏社会常识,难以模拟不同行为的后果。本文提出VISTA,一个可控平台:用户只需提供一段自然语言描述的场景种子,即可生成可编辑的计划、第一人称视角视频及可审计的审查记录。VISTA围绕三种交互模式(被动响应、显式主动、隐式主动)和两类后果(紧急安全与日常不便),并以无协助情况为对照。其六阶段流程暴露设计说明、时间事件脚本、首帧计划、运动规划与视频规划,支持用户以自然语言修改各环节,再授权生成媒体。人类评估表明,完整VISTA流程生成的视频比两种单次通过基线更贴近场景种子。因此,VISTA实现了目标导向的第一人称场景生成的可检查、可修改与实证可审计。
原文摘要 · Abstract (English)
Evaluating whether AI agents can proactively assist humans in daily activities, ranging from routine household tasks to urgent safety-critical situations, requires diverse visual data. However, collecting such scenarios in the real world is often difficult, costly, or unsafe, and simulation environments often lack the social commonsense needed to simulate the consequences of different actions. In this work, we present VISTA, a controllable platform that uses a user-provided scenario seed, defined as a short natural-language description of the intended assistance situation, to generate editable plans, egocentric videos, and an auditable review trail. VISTA structures scenario intent around three interaction modes, including reactive, explicit proactive, and implicit proactive, and two consequence families, including safety-critical and everyday inconvenience, with no-assistance cases as controls. Its six-stage pipeline exposes the design brief, timed event script, first-frame plan, motion plan, and video plan, allowing users to revise each artifact in natural language before explicitly authorizing media generation. A human evaluation shows that videos retained by the complete VISTA workflow align more closely with their scenario seeds than outputs from two one-pass baselines. VISTA thereby makes targeted egocentric scenario generation inspectable, revisable, and empirically auditable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。