让普通人也能协作生成高质量虚拟现实3D资产
CoGen3D: An Agentic Human-AI Co-Design Pipeline for 3D Asset Generation for Virtual Reality

- 通过对话引导用户逐步明确设计意图,再生成3D模型
- 用户研究显示协作设计使场景参与度提升,情绪反应更丰富
- 适合非专业创作者、希望降低VR内容创作门槛的人
为虚拟现实创建3D资产需要建模专业知识,限制了沉浸式体验的创作主体。现有生成式AI工具依赖无约束的指令输入,缺乏对话式引导来帮助用户表达意图并验证设计。为此,我们提出CoGen3D,一个主动引导用户完成对话式意图提取、概念图确认及图像到3D生成的智能人机协同设计流程,并直接部署至沉浸式场景。通过一项包含120名参与者(60名设计组,60名体验组)的用户研究,评估了六个情感多样化的沉浸式场景中的表现。结果表明,协作设计的资产显著提升场景参与度与情感响应;用户普遍偏好概念图而非最终3D模型,且对自己创作的瑕疵未表现出更高容忍度。对人机对话的分析显示,目标环境会影响用户的对话模式。研究证明,分阶段、以意图为导向的协同设计可推动虚拟现实创作民主化,使内容生产从技术执行转向协作空间设计。
原文摘要 · Abstract (English)
Creating 3D assets for virtual reality requires modeling expertise, which restricts the authorship of immersive experiences. Existing generative AI tools rely on unconstrained, command-driven prompting, lacking the conversational scaffolding needed for users to articulate their intent and validate designs prior to rendering. To address this, we introduce CoGen3D, an agentic human-AI co-design pipeline that proactively guides users through conversational intent elicitation, a concept image confirmation, and image-to-3D generation that directly deploys to immersive scenes. We evaluated this system through a user study (N=120) across six affectively diverse immersive scenes, observing 60 Design group participants who co-created 3D assets for the scenes, and 60 Validation group participants who experienced the scenes with generated assets. Our findings show that co-designed assets are associated with higher scene engagement and shifted affective responses, while participants generally preferred concept images over the final 3D assets, with no increased leniency toward degradation in their own creations. Analysis of the human-AI conversations further shows that target environments shape users' conversational patterns. Our results suggest that our staged, intent-based co-design can democratize virtual reality authoring and shift immersive content creation from technical execution toward collaborative spatial design.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。