用程序库生成真实场景，自动设计家具摆放位置。
From Programs to Poses: Factored Real-World Scene Generation via Learned Program Libraries
- 将房间结构与物体姿态分离建模，通过可复用的程序库生成布局。
- 在ScanNet数据上生成的场景与真实场景难以区分，姿态自然多样。
- 适合做虚拟现实、智能家居等需要逼真3D场景的应用。
真实世界场景（如ScanNet）采集困难，数据极为有限。生成具有多样化物体姿态的逼真3D场景仍是开放难题。本文提出FactoredScenes框架，通过挖掘房间的潜在结构并从生活化场景中学习物体姿态变化，实现真实场景合成。该框架采用分解式表示，将场景分解为分层组织的房间程序与物体姿态两部分。为编码结构，FactoredScenes从真实场景中学习一组可复用的布局函数构成程序库，并利用大语言模型生成高层级程序，由学习到的程序库进行正则化。为表达场景变化，该框架构建了一个程序条件化的模型，层级化预测物体姿态，并检索与放置3D物体。实验表明，FactoredScenes生成的场景在视觉上与真实ScanNet场景难以区分。
原文摘要 · Abstract (English)
Real-world scenes, such as those in ScanNet, are difficult to capture, with highly limited data available. Generating realistic scenes with varied object poses remains an open and challenging task. In this work, we propose FactoredScenes, a framework that synthesizes realistic 3D scenes by leveraging the underlying structure of rooms while learning the variation of object poses from lived-in scenes. We introduce a factored representation that decomposes scenes into hierarchically organized concepts of room programs and object poses. To encode structure, FactoredScenes learns a library of functions capturing reusable layout patterns from which scenes are drawn, then uses large language models to generate high-level programs, regularized by the learned library. To represent scene variations, FactoredScenes learns a program-conditioned model to hierarchically predict object poses, and retrieves and places 3D objects in a scene. We show that FactoredScenes generates realistic, real-world rooms that are difficult to distinguish from real ScanNet scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。