用程序生成+自训练,让模型更准地预测家具摆放位置。
Learning to Place Objects with Programs and Iterative Self Training
- 设计领域语言描述物体间空间约束,生成候选摆放位置。
- 自训练算法提升在少量数据下的表现,比直接训练好37%以上。
- 适合室内布局、机器人规划等需要精准摆放的场景。
本文研究室内场景中的物体放置问题。给定一个3D室内场景和一个物体,目标是预测该物体在场景内的合理放置位置。现有数据驱动方法常遗漏合理的放置模式。为此,我们设计一种领域特定语言(DSL),用于表达物体间的空间关系约束;执行程序可从部分场景和物体信息中预测可能的放置位置。我们构建生成模型自动编写此类程序。由于现有3D场景数据集缺乏标注程序,且直接提取的程序仅能复现原始位置,导致训练效果不佳。因此我们提出新的程序自举算法,显著提升系统性能。为量化评估,我们引入新评测方法:请人工标注物体在场景中所有可能的放置位置,并与系统输出对比。实验表明,我们的系统生成的位置分布更贴近人类判断,优于现有数据驱动方法及零样本大模型方法。在训练数据稀疏时,其他系统性能明显下降,而本系统仍保持稳定表现。
原文摘要 · Abstract (English)
In this work we study indoor scene object placement. Given a 3D indoor scene and an object, the task is to predict placement locations within the scene. Empirical observations of data-driven approaches to the problem show their tendency to miss placement modes. We introduce a system which helps to address this flaw. We design a Domain Specific Language (DSL) that specifies object relational constraints. Upon execution, programs from our language predict possible placements from a partial scene and object. We design a generative model which writes these programs automatically. Available 3D scene datasets do not contain programs to train on, and naively extracted programs only predict the original placement location of scene objects. Training on these programs results in subpar performance so we introduce a new program bootstrapping algorithm that improves our system's performance compared to the naive approach. To quantify our qualitative observations, we introduce a new evaluation procedure which captures how well a system models per-object location distributions. We ask human annotators to label all the possible places an object can go in a scene and compare this set against locations produced by the system in question. Our system produces per-object location distributions more consistent with human annotators than those produced by existing data-driven approaches and a zero-shot approach using an LLM. While other systems degrade in performance when training data is sparse, our system does not degrade to the same degree.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。