用2.5维分解让大模型精准建模三维积木结构
2.5-D Decomposition for LLM-Based Spatial Construction
- 将三维构建拆解为平面规划+确定性垂直填充,避开坐标错误
- 在160轮测试中达94.6%准确率,接近理论上限97.6%
- 可部署到边缘设备,延迟降低3倍至19.7秒
自主系统需根据自然语言指令构建结构,但大语言模型(LLMs)在生成三维积木位置时易出现系统性坐标错误。本文提出基于2.5维分解的神经符号流水线:LLM在二维水平面规划,确定性执行器根据列占用情况计算所有垂直位置,彻底消除一类错误。在Build What I Mean基准测试(160轮)中,GPT-4o-mini搭配该流水线实现94.6%的平均结构准确率,仅比由建筑师代理造成的3.0个百分点上限低。此结果优于GPT-4o的90.3%和最佳竞争系统76.3%。受控消融实验确认2.5维分解贡献28.7个百分点准确率提升。该流水线可直接迁移至边缘硬件:Nemotron-3 120B在NVIDIA Jetson Thor AGX上实现96.0%准确率,略超云端表现。通过增加四个目标示例使提示超出模型8,320令牌前缀缓存页大小,结合低开销推理,平均请求延迟降低3倍至19.7秒,准确率达95.6%。其核心原理——将确定性维度从LLM输出空间中移除——适用于任何受重力等物理约束固定自由度的自主建造或装配任务。在500个IGLU协作构建任务上的迁移实验验证了该方法的泛化能力。
原文摘要 · Abstract (English)
Autonomous systems that build structures from natural-language instructions need reliable spatial reasoning, yet large language models (LLMs) make systematic coordinate errors when generating three-dimensional block placements. We present a neuro-symbolic pipeline based on 2.5-D decomposition: the LLM plans in the two-dimensional horizontal plane while a deterministic executor computes all vertical placements from column occupancy, eliminating an entire class of errors. On the Build What I Mean benchmark (160 rounds), GPT-4o-mini with this pipeline achieves 94.6% mean structural accuracy across 12 independent runs, within 3.0 percentage points of the 97.6% ceiling imposed by architect-agent errors that no builder-side improvement can address. This outperforms both GPT-4o at 90.3% and the best competing system at 76.3%. A controlled ablation confirms that 2.5-D decomposition is the dominant contributor, accounting for 28.7 percentage points of accuracy. The pipeline transfers directly to edge hardware: Nemotron-3 120B on an NVIDIA Jetson Thor AGX achieves 96.0% mean structural accuracy with the identical pipeline, slightly exceeding the cloud result. Expanding the system prompt by four targeted examples to exceed the model's 8,320-token prefix cache page size, combined with low-effort reasoning, reduces mean per-request latency by 3X to 19.7 seconds at 95.6% accuracy. The underlying principle, removing deterministic dimensions from the LLM's output space, applies to any autonomous construction or assembly task where gravity or other physical constraints fix one or more degrees of freedom. A transfer experiment on 500 IGLU collaborative building tasks confirms the effect generalizes beyond the primary benchmark.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。