arXiv:2504.01786cs.GRcs.LG2025-04CVPR被引 27

首个3D图形编辑基准,测试视觉语言模型真实场景下的编辑能力

BlenderGym: Benchmarking Foundational Model Systems for Graphics Editing

  • 用代码驱动的3D重建任务评估VLM系统性能
  • 顶尖VLM在基础操作上仍远逊于人类用户
  • 推理扩展可提升生成与验证环节,策略分配计算资源更高效

3D图形编辑在影视制作和游戏设计中至关重要,但过程耗时且依赖高度专业技能。自动化面临挑战,因图形编辑需涵盖多种不同技能的任务。近年来,视觉语言模型(VLM)成为自动化编辑的有力工具,但其发展与评估受限于缺乏能体现人类级感知、具备真实编辑复杂性的综合基准。本文提出BlenderGym,首个面向3D图形编辑的VLM系统综合基准。该基准通过代码基3D重建任务评估VLM系统。我们测试了闭源与开源VLM系统,发现即使最先进的VLM在对人类Blender用户而言相对简单的任务上仍表现不佳。借助BlenderGym,我们研究了推理扩展技术对VLM在图形编辑任务中性能的影响。结果表明,用于引导生成扩展的验证器本身也可通过推理扩展得到提升,这补充了近期关于大模型在编码与数学任务中推理扩展的发现。此外,我们证明推理算力并非均匀有效,可通过合理分配生成与验证阶段的计算资源实现优化。

原文摘要 · Abstract (English)

3D graphics editing is crucial in applications like movie production and game design, yet it remains a time-consuming process that demands highly specialized domain expertise. Automating this process is challenging because graphical editing requires performing a variety of tasks, each requiring distinct skill sets. Recently, vision-language models (VLMs) have emerged as a powerful framework for automating the editing process, but their development and evaluation are bottlenecked by the lack of a comprehensive benchmark that requires human-level perception and presents real-world editing complexity. In this work, we present BlenderGym, the first comprehensive VLM system benchmark for 3D graphics editing. BlenderGym evaluates VLM systems through code-based 3D reconstruction tasks. We evaluate closed- and open-source VLM systems and observe that even the state-of-the-art VLM system struggles with tasks relatively easy for human Blender users. Enabled by BlenderGym, we study how inference scaling techniques impact VLM's performance on graphics editing tasks. Notably, our findings reveal that the verifier used to guide the scaling of generation can itself be improved through inference scaling, complementing recent insights on inference scaling of LLM generation in coding and math tasks. We further show that inference compute is not uniformly effective and can be optimized by strategically distributing it between generation and verification.

3D编辑视觉语言模型基准测试推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。