arXiv:2606.20364cs.LG2026-06

用视觉模型当裁判,让3D生成模型更懂家具质量。

Judging to Improve: A De-biased VLM-as-3D-Judge Protocol for Single-Image 3D Generation

  • 用双模型裁判机制避免评估循环,提升判断可靠性。
  • 在家具类3D生成中,仅靠轻量适配无法超越基础模型性能。
  • 裁判协议可复用,适合无标注数据下的高效模型优化。

一项配套研究构建了去偏、跨模型的VLM作为3D裁判,能可靠评估单图生成3D网格的质量,而传统几何与CLIP代理则表现不佳。本文提出:能否将该裁判用于优化强开源生成器TRELLIS,在无需人工标注且成本低廉的前提下,专精于家具类别?核心在于将裁判从评分转向优化。将VLM裁判引入训练与评估流程后,暴露了评分从未触发的失败模式。因此,本文贡献在于构建一个优化级强化的裁判协议:训练裁判(Qwen2.5-VL-7B)与评估裁判(InternVL3-8B)分离以避免循环;修正位置偏差;修复三类失败模式(图像过载、几何隐藏点云渲染、无参考评判导致奖励干净但错误输出)。校准证据显示,清晰差距胜率在0.83–1.0之间,基线对基线约0.5。以该协议为独立评估器,仅使用公开模型与数据,配合轻量参数高效适应,结果表明方法仅能匹配基线,未实现超越。独立基线样本几乎无学习偏好(0.94顺序翻转率),说明信号必须通过质量对比构造。六种适配方法、两种输入方式及严重度扫描下,最针对性的方法(严重退化时的条件修复)达到与基线相当的0.50胜率,但无方法达到≥65%胜率目标。结果表明:清洁输入使裁判饱和,flow-DIT微调被采样器稀释,而条件修复是影响几何的关键。在n=8个物体上,胜率具有方向性。以低成本适配匹配强公共数据基线本身即具信息量:超越它需要的不只是公共数据上的轻量PEFT,且该裁判协议具备可复用性。

原文摘要 · Abstract (English)

A companion study established a de-biased, cross-model VLM-as-3D-judge that reliably ranks single-image-to-3D mesh quality where cheap geometry and CLIP proxies fall short. This paper asks: can that judge's preferences specialize a strong open generator, TRELLIS, on one asset class (furniture), cheaply and without human labels? Taking the judge from ranking to optimization is where the work lives. Pushing a VLM judge into the training and evaluation loop exposes failure modes ranking never triggered, so our contribution is an optimization-grade hardening of the judge: a training judge (Qwen2.5-VL-7B) held distinct from an evaluation judge (InternVL3-8B) to break circularity; position-bias correction; and fixes for three failure modes (image overload, geometry-hiding splat renders, and reference-free judging that rewards clean-but-wrong outputs), with calibration evidence (clear-gap win-rate 0.83-1.0; base-vs-base ~0.5). Using this protocol as an independent evaluator, and working only from public models and data with lightweight parameter-efficient adaptation, we find our methods match the strong base rather than exceed it. Independent base samples carry essentially no learnable preference (0.94 order-flip rate), so signal must be engineered by quality-contrastive construction. Across six adaptation methods, two input regimes, and a severity sweep, the most targeted - conditioner repair under severe degradation - reaches parity (0.50) with the base, while no method clears the >=65% win-rate target. The result is mechanistic: clean inputs saturate the judge, flow-DIT fine-tuning washes out through the sampler, and conditioning repair is the locus that moves geometry. Win-rates are directional at n=8 objects. Matching a strong public-data base with cheap adaptation is itself informative: exceeding it needs more than lightweight PEFT on public data, and the judge protocol is reusable.

3D生成视觉模型模型优化去偏评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。