arXiv:2606.09278cs.LGcs.AI2026-06

解决几何生成中约束冲突导致的模型失效问题,提升精密设计生成准确性。

Internalizing Geometric Law: Learning from Solver Residuals for Precision-Critical Generation

论文配图:Internalizing Geometric Law: Learning from Solver Residuals for Precision-Critical Generation
图 1 · 摘自论文原文
  • 用可微分约束语言PyGeoX将几何规则转为损失函数,实现精准验证
  • 发现单个异常约束会掩盖其他约束信号,导致训练失效
  • 提出分段饱和奖励机制,显著提升复杂几何问题求解率

大语言模型在技术绘图与机械设计等精度敏感领域常产生幻觉,其输出需满足数十个相互关联的几何约束。本文研究从自然语言生成开放式的精确几何构造,提出PyGeoX这一可编程几何领域特定语言,能将声明式约束编译为可微分损失;同时发布包含300个分层问题的PyGeoX-Bench基准,支持逐约束验证。通过该工具发现一种名为‘离群梯度屏蔽’的失效模式:在全局范数奖励(如$\ ext{exp}(-\mathrm{MSE})$)下,单个异常约束会使所有其他约束的学习信号消失。为此提出饱和加性奖励(SAR),将奖励分解为有界个体项,保留部分进展并维持稳定梯度。相比基于MSE的基线,SAR使难题层级求解率提升2.3倍,所训练的8B模型在该基准上达到远超自身规模的性能,媲美更大规模前沿系统。代码、数据与基准已开源。

原文摘要 · Abstract (English)

Large Language Models frequently hallucinate in precision-critical domains such as technical diagramming and mechanical design, where outputs must satisfy strict geometric constraints. We study open-ended geometric synthesis from natural language: translating free-form descriptions into precise constructions whose entities must simultaneously satisfy dozens of interacting constraints. To make this tractable, we release PyGeoX, a programmable geometric DSL that compiles declarative constraints into a differentiable loss, and PyGeoX-Bench, a stratified suite of 300 problems with per-constraint verifiable rewards. Using PyGeoX as a verifier, we identify a failure mode we call Outlier Gradient Masking: under global-norm rewards (any scheme that aggregates residuals through a single norm, for example, $\exp(-\mathrm{MSE})$), a single outlier constraint can nullify the learning signal across all others. To address this, we propose Saturating Additive Rewards (SAR), which decompose the reward into bounded per-constraint terms, preserving partial progress and ensuring consistent gradients even under severe violations. Against MSE-based rewards, the natural baseline for geometry solvers, SAR improves the hard-tier solving rate by $2.3\times$, and the resulting 8B model is competitive with much larger frontier systems on this benchmark. We release the engine, benchmark, and data at https://github.com/Huawei-AI4Math/PyGeoX.

几何生成可微分推理约束求解语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。