arXiv:2606.09930cs.PLcs.LG2026-06

让代码自执行并可微分,实现程序与参数的联合优化。

Compile Once, Differentiate Everywhere: A Differentiable Meta-Circular Interpreter

  • 用自宿主的Scheme子集编译出可微计算图,支持递归和闭包。
  • 在171组递归与高阶程序上,梯度精度与直接编译一致。
  • 适合需要自动优化可执行程序的科研与工程场景。

程序执行与基于梯度的优化之间的界限长期限制了代码作为可学习科学模型的应用。我们提出一个编译器,将自宿主的Scheme子集转换为autograd后端的可微计算图。由于该子集能编译自身的求值器,从而实现可微分元循环解释(DMCI):已编译的Scheme解释器以数据形式执行程序,反向模式自动微分将梯度传播至嵌入其中的连续常量。解释器仅需编译一次,新程序即可继承可微性,无需重新编译或定制梯度机制,同时保留闭包、递归和数据结构。我们证明了编译后解释器的梯度几乎处处正确,并在171对递归与高阶程序-种子中验证其梯度与直接编译达到数值精度一致。随后,我们使用DMCI进行程序与参数联合搜索:大语言模型生成Scheme程序,精确梯度通过单一冻结的解释器校准其连续参数。这实现了OpenEvolve风格的程序搜索,外层循环提出离散程序结构,而DMCI提供每个候选程序的精确梯度校准。在电池容量衰减数据上,搜索恢复了拐点式退化结构,在更难的早期外推划分上优于人工基线;在高维厄尔尼诺逆问题中,DMCI优化了解释型卡尔曼滤波似然,而梯度无关搜索失败。这些结果将符号回归与神经符号搜索从封闭表达式扩展到可执行的状态化程序,使模型生成的代码可直接基于数据优化。

原文摘要 · Abstract (English)

The boundary between program execution and gradient-based optimization has long limited the use of code itself as a learnable scientific model. We present a compiler that translates a self-hosting subset of Scheme into differentiable computation graphs for autograd backends. Because the subset can compile its own evaluator, this yields differentiable meta-circular interpretation (DMCI): a compiled Scheme interpreter executes programs supplied as data, while reverse-mode autodiff propagates gradients to continuous constants embedded in those programs. The interpreter is compiled once, so new programs inherit differentiability without recompilation or custom gradient machinery, while retaining closures, recursion, and data structures. We prove that gradients through the compiled interpreter are correct almost everywhere and show that they match direct compilation to numerical precision across 171 recursive and higher-order program-seed pairs. We then use DMCI for program-and-parameter co-search, where a large language model proposes Scheme programs and exact gradients calibrate their continuous parameters through a single frozen interpreter. This enables OpenEvolve-style program search in which an outer loop proposes discrete program structures and DMCI supplies exact gradient-based calibration of each candidate's continuous parameters. On battery capacity-fade data, the search recovers a knee-like degradation structure and improves held-out extrapolation over hand-crafted baselines on the harder early-extrapolation split, matching them on the later split. On a high-dimensional El Nino inverse problem, DMCI optimizes an interpreted Kalman-filter likelihood where gradient-free search fails. These results extend symbolic regression and neurosymbolic search from closed-form expressions to executable, stateful programs, making model-generated code directly optimizable against data.

可微编程程序搜索符号回归自动微分

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。