arXiv:2604.18907cs.LGcs.AI2026-04被引 2

用可微分语言让神经网络自动学会编程,边学边调,适合复杂任务快速适应。

Gradient-Based Program Synthesis with Neurally Interpreted Languages

论文配图:Gradient-Based Program Synthesis with Neurally Interpreted Languages
图 1 · 摘自论文原文
  • 自动生成符号化编程语言,通过可微执行器处理变长程序序列
  • 在组合泛化任务上优于上下文学习和测试时训练,实现零样本迁移
  • 支持推理时梯度优化,适合快速适应新任务的场景

程序归纳中的核心挑战是符号方法与神经方法之间的权衡。符号方法具备组合泛化与数据高效性,但受限于领域特定语言(DSL)等形式系统,其构建耗时且难以迁移。神经网络虽能灵活从数据中学习,但在组合及分布外场景下泛化能力差。本文提出一种名为神经语言解释器(NLI)的潜在适配网络架构,端到端学习自有的离散、类符号编程语言。NLI 自主发现基本操作词汇,并使用新型可微神经执行器解释可变长度的指令序列,从而表示不固定计算步数的程序,解决训练中未见的更复杂问题。为使离散程序结构适用于梯度优化,采用 Gumbel-Softmax 放松方法,实现全模型端到端训练。关键在于该可微性支持强大的推理时适配:在测试阶段,程序生成器提供初始程序猜测,再通过神经执行器进行梯度下降优化,高效搜索最佳解释数据的神经程序。实验表明,NLI 在需要组合泛化的任务上,显著优于上下文学习、测试时训练及连续潜空间程序网络。结果确立了一条融合离散语言组合性与神经网络梯度搜索能力的新路径。

原文摘要 · Abstract (English)

A central challenge in program induction has long been the trade-off between symbolic and neural approaches. Symbolic methods offer compositional generalisation and data efficiency, yet their scalability is constrained by formalisms such as domain-specific languages (DSLs), which are labour-intensive to create and may not transfer to new domains. In contrast, neural networks flexibly learn from data but tend to generalise poorly in compositional and out-of-distribution settings. We bridge this divide with an instance of a Latent Adaptation Network architecture named Neural Language Interpreter (NLI), which learns its own discrete, symbolic-like programming language end-to-end. NLI autonomously discovers a vocabulary of primitive operations and uses a novel differentiable neural executor to interpret variable-length sequences of these primitives. This allows NLI to represent programs that are not bound to a constant number of computation steps, enabling it to solve more complex problems than those seen during training. To make these discrete, compositional program structures amenable to gradient-based optimisation, we employ the Gumbel-Softmax relaxation, enabling the entire model to be trained end-to-end. Crucially, this same differentiability enables powerful test-time adaptation. At inference, NLI's program inductor provides an initial program guess. This guess is then refined via gradient descent through the neural executor, enabling efficient search for the neural program that best explains the given data. We demonstrate that NLI outperforms in-context learning, test-time training, and continuous latent program networks on tasks that require combinatorial generalisation and rapid adaptation to unseen tasks. Our results establish a new path toward models that combine the compositionality of discrete languages with the gradient-based search and end-to-end learning of neural networks.

程序合成可微编程组合泛化神经符号

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。