自动将领域理论转化为可证明正确的模型架构
From Specification to Architecture: A Theory Compiler for Knowledge-Guided Machine Learning
- 用机器可读的理论语言自动生成符合该理论的模型结构
- 生成的架构在函数空间上严格满足理论约束,无需正则化
- 适合需要高可靠性与泛化能力的科学建模场景
知识引导的机器学习表明,将真实的领域知识直接融入模型设计可提升性能、样本效率和分布外泛化能力。然而,将形式化领域理论转化为架构约束的过程仍完全依赖人工,针对特定领域形式化且缺乏形式正确性保证,难以跨领域迁移,无法验证,也不具可扩展性。本文提出理论编译器(Theory Compiler):接收类型化的、机器可读的领域理论作为输入,自动生成其函数空间在构造上严格符合该理论的模型架构,而非通过正则化实现。我们识别出三个基础性开放问题:(1) 设计具备可判定类型检查的通用理论形式化语言;(2) 构建从理论原子到架构模块的组合正确编译算法;(3) 建立形式验证的合理性和完备性标准。我们进一步推测,编译生成的架构在泛化性能上可媲美甚至超越手工设计,且所需训练数据显著更少,这一主张基于经典统计学习理论。我们认为,近年来形式化机器学习理论、大语言模型及跨学科研究共同体的发展,使这一范式首次成为现实。
原文摘要 · Abstract (English)
Theory-guided machine learning has demonstrated that including authentic domain knowledge directly into model design improves performance, sample efficiency and out-of-distribution generalisation. Yet the process by which a formal domain theory is translated into architectural constraints remains entirely manual, specific to each domain formalism, and devoid of any formal correctness guarantee. This translation is non-transferable between domains, not verified, and does not scale. We propose the Theory Compiler: a system that accepts a typed, machine-readable domain theory as input and automatically produces an architecture whose function space is provably constrained to be consistent with that theory by construction, not by regularisation. We identify three foundational open problems whose resolution defines our research agenda: (1) designing a universal theory formalisation language with decidable type-checking; (2) constructing a compositionally correct compilation algorithm from theory primitives to architectural modules; and (3) establishing soundness and completeness criteria for formal verification. We further conjecture that compiled architectures match or exceed manually-designed counterparts in generalisation performance while requiring substantially less training data, a claim we ground in classical statistical learning theory. We argue that recent advances in formal machine learning theory, large language models, and the growth of an interdisciplinary research community have made this paradigm achievable for the first time.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。