VALG用智能体系统自动推导机器学习理论定理,提升研究效率。
VALG: An Agentic System for ML Theory Research
- 构建多层级验证与图结构证明的智能体系统,自主推进理论研究
- 在9个COLT 2026开放问题中,产出匹配原题范围的定理或条件结论
- 支持假设调整、证明重构与数学关系保持,适合理论研究者使用
机器学习理论通过数学设定(如数据模型、训练协议、损失函数等)研究学习过程。解决开放问题需同步设计问题形式、定理目标与证明机制。研究人员提出假设,经初步理论或实验分析验证后不断修正假设与证明。本文探讨该过程能否被组织为自主的智能体工作流。我们开发了VALG——一个结合多级验证、自适应问题建模与图结构证明构建的智能体系统。在每个源自原始问题的定理分支中,VALG维持固定数学设定,检查类型化的证明依赖图结构,并按依赖顺序构造与评审局部证明。当证明失败时,系统识别障碍来源:推导错误、证明结构缺陷或定理表述不当,并相应调整后续尝试。若属形式层障碍,则生成相关变体或松弛形式,保持新定理与源问题的数学关联。在5个COLT 2026开放问题中的9个子问题上评估,两次运行产生与原题范围一致的定理候选;其余七例得到受限方法结果、特例或条件定理。案例显示,VALG能数学区分源问题匹配、松弛、条件结果与受阻尝试。代码开源:https://github.com/DechenZhang/VALG-ML-Theory-Agent。
原文摘要 · Abstract (English)
Machine learning theory studies learning procedures through mathematical setups in which the data model, training protocol, oracle access, loss, metric, and randomness define the phenomenon that a theorem is meant to explain. Solving an open problem therefore requires the problem formulation, theorem target, and proof mechanism to be developed in concert. Researchers formulate hypotheses, test them through preliminary theoretical or empirical analysis, and refine both assumptions and proofs. We investigate whether this process can be organized as an autonomous agentic workflow for ML theory research. We develop VALG, an agentic system that combines multi-level Verification, Adaptive formulation of Learning-theory problems, and Graph-structured proof development. Within each source-relative theorem branch, VALG maintains a fixed mathematical specification, checks the theorem-level composition of a typed proof-dependency graph, and constructs and reviews local proofs in dependency order. When a proof attempt fails, VALG identifies whether the obstruction lies in a derivation, the proof structure, or the theorem formulation and routes the next attempt accordingly. Formulation-level obstructions initiate an explicitly related variant or relaxation, preserving the mathematical relation between the resulting theorem and the source problem. We evaluate VALG on nine subproblems from five COLT 2026 open problems. Two runs produce internally finalized theorem candidates that match the scope of their source briefs; the remaining seven yield restricted-method results, special cases, or conditional theorems. These case studies show how VALG keeps source-scope matches, relaxations, conditional results, and blocked attempts mathematically distinct. VALG is open source at https://github.com/DechenZhang/VALG-ML-Theory-Agent.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。