arXiv:2510.03303math.OCcs.LG2025-10被引 2

用控制理论解析神经网络,揭示深度与宽度的权衡机制。

Machine Learning and Control: Foundations, Advances, and Perspectives

  • 以控制理论视角分析深度网络的可分类性与表征能力。
  • 提出混合协作学习框架,融合力学模型与数据驱动方法。
  • 解释生成AI成功背后的扩散过程原理,适合跨学科研究者。

控制理论为解决深度神经网络及其他机器学习架构中的挑战提供了强大框架。我们发现,同时可控性和集合可控性等概念能为深度神经网络的分类与表征特性提供新见解;静态系统的控制与优化可用于更好地理解浅层网络的性能。受经典折返定理启发,我们探索了动态与静态神经网络之间的关系,揭示了深度与宽度的权衡,并阐明了Transformer在加速传统神经网络任务中的作用。此外,利用神经网络的表达能力(如通用逼近定理),我们开发了一种新的混合建模方法——混合协作学习(HYCO),在博弈论框架下结合力学模型与数据驱动方法。最后,我们阐述了扩散过程的经典性质(长期建立于偏微分方程中)如何解释现代生成式人工智能的成功。本文综述了我们在这些领域的近期成果,展示了控制、机器学习、数值分析与偏微分方程如何交汇,为未来研究提供丰富土壤。

原文摘要 · Abstract (English)

Control theory of dynamical systems offers a powerful framework for tackling challenges in deep neural networks and other machine learning architectures. We show that concepts such as simultaneous and ensemble controllability offer new insights into the classification and representation properties of deep neural networks, while the control and optimization of static systems can be employed to better understand the performance of shallow networks. Inspired by the classical concept of turnpike, we also explore the relationship between dynamic and static neural networks, where depth is traded for width, and the role of transformers as mechanisms for accelerating classical neural network tasks. We also exploit the expressive power of neural networks (exemplified, for instance, by the Universal Approximation Theorem) to develop a novel hybrid modeling methodology, the Hybrid-Cooperative Learning (HYCO), combining mechanics and data-driven methods in a game-theoretic setting. Finally, we describe how classical properties of diffusion processes, long established in the context of partial differential equations, contribute to explaining the success of modern generative artificial intelligence (AI). We present an overview of our recent results in these areas, illustrating how control, machine learning, numerical analysis, and partial differential equations come together to motivate a fertile ground for future research.

控制理论神经网络生成模型混合建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。