arXiv:2411.12876cs.LGcs.AI2024-11被引 1

让卷积层参数像流水一样连续演化,按输入难易自动调节计算量。

Puppet-CNN: Continuous Parameter Dynamics for Input-Adaptive Convolutional Networks

  • 用神经微分方程建模参数连续变化,不再固定层数
  • 参数量减少一半以上,准确率仍保持竞争力
  • 适合需要自适应计算的轻量化部署场景

现代卷积神经网络将计算组织为离散的层堆叠,每层参数独立存储与学习,层数作为固定超参数。本文提出一种新视角:能否将网络参数化本身视为连续动态系统?我们引入Puppet-CNN框架,将卷积层参数表示为在神经普通微分方程(ODE)控制下的参数流状态。在此设定下,层参数通过参数空间中的连续演化生成,有效层数由学习到的动力学积分范围决定,可随输入复杂度调节,实现输入自适应计算。我们在标准图像分类基准上验证该方法,结果表明连续参数动力学可在显著降低可训练参数量的同时,达到具有竞争力的预测性能。这表明从动力系统角度看待神经网络参数化,为自适应卷积模型提供了结构化且灵活的设计空间。

原文摘要 · Abstract (English)

Modern convolutional neural networks (CNNs) organize computation as a discrete stack of layers whose parameters are independently stored and learned, with the number of layers fixed as an architectural hyperparameter. In this work, we explore an alternative perspective: can network parameterization itself be modeled as a continuous dynamical system? We introduce Puppet-CNN, a framework that represents convolutional layer parameters as states evolving along a learned parameter flow governed by a neural ordinary differential equation (ODE). Under this formulation, layer parameters are generated through continuous evolution in parameter space, and the effective number of generated layers is determined by the integration horizon of the learned dynamics, which can be modulated by input complexity to enable input-adaptive computation. We validate this formulation on standard image classification benchmarks and demonstrate that continuous parameter dynamics can achieve competitive predictive performance while substantially reducing stored trainable parameters. These results suggest that viewing neural network parameterization through the lens of dynamical systems provides a structured and flexible design space for adaptive convolutional models.

连续模型自适应计算微分方程

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。