用粒子物理模拟神经网络特征传播，实现更高效自适应学习。
KITINet: Kinetics Theory Inspired Network Architectures with PDE Simulation Approaches
- 将残差模块建模为粒子系统演化，通过玻尔兹曼方程数值求解更新特征。
- 训练中参数自动浓缩到少数主导通道，提升模型效率与表达力。
- 在图像、文本、科学计算任务中均优于经典模型，计算量几乎不变。
尽管残差连接在现代神经网络中广受认可，但其设计原理仍主要依赖经验。本文提出KITINet（基于动力学理论的网络架构），将特征传播重新诠释为非平衡粒子动力学过程，并采用偏微分方程（PDE）仿真方法。核心是设计一种残差模块，将特征更新建模为粒子系统的随机演化，通过离散化的玻尔兹曼输运方程（BTE）求解器进行数值模拟，模拟粒子碰撞与能量交换，实现基于物理信息的自适应特征优化。此外，该机制在训练过程中诱导网络参数发生凝聚现象，使参数逐渐集中于少数主导通道。在科学计算（PDE算子）、图像分类（CIFAR-10/100）和文本分类（IMDb/SNLI）任务上的实验表明，相比经典网络基线，性能持续提升，且浮点运算量（FLOPs）几乎无增加。
原文摘要 · Abstract (English)
Despite the widely recognized success of residual connections in modern neural networks, their design principles remain largely heuristic. This paper introduces KITINet (Kinetics Theory Inspired Network), a novel architecture that reinterprets feature propagation through the lens of non-equilibrium particle dynamics and partial differential equation (PDE) simulation. At its core, we propose a residual module that models feature updates as the stochastic evolution of a particle system, numerically simulated via a discretized solver for the Boltzmann transport equation (BTE). This formulation mimics particle collisions and energy exchange, enabling adaptive feature refinement via physics-informed interactions. Additionally, we reveal that this mechanism induces network parameter condensation during training, where parameters progressively concentrate into a sparse subset of dominant channels. Experiments on scientific computation (PDE operator), image classification (CIFAR-10/100), and text classification (IMDb/SNLI) show consistent improvements over classic network baselines, with negligible increase of FLOPs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。