让神经网络自带可学习的节律振荡，提升连续时间序列处理的抗干扰能力。
Pulse-Driven Neural Architecture: Learnable Oscillatory Dynamics for Robust Continuous-Time Sequence Processing
- 引入脉冲模块生成可学习相位与频率的正弦振荡，保持内部状态独立演化。
- 在断续输入下，自注意力变体比基线提升2.78个百分点，脉冲变体提升4.62个百分点。
- 结构化振荡机制优于随机噪声，适合需要高鲁棒性的时序建模任务。
我们提出PDNA（脉冲驱动神经架构），一种增强连续时间循环网络的方法，通过可学习的振荡动态维持内部状态演化，不依赖外部输入。基于闭式连续时间（CfC）网络，PDNA新增两个组件：(1) 脉冲模块生成结构化振荡 $A \cdot \sin(ωt + φ(h))$，具有可学习频率和状态依赖相位；(2) 自注意力模块对隐藏状态应用递归自注意力。在五组随机种子下的顺序MNIST（sMNIST）上进行消融实验，评估间隙鲁棒性——即测试时部分输入被移除后仍保持性能的能力。关键发现是：结构化振荡显著提升对输入中断的鲁棒性：自注意力变体相比基线取得2.78个百分点的多间隙优势（$p = 0.041$），脉冲变体则达到4.62个百分点的优势，效应量较大（Cohen's $d = 0.87$）。噪声控制（等幅随机扰动）无提升，确认优势源于结构而非单纯动态。结果表明，连续时间模型可通过类生物振荡机制增强时间鲁棒性。
原文摘要 · Abstract (English)
We introduce PDNA (Pulse-Driven Neural Architecture), a method for augmenting continuous-time recurrent networks with learnable oscillatory dynamics that maintain internal state evolution independently of external input. Built on Closed-form Continuous-time (CfC) networks, PDNA adds two components: (1) a pulse module that generates structured oscillations $A \cdot \sin(ωt + φ(h))$ with learnable frequencies and state-dependent phase, and (2) a self-attend module that applies recurrent self-attention to the hidden state. Through a controlled ablation study on sequential MNIST (sMNIST) with five random seeds, we evaluate gap robustness -- the ability to maintain performance when portions of the input sequence are removed at test time. Our key finding is that structured oscillatory dynamics significantly improve robustness to input interruptions: the self-attend variant achieves a statistically significant 2.78 percentage point multi-gap advantage over baseline ($p = 0.041$), while the pulse variant shows a 4.62 pp advantage with large effect size (Cohen's $d = 0.87$). A noise control (random perturbation of equal magnitude) provides no benefit, confirming that the advantage is structural rather than merely dynamic. These results provide evidence that continuous-time models can benefit from biologically-inspired internal oscillatory mechanisms for temporal robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。