将连续时间丢弃法建模为随机批次方法，实现高效且可证明的训练优化。
Convergence, design and training of continuous-time dropout as a random batch method
- 用时间区间内采样神经元批次模拟丢弃，构建无偏估计器。
- 轨迹误差以线性速率随h收敛，分布误差为h^{1/2}阶。
- 理论指导最优采样间隔选择，实测兼具速度与内存优势。
我们从随机批次方法的角度研究连续时间模型中的丢弃正则化——一种最初用于降低相互作用粒子系统计算成本的随机采样方案。我们构造了一个无偏且适定的估计器，通过在长度为$h$的时间区间内采样神经元批次来模拟丢弃。在轨迹层面，建立了期望一致误差的线性收敛率（关于$h$）。在分布层面，对关联的连续性方程建立了稳定性，总变差误差在温和矩假设下为$h^{1/2}$阶。在固定批次采样的训练过程中，基于庞特里亚金伴随分析，给出了最优代价、控制及梯度下降迭代偏差的界。在设计方面，比较了典型批次采样方案的收敛速率，将标准伯努利丢弃作为特例恢复，并推导出一个闭式表达的代价-精度权衡以确定最优$h$。随后，我们将方法应用于单层神经微分方程，在分类和流匹配任务上验证了理论预测的收敛速率、正则化效果以及有利的运行时和内存表现。
原文摘要 · Abstract (English)
We study dropout regularization in continuous-time models through the lens of random-batch methods -- a family of stochastic sampling schemes originally devised to reduce the computational cost of interacting particle systems. We construct an unbiased, well-posed estimator that mimics dropout by sampling neuron batches over time intervals of length $h$. Trajectory-wise convergence is established with linear rate in $h$ for the expected uniform error. At the distribution level, we establish stability for the associated continuity equation, with total-variation error of order $h^{1/2}$ under mild moment assumptions. During training with fixed batch sampling across epochs, a Pontryagin-based adjoint analysis bounds deviations in the optimal cost and control, as well as in gradient-descent iterates. On the design side, we compare convergence rates for canonical batch sampling schemes, recover standard Bernoulli dropout as a special case, and derive a cost--accuracy trade-off yielding a closed-form optimal $h$. We then specialize to a single-layer neural ODE and validate the theory on classification and flow matching, observing the predicted rates, regularization effects, and favorable runtime and memory profiles.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。