动态门控MLP让模型边推理边学习，且结果稳定。
Learning Inference Concurrency in DynamicGate MLP Structural and Mathematical Justification
- 分离路由与预测参数，实现推理时在线更新
- 即使异步或部分更新，输出仍对应有效模型快照
- 适合需要实时自适应的设备端学习系统
传统神经网络严格区分学习与推理，因推理时参数更新会导致输出不稳定，甚至使推理函数无法定义。本文证明动态门控MLP在结构上允许学习与推理并发。核心思想是将路由（门控）参数与表示（预测）参数分离，使得门控可在线调整，同时保持推理稳定性；或仅在非活跃子空间内选择性更新权重。我们数学形式化了并发的充分条件，并证明即便在异步或部分更新下，每一步的推理输出始终可解释为某个有效模型快照的前向计算。这表明动态门控MLP可作为在线自适应与设备端学习系统的实用基础。
原文摘要 · Abstract (English)
Conventional neural networks strictly separate learning and inference because if parameters are updated during inference, outputs become unstable and even the inference function itself is not well defined [1, 2, 3]. This paper shows that DynamicGate MLP structurally permits learning inference concurrency [4, 5]. The key idea is to separate routing (gating) parameters from representation (prediction) parameters, so that the gate can be adapted online while inference stability is preserved, or weights can be selectively updated only within the inactive subspace [4, 5, 6, 7]. We mathematically formalize sufficient conditions for concurrency and show that even under asynchronous or partial updates, the inference output at each time step can always be interpreted as a forward computation of a valid model snapshot [8, 9, 10]. This suggests that DynamicGate MLP can serve as a practical foundation for online adaptive and on device learning systems [11, 12].
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。