提出SEKF方法，在模型出现系统性漂移时在线更新权重，无需重新训练。
Staying Alive: Online Neural Network Maintenance and Systemic Drift
- 用损失梯度筛选需更新参数，仅通过SEKF调整这些参数。
- 在4个动态回归案例中，性能不逊于微调，迭代速度更快。
- 适合需要持续在线维护、资源受限的实时系统应用。
我们提出子集扩展卡尔曼滤波器（SEKF）方法，用于在模型所代表的系统发生偏离其原始训练条件时，对已训练模型权重进行在线更新，而非重新训练或微调。通过损失函数梯度识别需更新的参数，并仅使用SEKF更新这些参数。我们在四个动态回归案例中对比了微调与SEKF在系统性漂移下的在线模型维护表现，发现SEKF在保持模型准确率方面与微调相当甚至更优，同时每次迭代所需时间显著减少，且超参数调优需求更低。
原文摘要 · Abstract (English)
We present the Subset Extended Kalman Filter (SEKF) as a method to update previously trained model weights online rather than retraining or finetuning them when the system a model represents drifts away from the conditions under which it was trained. We identify the parameters to be updated using the gradient of the loss function and use the SEKF to update only these parameters. We compare finetuning and SEKF for online model maintenance in the presence of systemic drift through four dynamic regression case studies and find that the SEKF is able to maintain model accuracy as-well if not better than finetuning while requiring significantly less time per iteration, and less hyperparameter tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。