arXiv:2501.16932cs.LG2025-01中稿 · publication in Sci…被引 3

提出高效在线学习框架,解决模型精度与更新速度难题

Online-BLS: An Accurate and Efficient Online Broad Learning System for Data Stream Classification

  • 用乔里斯基分解替代矩阵求逆,提升权重估计精度
  • 设计轻量更新策略,显著降低在线学习时间开销
  • 适用于存在概念漂移的数据流场景,性能超越现有方法

现有在线学习模型在新样本到达时仅执行一次梯度下降,导致模型权重次优。为此,本文提出一种具有闭式解的在线广义学习系统框架。不同于传统增量广义学习算法在在线任务中常导致精度下降和高更新开销,我们分别设计了有效的权重估计算法和高效的在线更新策略以解决上述问题。首先,通过将复杂的矩阵求逆操作替换为乔里斯基分解与前向-后向代入,提升了模型精度;其次,提出一种高效的在线更新机制,大幅减少更新时间。理论分析表明,该模型具备优异误差界和低时间复杂度。在多个真实数据集上的测试-训练评估验证了其优越性与高效性。此外,本框架可自然扩展至存在概念漂移的数据流场景,性能优于当前最优基线。

原文摘要 · Abstract (English)

The state-of-the-art online learning models generally conduct a single online gradient descent when a new sample arrives and thus suffer from suboptimal model weights. To this end, we introduce an online broad learning system framework with closed-form solutions for each online update. Different from employing existing incremental broad learning algorithms for online learning tasks, which tend to incur degraded accuracy and expensive online update overhead, we design an effective weight estimation algorithm and an efficient online updating strategy to remedy the above two deficiencies, respectively. Specifically, an effective weight estimation algorithm is first developed by replacing notorious matrix inverse operations with Cholesky decomposition and forward-backward substitution to improve model accuracy. Second, we devise an efficient online updating strategy that dramatically reduces online update time. Theoretical analysis exhibits the splendid error bound and low time complexity of our model. The most popular test-then-training evaluation experiments on various real-world datasets prove its superiority and efficiency. Furthermore, our framework is naturally extended to data stream scenarios with concept drift and exceeds state-of-the-art baselines.

在线学习广义学习数据流高效更新

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。