arXiv:2506.12389cs.LGcs.AI2025-06KDD被引 3

提出动态重置机制,让神经网络推荐模型更灵活适应用户偏好变化。

Revisiting Clustering of Neural Bandits: Selective Reinitialization for Mitigating Loss of Plasticity

  • 根据贡献度筛选并重置低效神经单元,保持模型可塑性。
  • 在六个真实数据集上,累计损失降低30%以上,适应性显著提升。
  • 适合需要长期自适应的推荐系统、动态环境下的强化学习场景。

聚类强化学习(CB)方法通过基于相似性分组并引入群体上下文信息,提升了个性化推荐等序列决策任务的表现。然而,其神经版本(称为神经聚类强化学习,CNB)在长期运行中会出现参数僵化、可塑性下降的问题,难以应对非平稳环境(如用户偏好的动态变化)。为此,本文提出选择性重置(SeRe)框架,通过贡献度度量识别并选择性重置使用率低的神经单元,在保留知识稳定性的同时恢复模型灵活性。结合自适应变化检测机制,SeRe可根据环境非平稳程度动态调整重置频率,避免过度重置。理论上证明,SeRe可在分段平稳环境中实现次线性累积损失,优于传统CNB方法。六组真实推荐数据集上的实验表明,采用SeRe的CNB算法显著缓解了可塑性丧失问题,累积损失更低,适应性与鲁棒性更强。

原文摘要 · Abstract (English)

Clustering of Bandits (CB) methods enhance sequential decision-making by grouping bandits into clusters based on similarity and incorporating cluster-level contextual information, demonstrating effectiveness and adaptability in applications like personalized streaming recommendations. However, when extending CB algorithms to their neural version (commonly referred to as Clustering of Neural Bandits, or CNB), they suffer from loss of plasticity, where neural network parameters become rigid and less adaptable over time, limiting their ability to adapt to non-stationary environments (e.g., dynamic user preferences in recommendation). To address this challenge, we propose Selective Reinitialization (SeRe), a novel bandit learning framework that dynamically preserves the adaptability of CNB algorithms in evolving environments. SeRe leverages a contribution utility metric to identify and selectively reset underutilized units, mitigating loss of plasticity while maintaining stable knowledge retention. Furthermore, when combining SeRe with CNB algorithms, the adaptive change detection mechanism adjusts the reinitialization frequency according to the degree of non-stationarity, ensuring effective adaptation without unnecessary resets. Theoretically, we prove that SeRe enables sublinear cumulative regret in piecewise-stationary environments, outperforming traditional CNB approaches in long-term performances. Extensive experiments on six real-world recommendation datasets demonstrate that SeRe-enhanced CNB algorithms can effectively mitigate the loss of plasticity with lower regrets, improving adaptability and robustness in dynamic settings.

强化学习推荐系统神经网络自适应

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。