最小化持续学习系统展现无需正则化的稳定与可塑性平衡
Representation Stability in a Minimal Continual Learning Agent
- 用状态向量逐步更新实现简单持续学习
- 输入一致时从易变到稳定的表征转变,相似度下降后恢复
- 为研究表征演化提供透明基准,适合关注机制的学者
持续学习系统在无法重训或重置的环境中日益重要,但多数方法侧重任务性能而非内部表征的演变。本文设计了一个极简持续学习代理,仅依赖持久状态向量,在新文本数据引入时逐步更新。通过连续归一化状态向量间的余弦相似度量化表征变化,并定义时间区间上的稳定性指标。八次纵向实验显示,在一致输入下经历从初始易变到稳定表征的转变;人为引入语义扰动导致相似度有限下降,随后在后续一致输入下恢复并重新稳定。结果表明,无需显式正则化、回放或复杂架构,最小状态学习系统仍能自然涌现有意义的稳定-可塑性权衡。该工作建立了研究持续学习中表征累积与适应的透明实证基线。
原文摘要 · Abstract (English)
Continual learning systems are increasingly deployed in environments where retraining or reset is infeasible, yet many approaches emphasize task performance rather than the evolution of internal representations over time. In this work, we study a minimal continual learning agent designed to isolate representational dynamics from architectural complexity and optimization objectives. The agent maintains a persistent state vector across executions and incrementally updates it as new textual data is introduced. We quantify representational change using cosine similarity between successive normalized state vectors and define a stability metric over time intervals. Longitudinal experiments across eight executions reveal a transition from an initial plastic regime to a stable representational regime under consistent input. A deliberately introduced semantic perturbation produces a bounded decrease in similarity, followed by recovery and restabilization under subsequent coherent input. These results demonstrate that meaningful stability plasticity tradeoffs can emerge in a minimal, stateful learning system without explicit regularization, replay, or architectural complexity. The work establishes a transparent empirical baseline for studying representational accumulation and adaptation in continual learning systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。