让AI模型自己学会持续学习,自动适应新任务不遗忘旧知识。
SOLAR: A Self-Optimizing Open-Ended Autonomous Agent for Lifelong Learning and Continual Adaptation

- 用元学习让模型自主发现最佳调整策略,无需人工干预。
- 在多个领域任务上表现优于现有方法,且能长期保持知识不丢失。
- 适合需要长期运行、动态变化的智能系统开发人员使用。
尽管大语言模型取得显著进展,但在动态真实环境中部署仍面临概念漂移和基于梯度微调成本高的瓶颈。传统微调易引发灾难性遗忘或需大量人工数据标注。为应对流式与持续学习挑战,我们提出自优化终身自主推理器SOLAR,一种开放闭环的自主代理,通过参数级元学习将模型权重视为可探索环境,先构建常识知识强先验以支持迁移学习。利用多层次强化学习,SOLAR自主发现适应策略,实现高效的测试时适应。关键在于其维护一个不断演化的有效修改策略知识库,隐式充当情景记忆缓冲区,平衡可塑性(适应新任务)与稳定性(保留元知识)。实验表明,SOLAR在常识、数学、医疗、编程、社交及逻辑推理任务上均优于强基线,标志着向能在演化环境中实现终身适应的自主代理迈出重要一步。
原文摘要 · Abstract (English)
Despite the remarkable success of large language models (LLMs), they still face bottlenecks while deploying in dynamic, real-world settings with primary challenges being concept drift and the high cost of gradient-based adaptation. Traditional fine-tuning (FT) struggles to adapt to non-stationary data streams without resulting in catastrophic for getting or requiring extensive manual data curation. To address these limitations within the streaming and continual learning paradigm, we propose the Self-Optimizing Lifelong Autonomous Reasoner (SOLAR) which is an open-ended autonomous agent that leverages parameter-level meta-learning to self-improve, treating model weights as an environment for exploration. It initiates the process by consolidating a strong prior over common-sense knowledge making it effective for transfer-learning. By utilizing a multi-level reinforcement learning approach, SOLAR autonomously discovers adaptation strategies, enabling efficient test-time adaptation to unseen domains. Crucially, SOLAR maintains an evolving knowledge base of valid modification strategies, implicitly acting as an episodic memory buffer to balance plasticity (adaptation to new tasks) and stability (retention of meta-knowledge). Experiments demonstrate that SOLAR outperforms strong baselines on common-sense, mathematical, medical, coding, social and logical reasoning tasks, marking a significant step toward autonomous agents capable of lifelong adaptation in evolving environments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。