自动筛选新数据，让大模型持续学习不遗忘还省算力。
Towards Automatic Continual Learning: A Self-Adaptive Framework for Continual Instruction Tuning
- 用小代理模型按困惑度动态过滤新增数据，减少重复信息。
- 实测医疗场景下计算成本降66.7%，性能提升且支持自动更新。
- 适合需要长期迭代、资源受限的行业应用，如医疗、金融。
持续指令微调使大语言模型能增量式学习并保留旧知识，但现有方法多关注如何保留旧知识，而忽视应学习哪些新知识。在领域特定场景中，数据质量维护与系统约束管理仍是关键挑战。为此，我们提出一种自动化持续指令微调框架，可动态过滤输入数据，识别并消除后续更新中的冗余内容。该方法利用小型代理模型进行高效的困惑度筛选，并持续更新代理以确保过滤标准与部署模型的演进状态保持一致。相比静态数据选择方法，本框架能有效应对增量数据与分布漂移问题。此外，它通过支持无缝模型更新、版本回滚和自动检查点评估，解决了实际部署难题。我们在真实医疗场景中进行了评估，结果显示计算成本降低66.7%，模型性能提升，实现了自主更新，验证了其在自动持续指令微调中的有效性。
原文摘要 · Abstract (English)
Continual instruction tuning enables large language models (LLMs) to learn incrementally while retaining past knowledge, whereas existing methods primarily focus on how to retain old knowledge rather than on selecting which new knowledge to learn. In domain-specific contexts, maintaining data quality and managing system constraints remain key challenges. To address these issues, we propose an automated continual instruction tuning framework that dynamically filters incoming data, which identify and reduce redundant data across successive updates. Our approach utilizes a small proxy model for efficient perplexity-based filtering, and updates the proxy to ensure that the filtering criteria remain aligned with the evolving state of the deployed model. Compared to existing static data selection methods, our framework can effectively handle incrementally acquired data and shifting distributions. Additionally, it addresses practical deployment challenges by enabling seamless model updates, supporting version rollback and incorporating automatic checkpoint evaluation. We evaluated the system in real-world medical scenarios. It reduced computational costs by 66.7% and improved model performance, and achieved autonomous updates, thus demonstrating its effectiveness for automatic continual instruction tuning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。