让小模型持续学习：用教师整理的知识库提升性能，减少调用大模型次数。
Evolve: A Persistent Knowledge Lifecycle for Small Language Models
- 小模型搭配持久知识库，通过教师整理与定期合并更新知识。
- 准确率从20-33%提升至60-84%，教师调用减少50%以上。
- 支持严格溯源和增强生成两种模式，适合需可靠性的应用。
Evolve 将小型本地语言模型与一个持久的、由教师模型编译的知识库相结合——该知识库通过睡眠式巩固和使用驱动的刷新机制不断优化，显著提升模型准确性，同时通过跨查询知识复用分摊教师成本。不同于查询时检索文档片段，Evolve 在自然概念边界处由教师模型编译出语义连贯的知识段落；新段落先暂存，再通过教师主导的合并离线整合，过期后在线刷新。一个20亿参数的本地模型负责分类与生成，大教师模型仅在知识操作时被调用。在涵盖自定义专业问题、NaturalQuestions和TriviaQA的750个基准测试中，2B模型经Evolve增强后准确率从20-33%提升至60-84%（+40-52个百分点），教师调用减少超50%。巩固后知识库压缩率达31-33.5%，在三个独立基准上保持准确率；基于段落的检索在所有生命周期条件下均优于基于块的检索，提升5-9个百分点。该架构支持同一生命周期内的两种生成模式：抑制（严格段落依赖，可审计）与增强（段落补充回答）。
原文摘要 · Abstract (English)
Evolve pairs a small local language model with a persistent, teacher-compiled knowledge store -- refined through sleep consolidation and usage-driven refresh -- to deliver substantial accuracy gains over the model's parametric baseline while amortizing teacher costs through cross-query knowledge reuse. Rather than retrieving document fragments at query time, Evolve constructs a store of semantically coherent sections compiled by teacher models at natural conceptual boundaries; new sections are staged on acquisition, consolidated offline through teacher-mediated merging, and refreshed inline when expired. A 2B-parameter local model handles classification and generation; large teacher models are invoked only for knowledge operations. Across 750 benchmark queries spanning custom specialist questions, NaturalQuestions, and TriviaQA, the 2B model augmented by Evolve improves from 20-33% baseline accuracy to 60-84% (+40-52pp) while reducing teacher invocations by over 50% through reuse. Post-consolidation compresses the knowledge store by 31-33.5% across three independent benchmarks while preserving accuracy; section-based retrieval outperforms chunk-based retrieval by 5-9pp across every lifecycle condition. The architecture supports two generation modes over the same lifecycle -- suppress (strict section-only grounding, auditable) and augment (section-supplemented responses).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。