让大模型在测试时自动生成训练数据并自我优化
Test-Time Meta-Adaptation with Self-Synthesis
- 通过生成特定问题的合成数据，实现测试时自适应
- 在数学推理任务中实现高效小样本适应，提升性能
- 适合需要实时自适应的部署场景，如智能客服
大型语言模型（LLMs）在面对多样领域和任务时，具备在测试时自适应和自我改进的能力尤为关键。本文提出MASS框架，使LLM能通过生成针对具体问题的合成训练数据，并执行针对性的自我更新，以优化下游任务表现。该方法采用双层优化：内层基于自生成样例进行适应，外层元学习数据归属信号与更新后任务奖励。合成数据通过可扩展的元梯度优化，将下游损失反向传播至内层更新过程，以奖励有效生成。实验表明，MASS能够学会为每道题构建个性化训练序列，在数学推理任务中实现高效、数据节约的测试时自适应。
原文摘要 · Abstract (English)
As strong general reasoners, large language models (LLMs) encounter diverse domains and tasks, where the ability to adapt and self-improve at test time is valuable. We introduce MASS, a meta-learning framework that enables LLMs to self-adapt by generating problem-specific synthetic training data and performing targeted self-updates optimized for downstream performance at inference time. We train this behavior end-to-end via bilevel optimization: an inner loop adapts on self-generated examples while an outer loop meta-learns data-attribution signals and rewards post-update task performance. The synthetic data is optimized with scalable meta-gradients, backpropagating the downstream loss through the inner updates to reward useful generations. Experiments on mathematical reasoning show that MASS learns to synthesize per-instance curricula that yield effective, data-efficient test-time adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。