让大模型在测试时用无标签数据自适应优化,提升专业领域表现。
Test-Time Learning for Large Language Models
- 基于输入困惑度最小化,利用测试数据动态调整模型。
- 在领域适应任务中性能提升至少20%。
- 通过选择高困惑度样本和低秩更新,防止遗忘且效率高。
尽管大型语言模型(LLMs)在大规模预训练后展现出显著的涌现能力,但在应对特定领域和多样语言变化时仍面临泛化瓶颈,即分布偏移问题。本文提出一种面向LLM的测试时学习(TTL)范式——TLM,通过仅使用测试阶段的无标签数据动态适配模型至目标领域。我们首先通过实证与理论分析发现,降低无标签测试数据的输入困惑度可提升预测准确性。据此,将TTL过程建模为输入困惑度最小化,实现自监督性能增强。进一步观察到高困惑度样本对模型优化更具信息量,因此引入高效采样策略,主动选择并强化这些样本进行测试时更新。为缓解灾难性遗忘并保证适应稳定性,采用低秩适配(LoRA)而非全参数优化,实现轻量化更新的同时保留原始知识。我们构建了AdaptEval基准用于评估TTL,并实验表明TLM在领域知识适配上相比原始模型性能提升至少20%。
原文摘要 · Abstract (English)
While Large Language Models (LLMs) have exhibited remarkable emergent capabilities through extensive pre-training, they still face critical limitations in generalizing to specialized domains and handling diverse linguistic variations, known as distribution shifts. In this paper, we propose a Test-Time Learning (TTL) paradigm for LLMs, namely TLM, which dynamically adapts LLMs to target domains using only unlabeled test data during testing. Specifically, we first provide empirical evidence and theoretical insights to reveal that more accurate predictions from LLMs can be achieved by minimizing the input perplexity of the unlabeled test data. Based on this insight, we formulate the Test-Time Learning process of LLMs as input perplexity minimization, enabling self-supervised enhancement of LLM performance. Furthermore, we observe that high-perplexity samples tend to be more informative for model optimization. Accordingly, we introduce a Sample Efficient Learning Strategy that actively selects and emphasizes these high-perplexity samples for test-time updates. Lastly, to mitigate catastrophic forgetting and ensure adaptation stability, we adopt Low-Rank Adaptation (LoRA) instead of full-parameter optimization, which allows lightweight model updates while preserving more original knowledge from the model. We introduce the AdaptEval benchmark for TTL and demonstrate through experiments that TLM improves performance by at least 20% compared to original LLMs on domain knowledge adaptation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。