LoRA微调本质是加常数向量，让大模型泛化到未见任务。
Simple Mechanistic Explanations for Out-Of-Context Reasoning
- 用LoRA微调相当于添加固定方向的引导向量。
- 该向量能提升原任务及关联领域的表现，引发意外泛化。
- 即使需条件判断的任务，无条件加向量也足够有效。
Out-of-context reasoning (OOCR) 是指微调后的大型语言模型展现出超出分布范围的惊人泛化能力。不同于学习浅层启发式规则，它们隐式地内化并利用微调数据中分散的观察结果。本文从机制角度研究该现象，发现文献中许多OOCR实例可被简单解释：LoRA微调本质上是添加一个恒定的引导向量，使模型朝向某一通用概念偏移。这不仅提升了微调任务的表现，也在多个相关领域带来性能改善，从而产生看似深度推理的泛化效果。此外，我们可直接从零训练此类引导向量，同样诱发OOCR。实验表明，这一结论在看似必须依赖条件行为的任务（如模型后门）上依然成立——仅无条件添加引导向量即已足够。本工作为微调过程中学到的内容提供了机制性解释，有助于回答大模型为何能进行上下文外推理这一关键问题，对模型的安全可靠部署具有重要意义。
原文摘要 · Abstract (English)
Out-of-context reasoning (OOCR) is a phenomenon in which fine-tuned LLMs exhibit surprisingly deep out-of-distribution generalization. Rather than learning shallow heuristics, they implicitly internalize and act on the consequences of observations scattered throughout the fine-tuning data. In this work, we investigate this phenomenon mechanistically and find that many instances of OOCR in the literature have a simple explanation: the LoRA fine-tuning essentially adds a constant steering vector, steering the model towards a general concept. This improves performance on the fine-tuning task and in many other concept-related domains, causing the surprising generalization. Moreover, we can directly train steering vectors for these tasks from scratch, which also induces OOCR. We find that our results hold even for a task that seems like it must involve conditional behavior (model backdoors); it turns out that unconditionally adding a steering vector is sufficient. Overall, our work presents one explanation of what gets learned during fine-tuning for OOCR tasks, contributing to the key question of why LLMs can reason out of context, an advanced capability that is highly relevant to their safe and reliable deployment.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。