微调扰动可预测，但多任务组合易失效，揭示大模型局部适应的脆弱性
First-Order Predictable but Pairwise Fragile: Local Task Adaptation in Trained Transformers
- 在预训练模型上施加小扰动，沿梯度方向变化可线性预测损失
- 超过三分之一任务对在扰动量10⁻²内出现顺序敏感，且子空间旋转迅速
- 适用于快速微调场景，但不适用于多任务联合更新或精确控制
任务算术、序列微调、激活调控和一阶随机搜索均通过在已训练检查点附近施加小扰动实现，依赖不同的局部近似假设：单个扰动应具有一阶可预测性，任务更新应可控干扰，切线结构应稳定可估计，权重修改应在表示空间有对应。我们在9个Transformer模型（82M-7B）上，以多任务LoRA为基准，统一评估8项性质，采用预注册属性列表、阈值与测试集。发现所有模型在扰动量10⁻²内存在单向有效窗口，但无通用的成对组合半径。沿单一方向，探针损失变化始终可一阶预测；扰动对损失的影响基本等于其在梯度上的投影，这也解释了局部随机搜索的有效性。然而，成对结构极为脆弱：超三分之一（模型，任务）组合在10⁻²内即出现顺序敏感；任务梯度子空间在数十步内发生旋转；固定激活探针下，全任务向量尺度的可加性在多个模型（包括两个7B模型）中失败；无一模型中位数通过全局权重到调控的对应标准。对于两步梯度更新，主导项为李括号 $H_B\textbf{g}_A-H_A\textbf{g}_B$；其归一化预测 $c(η)=ηκ+O(η^2)$ 在中位数比值达1.002，而起始尺度 $η^⇒\approx0.10/κ$ 在模型与任务对间跨度达三个数量级。
原文摘要 · Abstract (English)
Task arithmetic, sequential fine-tuning, activation steering, and first-order random search all operate through relatively small perturbations around an already trained checkpoint, and they rely on different local approximations: individual perturbations should be first-order predictable, task updates should compose with controlled interference, useful tangent structure should be stable and possible to estimate, and weight edits should have counterparts in representation space. We measure 8 such properties with the same harness around a multitask LoRA operating point, on 9 transformers (82M-7B), with a prospectively registered property list, thresholds, and test split. We find a shared one-direction validity window up to the tested scale $10^{-2}$, but no universal radius for pairwise composition or update ordering. Along individual directions, changes of the probe loss remain first-order predictable throughout the grid: a perturbation's effect on the loss is essentially its projection onto the gradient, which is also what makes local random search work. Pairwise structure, however, proves to be far more fragile: on over a third of the measured (model, task pair) combinations, two-update order sensitivity sets in strictly inside that window; task-gradient subspaces rotate within tens of steps; additivity under our fixed activation probe fails at full task-vector scale on several models, including both held-out 7B models; and no model median passes the registered global mean-vector weight-to-steering correspondence bar. For two sequential task-gradient steps, the leading order-dependent term is the Lie bracket $H_B\textbf{g}_A-H_A\textbf{g}_B$; its normalized prediction $c(η)=ηκ+O(η^2)$ tracks the measured defect at median ratio 1.002, while the onset scale $η^\dagger\approx0.10/κ$ spans three orders of magnitude across models and task pairs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。