arXiv:2606.24396cs.LG2026-06中稿 · ICLR

通过能量场调节实现大模型高效适配,避免遗忘与计算开销。

Parallel Manifold Steering: Efficient Adaptation of Large Associative Memories via Residual Energy Shaping

论文配图:Parallel Manifold Steering: Efficient Adaptation of Large Associative Memories via Residual Energy Shaping
图 1 · 摘自论文原文
  • 在不改变模型结构的前提下,通过残差能量调节引导注意力轨迹。
  • 在关联检索任务上比全局权重调整提升26%,且无提示词计算开销。
  • 适合需要快速适配、保持稳定性的大模型部署场景。

大型Transformer模型可视为稠密关联记忆系统(DAM),通过自注意力机制驱动的高维吸引子动态进行知识检索。然而,将这些冻结的记忆系统适应新任务面临根本性的「可塑性-稳定性」困境。现有方法要么直接修改连接权重导致灾难性遗忘(如LoRA),要么用静态提示词堵塞检索缓冲区,降低关联能力(如VPT)。本文提出H-Res(层级残差引导),一种在不改变全局平衡点或序列长度前提下,调节Transformer有效能量景观的机制。通过将适配建模为激活流形上的控制问题,H-Res学习一个状态相关的向量场,引导令牌轨迹进入特定任务的吸引子盆地。理论上证明其保持基础模型注意力熵并促进神经坍缩。实验表明,该方法在关联检索任务上比全局权重修改高出26%,并消除提示法的计算开销,能有效扩展至结构化领域(如VTab)。

原文摘要 · Abstract (English)

Large Transformer models function as Dense Associative Memories (DAMs), retrieving knowledge via high-dimensional attractor dynamics driven by the self-attention mechanism \citep{ramsauer2020hopfield, wu2024attention}. However, adapting these frozen memory systems to new tasks presents a fundamental ``Plasticity-Stability'' dilemma. Current methods either risk catastrophic interference by modifying synaptic weights directly (e.g., LoRA) \citep{hu2021lora} or degrade associative capacity by clogging the retrieval buffer with static prompt tokens (e.g., VPT) \citep{jia2022vpt}. In this work, we propose \textbf{H-Res} (Hierarchical Residual Steering), a mechanism that modulates the effective energy landscape of the Transformer without altering its global equilibrium or expanding its sequence length. By formulating adaptation as a control problem on the activation manifold \citep{chen2018neuralode}, H-Res learns a state-dependent vector field that steers token trajectories into task-specific basins of attraction. We formally prove that H-Res preserves the attention entropy of the foundation model and facilitates Neural Collapse \citep{papyan2020prevalence}. Empirically, Manifold Steering outperforms global weight modification by 26\% on associative retrieval tasks and eliminates the computational overhead of prompt-based methods, scaling effectively to structured domains \citep{zha2023vtab}.

大模型适配能量场注意力机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。