LARA在冻结模型的残差流中添加轻量级修正，实现高效可组合的适配与对齐。
LARA: Lightweight Adapters in the Residual Stream for Composable Adaptation and Alignment
- 在残差流中添加低秩修正，不修改原始权重，保持基模型不变。
- 相同参数量下性能媲美LoRA，支持推理时平滑调节基模型与适配行为间比例。
- 多个行为可同时驻留，按令牌自动路由，适合设备端部署多行为模型。
我们提出LARA（轻量级加性残差适配），一种在冻结模型的残差流中进行高效适配的方法，而非修改其权重。与LoRA在权重矩阵上添加低秩更新不同，LARA在少量层的隐藏状态上读取信息，并将一个低秩修正加回残差流，保持所有基础权重不变。在代码微调任务和偏好优化（DPO）上，当参数量相等时，LARA性能可媲美LoRA。由于适配是冻结基模型加上残差修正，LARA在推理时引入一个缩放系数γ，可平滑插值于基模型与适配行为之间，实现权重空间适配无法提供的分级控制。此外，由于每个行为都是基于共享冻结基模型的独立残差模块，多个行为可同时驻留并按令牌自动路由。我们在一个1.5B的冻结模型上部署了七个行为（六个微调、一个偏好优化），总开销约33 MB，而每个行为单独使用完整模型则需更多资源。因基模型不变，各行为可独立训练并按需选择，无需按需加载，适用于在单一设备上托管大量行为并灵活添加新行为。
原文摘要 · Abstract (English)
We present LARA (Lightweight Additive Residual Adaptation), a method for efficient adaptation that operates in the residual stream of a frozen model rather than in its weights. Where LoRA adds an update of low rank to weight matrices, LARA reads the hidden state at a small set of layers and adds a correction of low rank back to the residual stream, leaving all base weights untouched. On a code fine-tuning task and on preference optimization (DPO), LARA matches LoRA at equal parameter counts. Because adaptation is a frozen base plus a residual, LARA exposes a scale γ, applied at inference, that interpolates smoothly between base and adapted behavior, a form of graded control that adaptation in weight space does not offer. Finally, because each behavior is a small residual module over a shared frozen base, many behaviors can be held resident at once and routed automatically per token. We place seven behaviors, six fine-tuned and one optimized for preference, on one frozen 1.5B model for roughly 33 MB of overhead, against one full model for each behavior. Because the base is untouched, behaviors are trained separately and selected per token rather than loaded on demand, which suits hosting many behaviors, and adding new ones, on a single model on a device.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。