提出SCALE-LoRA框架，解决多LoRA组合后输出不可靠问题。
SCALE-LoRA: Auditing Post-Retrieval LoRA Composition with Residual Merging and View Reliability
- 引入稀疏残差组合机制，避免参数冲突
- 通过多视角分歧检测提升输出可靠性3.0倍
- 适合需要高可靠性的轻量化模型复用场景
低秩适配器（LoRA）库已成为参数高效微调的实用副产品。当适配器积累后，关键问题从‘为单任务训练一个适配器’转变为‘仅用少量支持样本，从开放池中复用适配器完成新任务’。已有工作表明可在任务层面组合LoRA，实例层面动态选择。但开放池复用并非自动：检索相关适配器不保证参数更新兼容，组合也不保证输出可靠。本文提出稀疏组合一致性层（SCALE），一种后检索审计与组合框架。包含可部署的1.0×融合路径——层自适应稀疏残差组合（LASRC），以及高成本可靠性分析层。LASRC通过保留线性锚点并分块残差化适配器更新方向，缓解融合干扰。可靠性层将稀疏组合视角间的分歧视为可观测不确定性信号，对比一致性、支持损失代理选择与理想头空间，在显式路径成本下评估。在匹配的FLAN-T5-Large、BIG-Bench Hard（BBH）及97-LoRA实验中，LASRC在固定检索下实现定向单视角增益；而SCALE-support被报告为无查询标签的3.0×可靠性分析变体，而非校准或吞吐量等价的选择器。协议特异的BBH-8验证显示，三种解码器仅有的骨干上趋势一致。详细分数、成对审计与路径成本记录见实验部分。
原文摘要 · Abstract (English)
Libraries of Low-Rank Adaptation (LoRA) adapters are becoming a practical by-product of parameter-efficient adaptation. Once such adapters accumulate, a natural question is no longer how to train one adapter for one task, but how to reuse an open pool of adapters for a new task given only a small support set. Prior work has shown that LoRA modules can be composed at the task level and dynamically selected at the instance level. However, open-pool LoRA reuse is not automatic: retrieving relevant adapters does not guarantee that their parameter updates are compatible, and composing adapters does not guarantee reliable outputs. We introduce the Sparse-Composition Agreement Layer (SCALE), a post-retrieval audit and composition framework for open-pool LoRA reuse. SCALE contains a deployable 1.0* merge path, Layer-Adaptive Sparse Residual Composition (LASRC), and a higher-cost reliability-analysis layer for multi-view disagreement. LASRC addresses merge interference by preserving a linear anchor while residualizing block-wise adapter update directions. The reliability layer treats disagreement among sparse composition views as an observable uncertainty signal and compares agreement, support-loss proxy selection, and oracle headroom under explicit path cost. In matched FLAN-T5-Large, BIG-Bench Hard (BBH), and 97-LoRA experiments, LASRC gives a directional single-view gain under fixed retrieval, while SCALE-support is reported as a query-label-free 3.0* reliability-analysis variant rather than as a calibrated or throughput-equivalent selector. Protocol-distinct BBH-8 validation shows the same qualitative trend on three decoder-only backbones. Detailed scores, paired audits, and path-cost records are reported in the experimental section.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。