让大模型在推理时动态调整，提升对不同输入的适应能力。
ChameleonLLM: Batch-Aware Dynamic Low-Rank Adaptation via Inference-Time Clusters
- 根据输入批次聚类，实时生成低秩权重更新
- 在多个数据集上优于传统LoRA方法，且无需维护多模型
- 适合需要灵活响应真实场景变化的部署应用
大型语言模型在各类任务中表现优异，但通常以固定权重部署，难以在推理阶段动态适应现实数据的多样性。本文提出ChameleonLLM框架，通过引入批次感知聚类和在线生成低秩更新，实现推理时的动态适配。该方法基于聚类后的输入统计信息，利用超网络生成上下文感知的低秩权重修正，无需预设专家模型或可变掩码。实验表明，ChameleonLLM在多个基准测试中显著优于传统LoRA方法，同时避免了多模型维护的开销。该方法具有高度适应性，可作为通用推理优化方案。代码已开源：https://anonymous.4open.science/r/ChamaleonLLM/
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs) have shown remarkable performance across diverse tasks. However, these models are typically deployed with fixed weights, which limits their ability to adapt dynamically to the variability inherent in real-world data during inference. This paper introduces ChameleonLLM, a novel framework that enables inference-time adaptation of LLMs by leveraging batch-aware clustering and on-the-fly generation of low-rank updates. Unlike traditional fine-tuning approaches such as Low-Rank Adaptation (LoRA) or methods that rely on a fixed set of pre-learned uniforms (changeable masks), our method dynamically generates adaptive modifications to the decoder weights based on the aggregated statistics of clustered batches. By intelligently grouping similar inputs and computing context-aware low-rank updates via a hyper-network, ChameleonLLM achieves significant performance gains, outperforming conventional LoRA methods while eliminating the overhead of maintaining multiple expert models. Our experiments highlight the potential of our approach to serve as a versatile and highly adaptive solution for language model inference. ChameleonLLM is open-sourced to ensure the reproducibility of our experiments: https://anonymous.4open.science/r/ChamaleonLLM/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。