通过调整模型内部子更新,让大模型输出更准确可靠。
LLM Braces: Straightening Out LLM Predictions with Relevant Sub-Updates
- 基于前馈层的子更新重加权,动态调节每部分贡献
- 零样本和微调下均提升性能,参数量减少75%
- 可精准控制文本情感与毒性,适合可控生成场景
近期研究表明,基于Transformer的大语言模型(LLM)中的知识主要编码在前馈网络(FFN)层中,每个FFN层可视为若干子更新之和,每个子更新对应于FFN值矩阵中一个加权列向量,常包含人类可解释的概念。受此启发,我们提出LLMBRACES:一种新颖高效的调节方法,通过计算FFN层中值向量的相关性得分,并据此动态调整子更新的贡献,从而优化预测过程,提升输出准确性与可靠性,如同‘支架’提供支持与稳定。此外,该方法可扩展至对生成风格(如情感)进行条件控制,实现细粒度输出引导。在Qwen2.5-1.5B、Llama2-7B、Llama3-8B等模型上的大量实验表明,LLMBRACES在零样本与微调设置下均优于基线方法,且可调参数最多减少75%(相比LoRA)。同时,在情感控制生成与毒性降低方面表现突出,展现出灵活可控文本生成的巨大潜力。
原文摘要 · Abstract (English)
Recent findings reveal that much of the knowledge in a Transformer-based Large Language Model (LLM) is encoded in its feed-forward (FFN) layers, where each FNN layer can be interpreted as the summation of sub-updates, each corresponding to a weighted column vector from the FFN's value parameter matrix that often encodes human-interpretable concepts. In light of this, we hypothesize that model performance and behaviors can be further enhanced and controlled by modulating the contributions of these sub-updates based on their relevance to the input or target output style, and propose LLMBRACES, a novel and efficient method that computes relevance scores associated with value vectors in FFN layers and leverages these scores to dynamically adjust the contribution of sub-updates. By optimizing sub-update contributions, LLMBRACES refines the prediction process, leading to more accurate and reliable outputs, much like a 'brace' providing support and stability. Moreover, LLMBRACES can be extended to support conditional control over generation characteristics, such as sentiment, thereby offering fine-grained steering of LLM outputs. Extensive experiments on various LLMs-including Qwen2.5-1.5B, Llama2-7B, and Llama3-8B-demonstrate that LLMBRACES outperforms baseline approaches in both fine-tuning and zero-shot settings while requiring significantly fewer tunable parameters, up to 75% fewer compared to LoRA. Furthermore, LLMBRACES excels in sentiment-controlled generation and toxicity reduction, highlighting its potential for flexible, controlled text generation across applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。