通过组合PEFT模块输出，实现多属性文本生成的即插即用控制。
Output Composability of QLoRA PEFT Modules for Plug-and-Play Attribute-Controlled Text Generation
- 在推理时直接叠加已训练好的QLoRA模块输出，实现多任务控制。
- 多属性控制下性能平均提升2个百分点，优于单任务专用模型。
- 无需重新训练，适合快速部署多属性文本生成系统。
参数高效微调（PEFT）技术可在极低成本下实现特定任务微调，但需为每个新任务单独训练。本文探索三种超越单任务训练/推理的方法：(i) 在多个相关数据集组合上训练；(ii) 推理时组合独立训练的PEFT模块权重矩阵；(iii) 推理时组合独立训练的PEFT模块输出。我们在三种不同大模型上测试这些方法，采用QLoRA作为PEFT技术，并使用情感控制、主题控制和多属性控制三类数据集。结果表明，叠加PEFT模块输出是一种尤为有效的组合方式，在所有情况下均表现优异或持平于其他方法。即使与单任务专用模块相比，三模块输出组合在情感控制任务上平均提升2个百分点，且适用于多种模型。
原文摘要 · Abstract (English)
Parameter-efficient fine-tuning (PEFT) techniques offer task-specific fine-tuning at a fraction of the cost of full fine-tuning, but require separate fine-tuning for every new task (combination). In this paper, we explore three ways of generalising beyond single-task training/inference: (i) training on combinations of multiple, related datasets; (ii) at inference, composing the weight matrices of separately trained PEFT modules; and (iii) at inference, composing the outputs of separately trained PEFT modules. We test these approaches on three different LLMs, QLoRA as the PEFT technique, and three sets of controlled text generation datasets for sentiment control, topic control, and multi-attribute control. We find that summing PEFT module outputs is a particularly strong composition method, which consistently either outperforms or matches the performance of alternative approaches. This is the case even when comparing against single-task specialised modules on the single-task test set, where three-module output composition achieves an average 2% point performance increase across all models for sentiment control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。