提出轻量自适应框架,实现多维度文本可控生成。
Towards Lightweight, Adaptive and Attribute-Aware Multi-Aspect Controllable Text Generation with Large Language Models
- 动态调整参数以适配不同文本属性
- 在多个数据集上达到最优性能
- 适合需要精准属性控制的生成任务
多方面可控文本生成旨在从多个属性维度控制生成内容,是自然语言处理中复杂但强大的任务。监督微调方法因简单高效被广泛采用,但仍存在局限:低秩适应(LoRA)仅微调少量参数,控制效果不佳;全量微调(FFT)需大量计算资源且易过拟合,尤其在数据有限时。此外,现有工作通常仅使用单方面标注数据训练模型,导致数据分布不一致,准确生成特定属性文本仍具挑战,需强属性感知能力。为此,本文提出一种轻量、自适应且属性感知的多方面可控文本生成框架。该框架能根据数据不同方面动态调整模型参数,实现跨多方面的可控生成,优化整体性能。实验表明,该框架优于多个强基线,达到当前最优水平,对数据分布差异有良好适应性,且在属性感知上更精准。
原文摘要 · Abstract (English)
Multi-aspect controllable text generation aims to control text generation in attributes from multiple aspects, making it a complex but powerful task in natural language processing. Supervised fine-tuning methods are often employed for this task due to their simplicity and effectiveness. However, they still have some limitations: low rank adaptation (LoRA) only fine-tunes a few parameters and has suboptimal control effects, while full fine-tuning (FFT) requires significant computational resources and is susceptible to overfitting, particularly when data is limited. Moreover, existing works typically train multi-aspect controllable text generation models using only single-aspect annotated data, which results in discrepancies in data distribution; at the same time, accurately generating text with specific attributes is a challenge that requires strong attribute-aware capabilities. To address these limitations, we propose a lightweight, adaptive and attribute-aware framework for multi-aspect controllable text generation. Our framework can dynamically adjust model parameters according to different aspects of data to achieve controllable text generation, aiming to optimize performance across multiple aspects. Experimental results show that our framework outperforms other strong baselines, achieves state-of-the-art performance, adapts well to data discrepancies, and is more accurate in attribute perception.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。