用连续情绪坐标控制语言模型，让生成更细腻真实。
VA-DPO: Valence-Arousal Direct Preference Optimization for Controllable Emotion Generation in Language Models
- 用情绪平面坐标精准指定想要的情绪状态
- 相比传统提示，情绪距离降低33%，相关性达0.93
- 适配多模型且不影响原有任务性能
如何精确地告诉语言模型‘感受’？现有方法多采用离散标签（如快乐、愤怒），难以表达‘轻微低落但平静’等细腻情绪。本文提出VA-DPO，将目标情绪表示为效价-唤醒度平面上的连续点（v*, a*），通过一个冻结的VA回归器计算生成文本与目标的欧氏距离，仅保留距离差超过阈值tau的样本对，再用普通DPO损失优化LoRA适配器。方法仅改变偏好数据构建方式，不改动DPO目标。在Llama-3.1-8B-Instruct上，该方法使平均情绪距离比系统提示降低33%，比少样本提示降低25%，效价和唤醒度相关性分别提升至r_v=0.93、r_a=0.75。效果在Qwen3-8B和Llama-3.2-3B上亦成立，且不牺牲原有性能：MMLU得分无变化（Delta=+0.0），HellaSwag和TruthfulQA保持不变。代码、配置及偏好构建流程已开源。
原文摘要 · Abstract (English)
How precisely can we tell a language model how to feel? Most work on emotional generation answers with a discrete label - happy, angry, sad - which cannot express a target like "mildly downcast but calm." We instead specify the desired affect as a continuous point (v*, a*) in the Valence-Arousal plane and train the model to hit it. Our method, VA-DPO, is a small modification to Direct Preference Optimization: a frozen VA regressor scores each sampled generation by its Euclidean distance to the target, we keep only candidate pairs whose distance gap clears a margin tau, and we optimize a LoRA adapter with the ordinary DPO loss against a frozen reference. The DPO objective itself is unchanged; what is new is how the preference data is built. On Llama-3.1-8B-Instruct this cuts mean VA distance to the target by 33% over system-prompting and 25% over few-shot prompting, lifting valence/arousal correlation to r_v=0.93 and r_a=0.75. The gains carry over to Qwen3-8B and Llama-3.2-3B, and they do not come at the usual price: MMLU is unchanged (Delta=+0.0) and HellaSwag and TruthfulQA are preserved. We release the code, configs, and the preference-construction pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。