探索生成式摘要中控制向量的优劣，发现混合使用更平衡。
Beyond Multiple Choice: Evaluating Steering Vectors for Summarization
- 用学习到的偏置向量控制摘要的主题、情感等属性。
- 强控制力导致重复和事实错误，影响摘要质量。
- 结合提示词与控制向量可实现最佳效果平衡。
控制向量是一种轻量级方法，通过在推理时向语言模型激活值添加学习到的偏置来调节文本属性。尽管其在多项选择和玩具任务中被广泛研究，但在自由生成场景中的有效性仍不明确。本文超越多选任务，评估了控制向量在SAMSum、NEWTS和arXiv数据集上对摘要的主题聚焦、情感、毒性及可读性的影响。结果表明，控制向量能有效调控目标属性，但过强的控制强度会引发重复和事实幻觉。仅使用提示词虽能保持摘要质量，但控制能力较弱。二者结合在中等控制强度下展现出最强的控制力与最优的质量权衡。本研究揭示了控制向量在自由生成中存在关键的控制-质量权衡，而混合方法在实践中表现最佳。
原文摘要 · Abstract (English)
Steering vectors are a lightweight method for controlling text properties by adding a learned bias to language model activations at inference time. While predominantly studied for multiple-choice and toy tasks, their effectiveness in free-form generation remains largely unexplored. Moving "Beyond Multiple Choice," we evaluate steering vectors for controlling topical focus, sentiment, toxicity, and readability in abstractive summaries across the SAMSum, NEWTS, and arXiv datasets. We find that steering effectively controls targeted properties, but high steering strengths consistently induce degenerate repetition and factual hallucinations. Prompting alone preserves summary quality but offers weaker control. Combining both methods yields the strongest control and the most favorable efficacy-quality trade-off at moderate steering strengths. Our work demonstrates that steering vectors face a critical control-quality trade-off in free-form generation, and that hybrid approaches offer the best balance in practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。