研究发现，语言模型的引导向量在复杂场景下效果有限。
On the Limitations of Steering in Language Model Alignment
- 用钩子干预和反义函数向量评估引导效果
- 简单任务中引导有效，复杂上下文则表现不佳
- 适合研究推理模型的引导能力边界
引导向量是一种有前景的在推理时对齐语言模型行为的方法。本文提出一个框架，用于评估引导向量作为对齐机制的局限性。通过使用Transformer钩子干预和基于反义词的功能向量，我们考察了提示结构和上下文复杂度对引导效果的影响。研究发现，引导向量在特定对齐任务（如价值对齐）中表现良好，但在复杂场景下可能无法为通用对齐提供可靠基础。本研究为未来探索推理模型引导能力奠定了方法论基础。
原文摘要 · Abstract (English)
Steering vectors are a promising approach to aligning language model behavior at inference time. In this paper, we propose a framework to assess the limitations of steering vectors as alignment mechanisms. Using a framework of transformer hook interventions and antonym-based function vectors, we evaluate the role of prompt structure and context complexity in steering effectiveness. Our findings indicate that steering vectors are promising for specific alignment tasks, such as value alignment, but may not provide a robust foundation for general-purpose alignment in LLMs, particularly in complex scenarios. We establish a methodological foundation for future investigations into steering capabilities of reasoning models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。