发现小模型也有隐式规划能力，可操控生成结果
What's the plan? Metrics for implicit planning in LLMs and their application to rhyme generation and question answering
- 用向量操控前句末尾影响后续生成
- 10亿参数以上模型均存在隐式规划现象
- 适用于诗歌押韵与问答任务，方法简单通用
已有研究指出，语言模型在训练中虽仅预测下一个词，却表现出隐式规划行为：可能为未来预期的词（如押韵词）提前选择当前词。我们提出更简单的评估方法，通过押韵诗歌生成和问答任务的案例研究，证明该方法可轻松扩展至多种模型。结果显示，通过在前一句末尾施加向量控制，可影响中间词的生成，进而操控最终押韵词（如“-ight”）或答案（如“whale”）。我们发现隐式规划是普遍机制，存在于此前未被关注的小型模型中，最低可达10亿参数。该方法为研究大模型隐式规划能力提供了普适且直接的路径。深入理解模型的规划能力，有助于推动人工智能安全与控制决策。
原文摘要 · Abstract (English)
Prior work suggests that language models, while trained on next token prediction, show implicit planning behavior: they may select the next token in preparation to a predicted future token, such as a likely rhyming word, as supported by a prior qualitative study of Claude 3.5 Haiku using a cross-layer transcoder. We propose much simpler techniques for assessing implicit planning in language models. With case studies on rhyme poetry generation and question answering, we demonstrate that our methodology easily scales to many models. Across models, we find that the generated rhyme (e.g. "-ight") or answer to a question ("whale") can be manipulated by steering at the end of the preceding line with a vector, affecting the generation of intermediate tokens leading up to the rhyme or answer word. We show that implicit planning is a universal mechanism, present in smaller models than previously thought, starting from 1B parameters. Our methodology offers a widely applicable direct way to study implicit planning abilities of LLMs. More broadly, understanding planning abilities of language models can inform decisions in AI safety and control.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。