评估大模型是否真正聚焦目标，发现多数模型并不完全专注。
Evaluating the Goal-Directedness of Large Language Models
- 通过子任务分析模型在信息搜集、认知与规划中的目标聚焦能力
- 不同任务间目标导向性稳定，但与整体表现不一致
- 激励提示仅中度提升目标导向，适合关注智能体设计的研究者
大语言模型在多大程度上将其能力用于实现给定目标？我们以此作为衡量其目标导向性的指标。我们在需要信息搜集、认知努力和计划执行的任务上进行评估，通过子任务推断模型的相关能力。对谷歌深脑、OpenAI 和 Anthropic 的大模型评估显示，目标导向性在不同任务间相对一致，与任务性能不同，且对动机提示的敏感度仅为中等。值得注意的是，大多数模型并未完全具备目标导向性。我们希望本研究的目标导向性评估能帮助更有效地监控大模型进展，并支持在大模型中更审慎地设计智能体特性。
原文摘要 · Abstract (English)
To what extent do LLMs use their capabilities towards their given goal? We take this as a measure of their goal-directedness. We evaluate goal-directedness on tasks that require information gathering, cognitive effort, and plan execution, where we use subtasks to infer each model's relevant capabilities. Our evaluations of LLMs from Google DeepMind, OpenAI, and Anthropic show that goal-directedness is relatively consistent across tasks, differs from task performance, and is only moderately sensitive to motivational prompts. Notably, most models are not fully goal-directed. We hope our goal-directedness evaluations will enable better monitoring of LLM progress, and enable more deliberate design choices of agentic properties in LLMs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。