通过角色代入提示，提升大模型心智理论能力。
Let's Put Ourselves in Sally's Shoes: Shoes-of-Others Prefilling Improves Theory of Mind in Large Language Models
- 用'换位思考'提示引导模型生成更符合角色心理的回应。
- 在对话与叙事任务中，五类心理状态表现均显著提升。
- 无需微调，适配广泛场景，适合提升AI共情能力研究者。
近期研究表明，大语言模型（LLM）的心智理论（ToM）能力尚未达到人类水平。尽管微调可提升性能，但会损害模型泛化能力。现有推理阶段方法多局限于世界状态变化的情境。本文提出一种新方法——他人视角预填充（Shoes-of-Others, SoO），仅需在输出前添加“让我们换位思考一下A的想法”，其中A为目标角色姓名。该方法不依赖特定情境假设，适用于更广泛场景。我们在两个评估对话与叙事中无世界状态变化的ToM基准上测试，发现其在五类心理状态上均实现稳定提升。分析表明，该方法能有效激发模型生成更贴近角色真实想法的回应，从而改善心智理论表现。
原文摘要 · Abstract (English)
Recent studies have shown that Theory of Mind (ToM) in large language models (LLMs) has not reached human-level performance yet. Since fine-tuning LLMs on ToM datasets often degrades their generalization, several inference-time methods have been proposed to enhance ToM in LLMs. However, existing inference-time methods for ToM are specialized for inferring beliefs from contexts involving changes in the world state. In this study, we present a new inference-time method for ToM, Shoes-of-Others (SoO) prefilling, which makes fewer assumptions about contexts and is applicable to broader scenarios. SoO prefilling simply specifies the beginning of LLM outputs with ``Let's put ourselves in A's shoes.'', where A denotes the target character's name. We evaluate SoO prefilling on two benchmarks that assess ToM in conversational and narrative contexts without changes in the world state and find that it consistently improves ToM across five categories of mental states. Our analysis suggests that SoO prefilling elicits faithful thoughts, thereby improving the ToM performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。