让AI模型自我反思,用代码化提示提升推理效率
Introspection of Thought Helps AI Agents
- 在提示中嵌入可执行代码,让大模型自主进行程序化反思推理
- 平均性能提升7.95%,推理令牌消耗比最优基线低58.3%
- 适用于文本与图像任务,适合追求高效推理的AI系统开发者
AI代理依赖大语言模型(LLMs)和多模态大语言模型(MLLMs)在无需微调的情况下完成文本与图像任务的解释与推理,其能力由这些模型决定。传统方法如思维链(Chain-of-Thought)、思考迭代(Iteration of Thought)和图像思维(Image-of-Thought)虽通过复杂提示工程提升交互效果,但仍受限于大模型对自然语言的理解能力,且迭代推理带来高昂的令牌开销。为此,本文提出一种带思想自省机制(INoT)的新推理框架,通过设计一种新型的提示内可执行代码(LLM-Read code),引导大模型按代码逻辑执行程序化对话推理,使自我否定与反思过程在模型内部发生而非外部,显著降低令牌成本。在六项基准测试、三类任务上的实验表明,INoT平均性能提升7.95%,优于所有基线;同时,其令牌消耗平均低于表现最佳的基线方法58.3%。此外,验证实验还展示了INoT在图像理解与推理中的通用性。
原文摘要 · Abstract (English)
AI Agents rely on Large Language Models (LLMs) and Multimodal-LLMs (MLLMs) to perform interpretation and inference in text and image tasks without post-training, where LLMs and MLLMs play the most critical role and determine the initial ability and limitations of AI Agents. Usually, AI Agents utilize sophisticated prompt engineering and external reasoning framework to obtain a promising interaction with LLMs, e.g., Chain-of-Thought, Iteration of Thought and Image-of-Thought. However, they are still constrained by the inherent limitations of LLM in understanding natural language, and the iterative reasoning process will generate a large amount of inference cost. To this end, we propose a novel AI Agent Reasoning Framework with Introspection of Thought (INoT) by designing a new LLM-Read code in prompt. It enables LLM to execute programmatic dialogue reasoning processes following the code in prompt. Therefore, self-denial and reflection occur within LLM instead of outside LLM, which can reduce token cost effectively. Through our experiments on six benchmarks for three different tasks, the effectiveness of INoT is verified, with an average improvement of 7.95\% in performance, exceeding the baselines. Furthermore, the token cost of INoT is lower on average than the best performing method at baseline by 58.3\%. In addition, we demonstrate the versatility of INoT in image interpretation and inference through verification experiments.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。