用类比生成+程序记忆，让大模型更懂复杂流程任务
Pairing Analogy-Augmented Generation with Procedural Memory for Procedural Q&A
- 通过类比生成和程序记忆库复用过往经验
- 在多个数据集上超越少样本和RAG基线
- 适合需要流程推理的智能助手开发
大型语言模型在处理复杂流程任务时，难以将零散信息整合为连贯计划。本文提出一种新的流程知识形式化框架，并基于LangChain教程构建了名为LCStep的流程知识数据集。为利用该知识解决新任务，我们提出类比增强生成（AAG），借鉴人类通过过往经验应对陌生问题的能力。AAG使用自定义程序记忆存储，检索并适配特定领域知识以回答新流程任务。实验表明，在LCStep、RecipeNLG和CHAMP数据集上，AAG在基于双模型评估中优于少样本和RAG基线，且在RecipeNLG上的结果经人工评估验证。
原文摘要 · Abstract (English)
Large language models struggle to synthesize disparate pieces of information into a coherent plan when approaching a complex procedural task. In this work, we introduce a novel formalism and structure for such procedural knowledge. Based on this formalism, we present a novel procedural knowledge dataset called LCStep, which we created from LangChain tutorials. To leverage this procedural knowledge to solve new tasks, we propose analogy-augmented generation (AAG), which draws inspiration from the human ability to assimilate past experiences to solve unfamiliar problems. AAG uses a custom procedure memory store to retrieve and adapt specialized domain knowledge to answer new procedural tasks. We demonstrate that AAG outperforms few-shot and RAG baselines on LCStep, RecipeNLG, and CHAMP datasets under a pairwise LLM-based evaluation, corroborated by human evaluation in the case of RecipeNLG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。