让大模型对话更连贯:用动态知识图谱和推理树追踪上下文
D-SMART: Enhancing LLM Dialogue Consistency via Dynamic Structured Memory And Reasoning Tree
- 构建动态知识图谱与可追溯的多步推理树,实时更新对话上下文
- 在MT-Bench-101上使对话一致性提升超48%,开源模型质量最高增10.1%
- 适合需要长期对话一致性的应用,如智能客服、教育助手
大型语言模型在多轮对话中常出现事实不一致和逻辑退化问题,根源在于依赖静态预训练知识且无法自适应推理对话历史。现有方法如RAG和代理工作记忆虽提升信息召回,但仍基于静态知识源,遵循固定推理路径,难以随上下文演变保持一致性。为此,我们提出D-SMART,一种模型无关框架,通过动态结构化记忆(DSM)增量构建符合OWL规范的对话知识图谱,并利用推理树(RT)在图谱上执行显式、可追溯的多步推理。针对现有评估指标易忽略逻辑缺陷的问题,引入基于NLI的新度量标准。在MT-Bench-101上的实验表明,D-SMART显著优于现有基线,使专有与开源模型的对话一致性评分均提升超48%,尤其使开源模型质量最高提升10.1%。
原文摘要 · Abstract (English)
Large Language Models (LLMs) often exhibit factual inconsistencies and logical decay in extended, multi-turn dialogues, a challenge stemming from their reliance on static, pre-trained knowledge and an inability to reason adaptively over the dialogue history. Prevailing mitigation strategies, such as Retrieval-Augmented Generation (RAG) and agentic working memories, improve information recall but still engage with fundamentally static knowledge sources and follow pre-defined single reasoning path. This hinders their ability to preserve factual and logical consistency of their responses in multi-turn dialogues while the context evolves over time. To address this issue, we propose D-SMART, a model-agnostic framework designed to maintain multi-turn dialogue consistency by enabling LLMs to build and reason over a dynamic, structured representation of the conversational context. This is achieved via two synergistic components: (1) a Dynamic Structured Memory (DSM), which incrementally constructs and maintains an authoritative, OWL-compliant knowledge graph of the conversation; and (2) a Reasoning Tree (RT), which executes inferences as an explicit and traceable multi-step search over the graph. As the popular-used quality score (judged by GPT-4) can overlook logical flaws, we introduce new NLI-based metrics to better measure multi-turn dialogue consistency. Comprehensive experiments on the MT-Bench-101 benchmark show that D-SMART significantly outperforms state-of-the-art baselines, elevating the dialogue consistency score by over 48\% for both proprietary and open-source models, and notably improves the quality score of the latter by up to 10.1\%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。