用动机图谱和苏格拉底对话提升大模型的创意质量
MotivGraph-SoIQ: Integrating Motivational Knowledge Graphs and Socratic Dialogue for Enhanced LLM Ideation
- 构建动机知识图谱，结构化存储问题、挑战和解决方案
- 通过苏格拉底式提问双代理系统，降低确认偏误并提升创意质量
- 在ICLR25数据集上优于现有方法，人类评估与评分均表现更优
大型语言模型（LLMs）在加速学术创意生成方面具有巨大潜力，但面临创意缺乏依据和确认偏误难以缓解的问题。本文提出MotivGraph-SoIQ框架，通过整合动机知识图谱（MotivGraph）与基于问题驱动的苏格拉底式创意生成器，增强大模型的创意生成能力。该框架采用三类节点（问题、挑战、解决方案）结构化存储动机信息，为创意过程提供依据。创意生成器为双代理系统，利用苏格拉底式提问实现严谨的优化流程，有效减轻确认偏误，并在新颖性、实验严谨性和动机合理性三个维度提升创意质量。在ICLR25论文主题数据集上的实验显示，该方法在基于LLM的评分、ELO排名及人工评估中均显著优于当前先进方法。
原文摘要 · Abstract (English)
Large Language Models (LLMs) hold substantial potential for accelerating academic ideation but face critical challenges in grounding ideas and mitigating confirmation bias for further refinement. We propose integrating motivational knowledge graphs and socratic dialogue to address these limitations in enhanced LLM ideation (MotivGraph-SoIQ). This novel framework provides essential grounding and practical idea improvement steps for LLM ideation by integrating a Motivational Knowledge Graph (MotivGraph) with a Q-Driven Socratic Ideator. The MotivGraph structurally stores three key node types(problem, challenge and solution) to offer motivation grounding for the LLM ideation process. The Ideator is a dual-agent system utilizing Socratic questioning, which facilitates a rigorous refinement process that mitigates confirmation bias and improves idea quality across novelty, experimental rigor, and motivational rationality dimensions. On the ICLR25 paper topics dataset, MotivGraph-SoIQ exhibits clear advantages over existing state-of-the-art approaches across LLM-based scoring, ELO ranking, and human evaluation metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。