用修辞结构理论提升生成图表的忠实度，减少幻觉。
When Looks Do Not Lie: Discourse Structure Guided In-Context Learning for Faithful Diagram Generation
- 基于修辞结构理论引导上下文学习生成图表
- 专家评估显示150张图质量显著提升
- 适合教育类AI系统开发者与评测研究者
生成式AI广泛应用于教育场景，但常产生内在和外在幻觉。本文提出一种基于修辞结构理论（Rhetorical Structure Theory）的上下文学习（ICL）图表生成方法，提升图表对源文本语境的忠实度。研究发现，ICL性能受任务分布和模型推理能力影响，更高推理能力能更好应对分布外任务。我们对150张生成图表进行专家评估，并采用贝叶斯广义线性混合模型（Bayesian GLMMs）分析结果。此外，利用评估量表和数据集样例实现自动化图表评估，与人工评估达到统计学显著一致性。
原文摘要 · Abstract (English)
GenAI is widespread in educational applications; however, it is known to generate content with intrinsic and extrinsic hallucination. We introduce a novel method for ICL diagram generation based on Rhetorical Structure Theory, which improves diagram faithfulness to its source text context. We find that ICL performance depends on task distribution and models' reasoning ability, with higher reasoning allowing better quality and performance for an out-of-distribution task. We perform an expert evaluation of 150 generated diagrams and analyze our findings using Bayesian GLMMs. Additionally, we use our evaluation rubric and samples from the data set for automated diagram evaluation, achieving statistically significant agreement with human evaluation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。