外部知识对自然语言生成的合理性至关重要，缺乏关键知识会导致生成结果严重失真。
Exploring the Influence of Relevant Knowledge for Natural Language Generation Interpretability
- 通过移除相关知识对比生成效果，检验知识影响
- 完整知识下正确率达91%，过滤后骤降至6%
- 适合关注可解释性与知识增强生成的研究者
本文研究外部知识整合对自然语言生成（NLG）的影响，聚焦常识生成任务。我们扩展了CommonGen数据集，构建KITGI基准，该数据集将输入概念集与从ConceptNet检索到的语义关系配对，并包含人工标注的输出。使用T5-Large模型，在两种条件下进行句子生成：使用全部外部知识和移除高相关关系后的过滤知识。我们的可解释性评估采用三阶段方法：（1）识别并移除关键知识，（2）重新生成句子，（3）人工评估输出的常识合理性与概念覆盖度。结果显示，使用完整知识时，两项指标综合正确率达到91%；而过滤知识后性能急剧下降至6%。这些发现表明，相关外部知识对维持NLG的连贯性与概念覆盖至关重要。本工作强调设计可解释、知识增强型NLG系统的重要性，并呼吁建立能捕捉深层推理过程的评估框架。
原文摘要 · Abstract (English)
This paper explores the influence of external knowledge integration in Natural Language Generation (NLG), focusing on a commonsense generation task. We extend the CommonGen dataset by creating KITGI, a benchmark that pairs input concept sets with retrieved semantic relations from ConceptNet and includes manually annotated outputs. Using the T5-Large model, we compare sentence generation under two conditions: with full external knowledge and with filtered knowledge where highly relevant relations were deliberately removed. Our interpretability benchmark follows a three-stage method: (1) identifying and removing key knowledge, (2) regenerating sentences, and (3) manually assessing outputs for commonsense plausibility and concept coverage. Results show that sentences generated with full knowledge achieved 91\% correctness across both criteria, while filtering reduced performance drastically to 6\%. These findings demonstrate that relevant external knowledge is critical for maintaining both coherence and concept coverage in NLG. This work highlights the importance of designing interpretable, knowledge-enhanced NLG systems and calls for evaluation frameworks that capture the underlying reasoning beyond surface-level metrics.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。