用实时外部知识提升大模型生成反驳意见的准确性与说服力。
Dynamic Knowledge Integration for Evidence-Driven Counter-Argument Generation with Large Language Models
- 动态引入网络知识，增强反驳内容的相关性与事实性。
- 相比传统方法，生成的反驳在相关性、说服力上提升显著。
- 适合需要高可信度反驳生成的研究者与AI系统开发者。
本文研究大语言模型（LLMs）在反驳生成任务中动态整合外部知识的作用。尽管LLMs在论辩任务中表现出潜力，但其常生成冗长且可能不实的内容，亟需更可控、基于证据的方法。我们构建了一个人工标注的论点-反驳对数据集，兼顾论辩复杂性与评估可行性。提出一种新的LLM-as-a-Judge评价方法，其与人类判断的相关性优于传统参考基准指标。实验表明，从网络动态获取外部知识可显著提升反驳生成质量，尤其在相关性、说服力和事实性方面。结果表明，结合实时外部知识检索的LLM是发展更有效、可靠反驳系统的重要方向。
原文摘要 · Abstract (English)
This paper investigates the role of dynamic external knowledge integration in improving counter-argument generation using Large Language Models (LLMs). While LLMs have shown promise in argumentative tasks, their tendency to generate lengthy, potentially unfactual responses highlights the need for more controlled and evidence-based approaches. We introduce a new manually curated dataset of argument and counter-argument pairs specifically designed to balance argumentative complexity with evaluative feasibility. We also propose a new LLM-as-a-Judge evaluation methodology that shows a stronger correlation with human judgments compared to traditional reference-based metrics. Our experimental results demonstrate that integrating dynamic external knowledge from the web significantly improves the quality of generated counter-arguments, particularly in terms of relatedness, persuasiveness, and factuality. The findings suggest that combining LLMs with real-time external knowledge retrieval offers a promising direction for developing more effective and reliable counter-argumentation systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。