检测大模型生成论文中的虚假引用，提升科学写作可信度。
CiteCheck: Retrieval-Grounded Detection of LLM Citation Hallucinations in Scientific Text
- 结合学术检索与结构化大模型比对，验证引用真实性
- 在982条物理领域引用测试中准确率达88.9%
- 适合科研人员审核文献引用，保障论文可靠性
大型语言模型（LLMs）越来越多用于生成科学报告，但可能产生看似合理却存在错误元数据或指向不存在论文的引用。我们提出CiteCheck，一种混合框架，用于检测引用幻觉：验证引用是否对应真实学术成果及其元数据是否准确。CiteCheck从外部学术源检索候选论文，利用结构化大模型验证器比对引用与检索结果，并将评分映射为三类标签：精确、轻微偏差、严重偏差。我们构建了一个包含982条引用的物理领域基准测试集，包含受控干扰，涵盖细微元数据偏差和完全虚构引用。在留出测试集上，CiteCheck实现88.7%宏平均F1和88.9%准确率，优于GPT、Claude和Gemini等基线，包括网络搜索和少样本变体。结果表明，可靠的引用验证需结合学术检索、结构化大模型比对和校准决策规则。
原文摘要 · Abstract (English)
Large language models (LLMs) are increasingly used to generate scientific reports, but they can produce references that appear plausible while containing corrupted metadata or pointing to papers that do not exist. We introduce CiteCheck, a hybrid framework for citation hallucination detection that verifies whether a citation corresponds to a real scholarly work and whether its metadata is faithful to that work. CiteCheck retrieves candidate publications from external scholarly sources, compares the citation against the retrieved candidate using a structured LLM verifier, and maps verifier scores into three labels: Exact, Minor, and Major. We also construct a 982-citation physics benchmark with controlled corruptions that capture both subtle metadata drift and fully fabricated references. On the held-out test set, CiteCheck achieves 88.7 macro-F1 and 88.9% accuracy, outperforming GPT, Claude, and Gemini baselines, including web-search and few-shot variants. These results show that reliable citation verification benefits from combining scholarly retrieval, structured LLM-based comparison, and calibrated decision rules.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。