用语法简化+分层修正,让大模型少说谎、更准抽关系。
Hallucination-Resistant Relation Extraction via Dependency-Aware Sentence Simplification and Two-tiered Hierarchical Refinement
- 先抓最短依存路径简化句子,再分两层修正预测结果。
- 在SciERC上将错误关联率从96.9%降到7.9%,F1提升9.3%。
- 适合需要高可靠性关系抽取的科研与知识库构建场景。
关系抽取(RE)为众多下游应用构建结构化知识。尽管大语言模型(LLMs)在此任务中展现出巨大潜力,但在复杂句法或细微语义情境下,往往难以可靠判断关系是否存在。例如,Qwen2.5-14B-Instruct在SciERC数据集上对96.9%的无关系样本错误预测了关系,暴露出严重幻觉问题。为此,我们提出DEPTH框架,将依存感知的句子简化与两级分层精炼集成到关系抽取流程中。给定句子和候选实体对,DEPTH分两阶段运行:(1) 基线模块通过提取每对实体的最短依存路径,将句子浓缩为最小但连贯的关系上下文,降低句法噪声并保留关键语义;(2) 精炼模块聚合所有局部预测,并基于全句整体理解修正遗漏与不一致。我们进一步引入因果驱动的奖励模型,通过解耦虚假相关性来缓解奖励劫持,支持通过人类反馈强化学习实现鲁棒微调。在八个主流基准上的实验表明,DEPTH将平均幻觉率降至7.9%,同时在平均F1分数上相比现有基于LLM的基线提升9.3%。
原文摘要 · Abstract (English)
Relation extraction (RE) enables the construction of structured knowledge for many downstream applications. While large language models (LLMs) have shown great promise in this task, they often struggle to reliably determine whether a relation exists, particularly in sentences with complex syntax or subtle semantics. For instance, we find that Qwen2.5-14B-Instruct incorrectly predicts a relation in 96.9% of NO-RELATION instances on SciERC, revealing a severe hallucination problem. To address these challenges, we propose DEPTH, a framework that integrates Dependency-aware sEntence simPlification and Two-tiered Hierarchical refinement into the relation extraction pipeline. Given a sentence and its candidate entity pairs, DEPTH operates in two stages: (1) the Grounding module extracts relations for each pair by leveraging their shortest dependency path, distilling the sentence into a minimal yet coherent relational context that reduces syntactic noise while preserving key semantics; (2) the Refinement module aggregates all local predictions and revises them based on a holistic understanding of the sentence, correcting omissions and inconsistencies. We further introduce a causality-driven reward model that mitigates reward hacking by disentangling spurious correlations, enabling robust fine-tuning via reinforcement learning with human feedback. Experiments on eight well-established benchmarks demonstrate that DEPTH reduces the average hallucination rate to 7.9% while achieving a 9.3% improvement in average F1 score over existing LLM-based extraction baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。