将医学指南构建时的质量信息融入问答系统,提升答案准确性与可信度。
Propagating construction-time knowledge quality into medical question answering: A framework grounded in clinical guidelines
- 构建双维度质量信号Q(t),融合结构合规性与证据支持度
- 在跨版本和跨模型场景下,质量区分度AUC达0.748,优于单一指标
- 显著减少知识遗漏(16.3%→5.3%)和矛盾输出(16.3%→2.7%),适合临床辅助决策
大语言模型虽可从临床指南中构建知识图谱,但抽取三元组的质量参差不齐。现有图增强问答系统多关注查询相关性,未能有效利用构建阶段的质量信息,导致质量控制与推理阶段脱节。本文提出一种质量感知框架,将结构合规性(SchemaConf)与证据支持度(EvidScore)融合为每三元组的质量信号Q(t)。不只用于过滤,而是作为图属性传播至质量加权子图检索与层级条件证据提示,同时保留段落级来源信息。在中文糖尿病指南上的实验表明,跨版本和跨模型迁移下,融合后的Q(t) AUC达0.748,优于EvidScore(0.703)和SchemaConf(0.645)。在指南驱动的问答中,知识遗漏率从16.3%降至5.3%,矛盾输出从16.3%降至2.7%,证据精准率达81.6%,无效引用近乎为零。盲评显示,临床医生评分(4.68)显著高于无检索(4.21),接近理想条件(4.80),跨生成器实验也呈现一致趋势。
原文摘要 · Abstract (English)
Large language models have facilitated knowledge graph (KG) construction from clinical guidelines, but extracted triples vary in structural validity and evidential support. Meanwhile, graph-augmented question answering (QA) systems typically optimize query relevance during retrieval, with limited reuse of quality information produced during KG construction. This creates a disconnect between construction-time quality control and inference-time evidence use. We investigate whether construction-time triple quality can serve as a persistent signal for downstream evidence selection and presentation. We propose a quality-aware framework that models structural conformance (SchemaConf) and evidential support (EvidScore) as complementary dimensions and fuses them into a per-triple quality signal, Q(t). Rather than using quality solely for filtering, the framework retains Q(t) and derived quality tiers as graph attributes and propagates them into quality-weighted subgraph retrieval and tier-conditioned evidence prompting, while preserving passage-level provenance. Experiments on Chinese diabetes clinical guidelines show that the utility of the quality signal is distribution dependent. Under cross-version and cross-model shift, the fused Q(t) provides stronger triple-quality discrimination than either component alone (AUC 0.748 vs. 0.703 for EvidScore and 0.645 for SchemaConf). In guideline-grounded QA, propagating construction-time quality reduces required-knowledge omission from 16.3% to 5.3% and conflicting outputs from 16.3% to 2.7%, with an evidence-grounded precision of 81.6% and near-zero invalid citations. Blinded clinician ratings favor the full framework over no retrieval (4.68 vs. 4.21 on a five-point scale) and approach the oracle condition (4.80), while cross-generator experiments show consistent trends.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。