解决医学问答中否定信息误判问题,提升答案可信度。
Negation is Not Semantic: Diagnosing Dense Retrieval Failure Modes for Trade-offs in Contradiction-Aware Biomedical QA
- 用解耦词法架构分离语义与否定信号,避免向量空间混淆。
- 矛盾检测F1达0.750,引用覆盖率达98.77%,无矛盾引用错误。
- 适合需要高可信医学AI系统的研发者和评测人员。
大型语言模型在生物医学问答中表现强劲,但其生成看似合理却未经验证的内容在临床场景中存在严重风险。TREC 2025 BioGen任务要求答案需显式呈现矛盾证据(任务A)并生成带完整引用的叙事性回答(任务B)。由于缺乏真实标注,我们基于SciFact数据集构建代理开发框架,系统优化检索架构。迭代评估揭示“简单性悖论”:复杂对抗性稠密检索策略因语义坍缩导致矛盾检测失败(MRR 0.023),否定信号在向量空间中无法区分。进一步发现“检索不对称性”:过滤嵌入虽提升矛盾检测,却降低支持文档召回率。为此提出基于统一BM25骨架的解耦词法架构,在保持支持召回率0.810的同时实现0.750的精准矛盾揭示。该方法在代理基准上取得最高加权MRR(0.790),且唯一可扩展至3000万条PubMed文档。答案生成方面引入叙事感知重排序与单次上下文学习,将引用覆盖率从零样本的50%提升至100%。官方TREC结果证实:系统在任务A矛盾F1中排名第二,在任务B引用覆盖率50个运行中位列第三(98.77%),矛盾引用率为零。本工作使大模型从随机生成转向诚实证据合成,表明医学AI的可信性需依赖精度与架构可扩展性的协同设计。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have demonstrated strong capabilities in biomedical question answering, yet their tendency to generate plausible but unverified claims poses serious risks in clinical settings. To mitigate these risks, the TREC 2025 BioGen track mandates grounded answers that explicitly surface contradictory evidence (Task A) and the generation of narrative driven, fully attributed responses (Task B). Addressing the absence of target ground truth, we present a proxy-based development framework using the SciFact dataset to systematically optimize retrieval architectures. Our iterative evaluation revealed a "Simplicity Paradox": complex adversarial dense retrieval strategies failed catastrophically at contradiction detection (MRR 0.023) due to Semantic Collapse, where negation signals become indistinguishable in vector space. We further identify a Retrieval Asymmetry: filtering dense embeddings improves contradiction detection but degrades support recall, compromising reliability. We resolve this via a Decoupled Lexical Architecture built on a unified BM25 backbone, balancing semantic support recall (0.810) with precise contradiction surfacing (0.750). This approach achieves the highest Weighted MRR (0.790) on the proxy benchmark while remaining the only viable strategy for scaling to the 30 million document PubMed corpus. For answer generation, we introduce Narrative Aware Reranking and One-Shot In-Context Learning, improving citation coverage from 50% (zero-shot) to 100%. Official TREC results confirm our findings: our system ranks 2nd on Task A contradiction F1 and 3rd out of 50 runs on Task B citation coverage (98.77%), achieving zero citation contradict rate. Our work transforms LLMs from stochastic generators into honest evidence synthesizers, showing that epistemic integrity in biomedical AI requires precision and architectural scalability isolated metric optimization.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。