提升医疗AI事实准确性，通过衡量文本中真实信息密度来优化检索。
Evaluating Factual Density in Multi-Source RAG: A Study in Medical AI Accuracy
- 引入事实密度（FD*）作为新检索信号，评估文本中真实命题占比。
- 在健康事实基准上，优化后检索在前5条结果中实现100%系统综述覆盖。
- 适合关注医疗AI可信赖性、需提升事实精度的研究者与开发者。
检索增强生成（RAG）是当前将AI与真实世界知识结合的行业标准。传统检索依赖关键词匹配和主题相似度，但忽视内容中实际包含的已验证事实数量。这种结构缺陷被称为‘专家盲视效应’，导致高事实密度证据被词频高的文本掩盖。本文提出事实密度（FD*），通过计算已验证原子命题占总词元的比例来量化事实密度。借助NexusAgentics Ghost Audit预处理流程，利用概率事实性分析对原始文本进行事实特异性评分，并在语料入库前过滤内容。初始版本存在严重的文档长度偏差（皮尔逊相关系数R = -0.8636，p = 2.27e-07）。通过在长度分组内采用Z-score归一化解决该偏差，验证了FD*为长度无关的事实密度信号（p = 0.0749）。在HealthFC基准（750个由医学专家标注为支持、反驳或无证据的健康主张）上的评估显示，FD*优化的检索是唯一在前5条结果中实现100%系统综述饱和的条件，成功召回了标准余弦相似度排在前十之外的Cochrane证据。真实情况验证确认了七个支持性主张中的25个映射关系。尽管因语料与基准对齐限制，完整统计验证（n=50查询）尚待未来工作，但结果确立了事实密度重排序是一种低成本、高效益的提升医疗RAG事实精确性的方法。
原文摘要 · Abstract (English)
Retrieval-Augmented Generation (RAG) is the current industry standard for grounding AI in real-world facts. Traditional retrieval methods rely on keyword matching and topic proximity, ranking content based on how closely it sounds like the user's query. What they do not measure is how many verified facts the content actually contains. This structural gap, termed the Expert Blindness Effect, causes standard RAG pipelines to consistently bury high-density factual evidence in favor of lexically dominant text on the same topic. To address this gap, this paper introduces Factual Density (FD*), a novel retrieval optimization signal that measures the proportion of verified atomic claims relative to total token count. Using the NexusAgentics Ghost Audit preprocessing pipeline, raw text is scored for factual specificity using probabilistic factuality analysis to filter content before corpus ingestion. An initial formulation introduced a severe document-length confound (Pearson R = -0.8636, p = 2.27e-07). Implementing Z-score normalization within length bins resolved this bias, validating FD* as a length-independent density signal (p = 0.0749). Evaluated against the HealthFC benchmark (750 health claims labeled Supported, Refuted, or No Evidence by medical experts), FD*-optimized retrieval was the only condition to achieve 100% systematic review saturation in top-5 results, surfacing Cochrane evidence that standard cosine similarity ranked outside the top ten. Ground truth verification confirmed 25 mappings across seven HealthFC-supported claims. While full statistical validation across n=50 queries remains future work due to constraints on corpus-benchmark alignment, these findings establish factual density reranking as a low-cost, high-impact intervention for improving factual precision in health RAG architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。