解决多语言检索生成中的中英文证据冲突问题，提升答案准确性。
X-MADAM-RAG: Diagnosing and Handling Chinese-English Evidence Conflict in Retrieval-Augmented Generation
- 设计分步处理流程，逐文档提取候选、修复可见证据、分组并感知冲突。
- 在受控数据集上严格准确率达0.9667，冲突感知成功率0.9767。
- 适用于研究多语言RAG系统鲁棒性，适合对齐与评估模型推理能力。
检索增强生成（RAG）系统可能接收到不仅噪声大且相互矛盾的证据。这一问题在多语言场景下尤为突出，中文和英文检索结果可能支持不一致的答案候选。我们通过X-RAMDocs-ZHEN——一个源自RAMDocs的中英文对照可控基准，诊断RAG中的证据冲突。该基准包含300个样本，涵盖六种平衡条件：单语支持、双语一致、相反冲突方向及含可选噪声的冲突。我们进一步提出X-MADAM-RAG，一个可解释的流水线，将证据处理分解为每文档候选提取、可见证据修复、确定性候选分组和冲突感知聚合。在原始基准上使用Qwen2.5-7B-Instruct，X-MADAM-RAG实现0.9667严格准确率和0.9767冲突感知成功率，优于证据归一化的单次调用基线。然而，仅规则的零调用提取器在相同基准上达到1.0000，揭示强烈模板规律性。为探究此局限，我们构建一个确定性的自然化压力测试，移除显式答案模板但保留候选字符串。在100样本子集上，规则提取降至0.0000，而X-MADAM-RAG也降至0.3000严格准确率，低于朴素和证据归一化基线。特权预言机仍保持完美表现，表明文档级提取是主要瓶颈。这些发现将X-RAMDocs-ZHEN和X-MADAM-RAG定位为控制性证据冲突诊断工具，而非通用幻觉检测或自然检索鲁棒性的证明。
原文摘要 · Abstract (English)
Retrieval-augmented generation (RAG) systems may receive evidence that is not merely noisy but mutually contradictory. This issue becomes particularly salient in multilingual settings, where retrieved Chinese and English evidence may support incompatible answer candidates. We study this problem through X-RAMDocs-ZHEN, a controlled Chinese-English benchmark derived from RAMDocs for diagnosing evidence conflict in RAG. The benchmark contains 300 examples across six balanced conditions, including monolingual support, bilingual agreement, reversed conflict directions, and conflict with optional noise. We further examine X-MADAM-RAG, an interpretable pipeline that decomposes evidence handling into per-document candidate extraction, visible-evidence repair, deterministic candidate grouping, and conflict-aware aggregation. On the original controlled benchmark with Qwen2.5-7B-Instruct, X-MADAM-RAG achieves 0.9667 strict accuracy and 0.9767 conflict-aware success, outperforming an evidence-normalized single-call baseline. However, a zero-call rule-only extractor reaches 1.0000 on the same benchmark, revealing strong template regularity. To probe this limitation, we construct a deterministic naturalized stress test that removes explicit answer templates while preserving candidate strings. On its 100-sample subset, rule-only extraction falls to 0.0000, but X-MADAM-RAG also drops to 0.3000 strict accuracy, below both naive and evidence-normalized baselines. A privileged oracle remains perfect, indicating that document-level extraction is the main bottleneck. These findings position X-RAMDocs-ZHEN and X-MADAM-RAG as diagnostic tools for controlled evidence conflict rather than as evidence of general hallucination detection or robustness to natural retrieval.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。