解决电子病历长文本中信息被忽略的临床难题
Inhibitory Attention for Clinical Long-Context Reasoning: Characterizing and Mitigating Lost-in-the-Middle Effects in EHR Processing
- 提出查询感知的抑制注意力机制，动态筛选关键病历片段
- 在中间位置的指令回答准确率提升至25.3%，远超其他方法
- 适合医疗AI研发者与临床推理系统优化人员参考
电子病历现普遍超过每患者10万词元。大语言模型存在丢失中间信息（LitM）现象：长上下文中心区域的信息召回率显著低于两端。在临床场景中尤为严重——最关键的诊断信息常位于记录中部。我们首次系统刻画此临床丢失中间问题（CLitM），基于MedAlign数据集分析六种模型、2196个指令-响应对，发现峰值准确率（59.5%）与谷值准确率（37.6%）相差21.9个百分点，67.8%的正确答案位于病历时间轴的10%-90%区间内，处于性能低谷区。提出轻量级的查询条件临床抑制机制（QCCS），在83个保留指令上对比测试，使用Qwen2.5-7B-Instruct（16k上下文）时，对中间位置指令的准确率达16.7%，显著优于BM25（3.3%）、交叉编码器（0.0%）、密集检索（0.0%）及全上下文（6.7%）。即使未检索到黄金证据句，QCCS仍达25.0%准确率，而其他方法即使检索到也仅2.6%准确。证明查询对齐的上下文选择比黄金句召回更能预测指令遵循效果。
原文摘要 · Abstract (English)
Electronic health records now routinely exceed 100,000 tokens per patient. Yet large language models exhibit the lost-in-the-middle (LitM) effect: information near the center of a long context is retrieved less reliably than information near the edges. In clinical use this is not benign: the single most consequential fact in a note can sit at its center. We term this the clinical lost-in-the-middle (CLitM) problem, give its first systematic characterization using MedAlign, and compare context-selection strategies as remedies. Across 2,196 instruction-response pairs and six language models, we observe a 21.9 percentage-point gap between peak accuracy (59.5%, 95% CI [46.3, 71.0], 20-30% decile) and trough accuracy (37.6% [23.2, 52.5] at 70-80%); 67.8% of reference answers fall between the 10th and 90th percentiles of the EHR timeline, inside the CLitM trough. We introduce Query-Conditioned Clinical Suppression (QCCS), a lightweight query-conditioned selection gate, and evaluate it against BM25, BM25 with section-header filtering, dense retrieval, and cross-encoder reranking (N=83 held-out instructions). With Qwen2.5-7B-Instruct (16k context), QCCS outperforms all five comparators under LLM-as-judge scoring: for middle-position instructions QCCS reaches 16.7% versus BM25 3.3%, cross-encoder 0.0%, dense 0.0%, and full context 6.7%; overall QCCS reaches 25.3% versus at most 3.6% for retrieval-only comparators. This advantage is not explained by retrieval recall: at k=20, BM25 retrieves the gold evidence sentence in 98.8% of instructions (QCCS 34.9%), yet retrieval arms stay at most 2.6% accurate even when they retrieve it, whereas QCCS reaches 25.0% even when it does not. In this proof-of-concept evaluation, query-aligned context selection predicts EHR instruction-following accuracy better than gold-sentence retrieval recall.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。