提出因果测量与闭环调度,显著提升生成搜索中上下文利用效率
The Laws of Context Allocation: Causal Measurement and Closed-Loop Orchestration in Generative Search
- 用因果消融探针替代传统相关性指标,精准测量大模型对证据的依赖程度
- 迭代式分步生成使组合召回率提升16.7至20.5个百分点,32B模型下仍有效
- 构建闭环调度器,通过反向注意力引导实现证据持续融合,适合复杂生成任务
随着检索增强生成(RAG)向多样化结果生成演进,其面临两大瓶颈:证据利用度测量不准确,以及上下文预算分配不合理。本文依次解决这两个问题。首先,揭示了普遍存在的“诊断幻觉”:标准相关性代理在困难负例上表现灾难性失败。为此,引入高效的因果留一探针,精确分离生成依赖,并形式化校准大模型注意力的结构稀释效应。其次,将该因果探针用于去混淆的因子网格实验,证明主流的单一上下文扩展策略是导致相关性衰减的架构陷阱。相反,跨多次生成迭代分配算力可带来16.7–20.5个百分点的绝对召回率提升,且在32B模型上依然稳健。最后,将两者整合为可部署的闭环子模调度器,结合归因驱动的对比解码器,克服大模型注意力惯性,系统性强制新证据融入。相比经典开环基线,本方法确立了序列化、反馈驱动的生成搜索范式。代码、数据及因果测量工具已开源。
原文摘要 · Abstract (English)
As Retrieval-Augmented Generation (RAG) shifts toward diverse portfolio generation, it is stymied by two critical bottlenecks: flawed measurement of evidence utilization, and suboptimal context budget allocation. We resolve both sequentially. To resolve measurement, we expose a pervasive ``diagnostic illusion'': standard relevance proxies fail catastrophically on hard negatives. We replace them with an efficient causal leave-one-out probe that accurately isolates generative reliance and formally calibrates the structural dilution of LLM attention. To resolve allocation, we deploy this causal probe in a deconfounded factorial grid. We prove that the prevailing strategy of monolithic context widening is an architectural trap penalized by relevance decay. Instead, allocating compute iteratively across multiple sequential generations drives transformative portfolio recall gains of 16.7--20.5 absolute percentage points, scaling robustly up to 32B models. Finally, we unify these solutions into a deployable closed-loop submodular scheduler. Augmented by an attribution-steered contrastive decoder to override LLM attention inertia, our architecture systematically forces fresh evidence integration. By dominating classical open-loop baselines, we establish sequential, feedback-driven orchestration as the definitive paradigm for generative search. Our code, data, and causal measurement instruments are available at https://github.com/PeiYangLiu/ascp.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。