修复大模型流水线瓶颈时,错选模块反而会恶化性能。
Diagnosis Is Not Prescription: Linguistic Co-Adaptation Explains Patching Hazards in LLM Pipelines
- 发现路由模块是故障主因,但直接修正它反致性能下降。
- 上游查询重写模块修复能稳定提升效果,且结果显著。
- 提出语言契约理论解释为何下游修复更安全,适合调试者参考。
当多模块大模型代理失败时,最应修复的模块未必是最佳干预点。我们实证揭示了这一诊断悖论:在三个独立代理家族中,因果分析始终将路由模块(负责选择下一工具)识别为关键瓶颈。然而,在该模块注入提示级修正示例会持续损害性能,有时严重恶化。相反,修复上游查询重写模块则可靠提升结果。该效应在两个代理家族中具有统计显著性,在第三个家族中具方向一致性;在路由模块采用指令重写或模型升级等替代修复策略均无显著影响,确认伤害特异于修正注入。我们通过语言契约假说解释此不对称性:下游模块隐式适应上游的错误分布,纠正瓶颈会破坏这种隐式对齐,而上游修正则不会。我们基于诊断数据构建了每代理共适应度量,其与修补伤害高度相关:共适应越高,伤害越强;共适应越低,越安全。该趋势在三个代理家族中一致出现,为该假说提供了超越单一案例的初步支持。
原文摘要 · Abstract (English)
When a multi-module LLM agent fails, the module most responsible for the failure is not necessarily the best place to intervene. We demonstrate this Diagnostic Paradox empirically: causal analysis consistently identifies the routing module -- which selects which tool to call next -- as the primary bottleneck across three independent agent families. Yet injecting prompt-level correction examples into this module consistently degrades performance, sometimes severely. Patching an upstream query-rewriting module instead reliably improves outcomes. The effect holds with statistical significance on two agent families and directional consistency on a third; alternative repair strategies at the routing module (instruction rewriting, model upgrade) are neutral, confirming that the harm is specific to correction-injection patching. We explain this asymmetry through the Linguistic Contract hypothesis: each downstream module implicitly adapts to its upstream's characteristic error distribution, so correcting the bottleneck breaks this implicit alignment in a way that upstream corrections do not. We operationalize this via a per-agent co-adaptation measure, derived from diagnosis alone, and show it is consistently associated with patching harm across agent families: higher co-adaptation co-occurs with harm, lower with safety. This trend holds across all three agent families, providing preliminary support for the hypothesis beyond a single-agent observation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。