组件越多越差？研究发现大模型代理系统中组件间干扰导致性能下降。
More Is Not Always Better: Cross-Component Interference in LLM Agent Scaffolding
- 通过全因子实验测试5个组件的32种组合，发现堆叠越多反而越差
- 单工具代理在HotpotQA上比全组件系统高32%，在GSM8K上高79%
- 最优组件数因任务和模型规模而异，需考虑交互影响选择子集
大型语言模型代理系统常通过叠加规划、工具、记忆、自省和检索等组件构建，假设组件越多越好。本文研究跨组件干扰（CCI）：组件间相互作用导致性能下降的现象。在HotpotQA和GSM8K数据集上，使用Llama-3.1-8B/70B对全部2^5=32种组件组合进行实验（共96组，每组最多10次随机种子）。结果显示，全组件系统（All-In）始终表现最差：在HotpotQA上，单工具代理优于全组件系统32%（F1 0.233 vs 0.177，p=0.023）；在GSM8K上，三组件组合优于全组件系统79%（0.43 vs 0.24，p=0.010）。最优组件数量依赖任务（k*=1–4）且受模型规模影响：70B模型下，某些在8B时有害的组合反而带来提升，但全组件仍不如最佳子集。通过主效应回归（R²=0.916，adj-R²=0.899，LOOCV=0.872）和精确Shapley值分析，发现325个子集中有183个违反子模性（56.3%），表明贪心选择不可靠。探索性发现工具使用、自省与检索三者间存在协同效应（INT_3=+0.175，95% CI [+0.003,+0.351]）。CCI在Qwen2.5模型族和提示改写下依然成立。研究建议用任务导向的子集选择替代默认全组件配置，并基于交互感知分析优化。
原文摘要 · Abstract (English)
LLM agent systems are built by stacking scaffolding components (planning, tools, memory, self-reflection, retrieval) assuming more is better. We study cross-component interference (CCI): degradation when components interact destructively. We run a full factorial experiment over all 2^5=32 subsets of five components on HotpotQA and GSM8K with Llama-3.1-8B/70B (96 conditions, up to 10 seeds). The All-In system is consistently suboptimal: on HotpotQA, a single-tool agent surpasses All-In by 32% (F1 0.233 vs 0.177, p=0.023); on GSM8K, a 3-component subset beats All-In by 79% (0.43 vs 0.24, p=0.010). Optimal component count is task-dependent (k*=1-4) and scale-sensitive: at 70B, combinations that hurt at 8B provide gains, though All-In still trails the best subset. We fit a main-effects regression (R^2=0.916, adj-R^2=0.899, LOOCV=0.872), compute exact Shapley values, and find 183/325 submodularity violations (56.3%), showing greedy selection is unreliable. A three-body synergy among Tool Use, Self-Reflection, and Retrieval (INT_3=+0.175, 95% CI [+0.003,+0.351]) is reported as exploratory. CCI replicates across model families (Qwen2.5) and is robust to prompt paraphrasing. Our findings suggest maximally-equipped agent defaults should be replaced by task-specific subset selection via interaction-aware analysis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。