对比了不同方法在多跳推理中的表现，发现强模型才适合用多代理增强。
Scaling Trends for Multi-Hop Contextual Reasoning in Mid-Scale Language Models
- 用多代理系统提升复杂推理，但仅对能力强的模型有效
- 最强模型提升46.7个百分点，弱模型无改善
- 模型架构优化比参数量更重要，适合研究推理机制的人看
我们对中等规模语言模型中的多跳上下文推理进行了受控研究，揭示了任务与方法间的解耦现象：基于规则的模式匹配在结构化信息检索中达到100%准确率，但在跨文档推理任务中仅6.7%；而基于LLM的多代理系统则在规则方法失败的任务中最高达80%。通过包含120次试验的合成评估框架，测试了四个模型（LLaMA-3 8B、LLaMA-2 13B、Mixtral 8x7B、DeepSeek-V2 16B），得出三个关键结论：(1) 多代理增强依赖基础能力，仅在具备足够推理能力的模型上出现显著提升（LLaMA-3 8B，p<0.001；Mixtral，p=0.014），最高提升46.7个百分点，弱模型无收益，表明是放大而非补偿；(2) 活跃参数预测推理性能：Mixtral的表现与约12B活跃参数一致，而非总参数47B，支持推理能力由推理时计算驱动的假设；(3) 架构质量更关键：尽管参数更少，LLaMA-3 8B仍优于LLaMA-2 13B，符合已知训练改进趋势。结果为多代理协作和MoE扩展提供了受控定量证据，并强调多代理优势对基础模型能力的依赖。评估框架已公开，以支持中等规模模型推理的可复现研究。
原文摘要 · Abstract (English)
We present a controlled study of multi-hop contextual reasoning in large language models, providing a clean demonstration of the task-method dissociation: rule-based pattern matching achieves 100% success on structured information retrieval but only 6.7% on tasks requiring cross-document reasoning, while LLM-based multi-agent systems show the inverse pattern, achieving up to 80% on reasoning tasks where rule-based methods fail. Using a synthetic evaluation framework with 120 trials across four models (LLaMA-3 8B, LLaMA-2 13B, Mixtral 8x7B, DeepSeek-V2 16B), we report three key findings: (1) Multi-agent amplification depends on base capability: statistically significant gains occur only for models with sufficient reasoning ability (p < 0.001 for LLaMA-3 8B, p = 0.014 for Mixtral), with improvements of up to 46.7 percentage points, while weaker models show no benefit, suggesting amplification rather than compensation; (2) Active parameters predict reasoning performance: Mixtral's performance aligns with its ~12B active parameters rather than 47B total, consistent with the hypothesis that inference-time compute drives reasoning capability in MoE architectures; (3) Architecture quality matters: LLaMA-3 8B outperforms LLaMA-2 13B despite fewer parameters, consistent with known training improvements. Our results provide controlled quantitative evidence for intuitions about multi-agent coordination and MoE scaling, while highlighting the dependence of multi-agent benefits on base model capability. We release our evaluation framework to support reproducible research on reasoning in mid-scale models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。