修复多智能体多模态大模型合并后的弱任务缺陷，提升通用性。
AgentPatch: Coarse-to-Fine Weak-Task Repair for Merging Agentic Multimodal Large Language Models
- 分步修复：先选稳定主干，再恢复弱任务信号，最后补全关键行为
- 在6个基准上显著改善合并模型的弱任务表现，保留搜索与视觉推理能力
- 无需训练、不依赖路由，适合打造统一通用多模态智能体
多智能体多模态大语言模型（MLLM）通过规划、工具使用和动态环境交互扩展了多模态感知与推理能力。然而现有模型仅针对特定工具或环境优化，难以整合为统一通用模型。本文提出‘合并’问题并识别两大挑战：能力不对称保留（不同交互复杂度的能力保留不均），导致合并后出现弱任务；行为关键遗忘（关键动作丢失）会破坏长周期执行。为此提出AgentPatch——一种无训练的粗到细修复框架。它选择稳定合并主干，通过弱任务专属残差恢复重建稀释的弱任务信号，并应用智能体引导的行为关键补丁，在显式保护能力的前提下恢复关键行为。该方法生成单一静态检查点，无需路由或集成。跨六项多智能体与多模态基准的实验表明，AgentPatch可有效提升多种合并主干，缓解弱任务退化，更好平衡弱任务恢复与互补搜索及智能体视觉处理能力的保留。代码已开源。
原文摘要 · Abstract (English)
Agentic multimodal large language models (MLLMs) extend multimodal perception and reasoning with planning, tool use, and interaction in dynamic environments. Yet current models are specialized for particular tools or environments, complicating consolidation into a single generalist. We formulate Agentic MLLM Merging and identify two challenges: asymmetric capability preservation, whereby capabilities with different interaction complexity are retained unevenly, producing weak tasks after merging, and behavior-critical forgetting, whereby losing decisive actions can derail long-horizon execution. We propose AgentPatch, a training-free coarse-to-fine repair framework. It selects a stable merged backbone, restores diluted weak-task-specific signals through Weak-Task Unique Residual Recovery, and applies an Agent-Guided Behavior-Critical Patch that recovers decisive behaviors under explicit capability protection. AgentPatch produces a single static checkpoint without routing or ensembles. Experiments across six agentic and multimodal benchmarks show that AgentPatch improves diverse merged backbones, alleviates weak-task degradation, and better balances weak-task recovery with the preservation of complementary search and agentic visual processing capabilities. Code is available at https://github.com/ziboshao/AgentPatch.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。