arXiv:2603.27742cs.CV2026-03被引 2

训练可探索的图像修复智能体,提升修复效率与效果。

TIR-Agent: Training an Explorative and Efficient Agent for Image Restoration

  • 通过两阶段训练构建可直接调用工具的决策策略。
  • 在多种退化场景下优于12个基线模型,推理速度提升2.5倍以上。
  • 适合需要高效自动化图像修复的工业级应用。

视觉-语言代理通过协调专用工具进行图像修复(IR)已成为一种有前景的方法,但现有框架多为无训练模式,依赖启发式任务调度和遍历所有工具,导致修复路径不佳且计算成本高昂。本文认为核心瓶颈在于缺乏学习到的决策策略,因视觉-语言模型难以高效处理退化感知的任务排序与工具组合。为此,提出TIR-Agent,一个可通过监督微调(SFT)与强化学习(RL)两阶段训练的可训练图像修复代理,直接执行工具调用策略。两个关键设计支撑有效强化学习:(i) 对SFT数据施加随机扰动,扩大策略在任务调度与工具组合上的探索范围;(ii) 多维自适应奖励机制,动态重加权异构图像质量指标,缓解奖励欺骗问题。为支持高吞吐、异步的GPU工具调用,构建全局共享的模型调用池。在域内与域外退化数据上实验表明,TIR-Agent超越12个基线,包括6个全合一模型、3个无训练代理和3个专有模型,并通过消除冗余工具执行实现超过2.5倍的推理加速。

原文摘要 · Abstract (English)

Vision-language agents that orchestrate specialized tools for image restoration (IR) have emerged as a promising method, yet most existing frameworks operate in a training-free manner. They rely on heuristic task scheduling and exhaustive tool traversal, resulting in sub-optimal restoration paths and prohibitive computational cost. We argue that the core bottleneck lies in the absence of a learned policy to make decision, as a vision-language model cannot efficiently handle degradation-aware task ordering and tool composition. To this end, we propose TIR-Agent, a trainable image restoration agent that performs a direct tool-calling policy through a two-stage training pipeline of supervised fine-tuning (SFT) followed by reinforcement learning (RL). Two key designs underpin effective RL training: (i) a random perturbation strategy applied to the SFT data, which broadens the policy's exploration over task schedules and tool compositions, and (ii) a multi-dimensional adaptive reward mechanism that dynamically re-weights heterogeneous image quality metrics to mitigate reward hacking. To support high-throughput, asynchronous GPU-based tool invocation during training, we further develop a globally shared model-call pool. Experiments on both in-domain and out-of-domain degradations show that TIR-Agent outperforms 12 baselines, including 6 all-in-one models, 3 training-free agents, and 3 proprietary models, and achieves over 2.5$\times$ inference speedup by eliminating redundant tool executions.

图像修复智能体强化学习工具调用

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。