arXiv:2603.17055cs.CV2026-03被引 1

让图像修复智能体学会总结经验,自动选对工具修复肖像图。

PaAgent: Portrait-Aware Image Restoration Agent via Subjective-Objective Reinforcement Learning

  • 用可自进化的人物特征库+检索生成技术,智能选择修复工具。
  • 在8个基准上测试,对复杂退化场景修复效果优于现有方法。
  • 适合需要精准修复人像的设计师或图像处理工程师使用。

图像修复(IR)智能体通过多模态大语言模型感知退化并调用修复工具,已展现出自动化修复的潜力。然而,现有智能体缺乏对过往交互的洞察总结机制,导致需反复尝试才能找到最优修复工具。为此,我们提出一种肖像感知的图像修复智能体PaAgent,引入自演化的人物特征库与检索增强生成(RAG),根据输入图像选择合适的修复工具。具体而言,PaAgent通过持续总结修复后的图像、选用的工具及原始退化图像的特征,不断丰富和演化人物特征库;同时利用RAG从特征库中检索相关洞察,以指导工具选择。此外,为提升复杂场景下的退化感知能力,我们设计了一种主客观强化学习策略,在奖励计算中融合图像质量评分与语义洞察,即使在局部或非均匀退化下也能准确捕捉退化信息。在涵盖六类单退化和八类混合退化场景的8个图像修复基准上的实验表明,PaAgent在应对复杂修复任务时具有显著优势。项目主页:https://wyjgr.github.io/PaAgent.html

原文摘要 · Abstract (English)

Image Restoration (IR) agents, leveraging multimodal large language models to perceive degradation and invoke restoration tools, have shown promise in automating IR tasks. However, existing IR agents typically lack an insight summarization mechanism for past interactions, which results in an exhaustive search for the optimal IR tool. To address this limitation, we propose a portrait-aware IR agent, dubbed PaAgent, which incorporates a self-evolving portrait bank for IR tools and Retrieval-Augmented Generation (RAG) to select a suitable IR tool for input. Specifically, to construct and evolve the portrait bank, the PaAgent continuously enriches it by summarizing the characteristics of various IR tools with restored images, selected IR tools, and degraded images. In addition, the RAG is employed to select the optimal IR tool for the input image by retrieving relevant insights from the portrait bank. Furthermore, to enhance PaAgent's ability to perceive degradation in complex scenes, we propose a subjective-objective reinforcement learning strategy that considers both image quality scores and semantic insights in reward generation, which accurately provides the degradation information even under partial and non-uniform degradation. Extensive experiments across 8 IR benchmarks, covering six single-degradation and eight mixed-degradation scenarios, validate PaAgent's superiority in addressing complex IR tasks. Our project page is \href{https://wyjgr.github.io/PaAgent.html}{PaAgent}.

图像修复智能体肖像处理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。