DiTTo能自动排序多种图像退化,高效实现高质量修复。
DiTTo: Scalable Order-aware All-in-One Image Restoration Agent

- 用模拟器预估每步修复效果,将训练开销从平方级降到线性级
- 在多退化图像上达到当前最优修复质量,比现有方法更优
- 新增修复工具只需微调模块,无需重新训练整个系统
真实图像常同时遭受多种退化,且修复顺序显著影响最终效果,推动了基于智能体的图像修复发展。然而,现有训练型智能体需对每张图像调用$/mathcal{O}((N^{ extbf{D}})^2)$次修复专家,构建最优修复路径数据集(ORTD),且受限于固定专家池,无法灵活扩展。为此,本文提出新框架DiTTo,由模拟器与智能体组成:模拟器结合$igcup$S-IR单步修复模拟与AiO-IQA动作质量预测,将构造ORTD的开销降至$/mathcal{O}(N^{ extbf{D}})$;智能体在模拟生成的ORTD上经SFT训练后,通过有序修复对齐(ORA)机制,在退化识别、修复顺序和输出格式三方面独立对齐。该设计实现即插即用的可扩展性:新增修复专家仅需更新轻量级ORA模块。在包含最多五种并发退化的MiO-100测试集上,所提DiTTo智能体在同类方法中达到最优修复性能。
原文摘要 · Abstract (English)
Real-world images rarely suffer from a single degradation, and the order in which degradations are removed substantially affects the final restoration quality, motivating agent-based image restoration (IR), where a vision-language model schedules a pool of pre-built restoration-experts. However, existing training-based agents require $\mathcal{O}((N^{\mathbf{D}})^{2})$ restoration-expert calls per image to construct the Optimal Restoration-action Trajectory Dataset (ORTD), where $N^{\mathbf{D}}$ denotes the number of degradation types in the universe $\mathbf{D}$, and couple agent training to a fixed restoration-expert pool, preventing extension to newly introduced restoration-experts without full retraining. To overcome these efficiency and extensibility bottlenecks, we propose \textbf{DiTTo}, a novel order-aware image restoration agent framework consisting of the DiTTo Simulator and the DiTTo Agent. The DiTTo Simulator combines $\cup$S-IR for single-step restoration-action simulation and AiO-IQA for per-action quality prediction, reducing ORTD construction to $\mathcal{O}(N^{\mathbf{D}})$ simulator calls per image; the DiTTo Agent is trained by SFT on the simulator-generated ORTD, followed by \textbf{Order-aware Restoration Alignment (ORA)} that aligns degradation identification, restoration-action-ordering, and output format along independent axes. This enables \textbf{plug-and-play scalable extensibility}: adding a new restoration-expert requires updating only the lightweight ORA stage. On the MiO-100 evaluation set with up to five concurrent degradations, our DiTTo Agent achieves state-of-the-art multi-degradation restoration quality among previous agent-based IR methods.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。