提出流映射蒸馏,让图像修复模型更精准地学习教师网络的动态特征关系。
Flow-Map Distillation on Relation Manifolds for Image Restoration

- 将知识蒸馏建模为关系流形上的连续映射,直接预测任意时间点的关系状态。
- 在五类图像修复任务中均超越现有方法,训练方差降低约50%。
- 适合追求高精度与稳定性的图像修复模型压缩研究者使用。
图像修复中的知识蒸馏通常将中间特征或关系矩阵作为静态目标进行对齐,忽略了知识迁移过程的动态性。本文提出关系流形上的流映射蒸馏(FoRM),将基于关系的知识迁移重新定义为关系流形上的连续流映射问题。不同于回归恒定速度场,FoRM学习一个流映射算子 $\mathcal{F}_θ(\mathbf{z}, t, s)$,可直接根据时间 $t$ 的当前状态预测目标时间 $s$ 的关系状态,实现更丰富的轨迹级监督。为保证流映射的全局自洽性,引入安全半群一致性约束,利用真实桥接状态强制组合一致性,消除虚态误差累积;端点锚定损失进一步防止算子偏离教师目标。在超分辨率、去雨、去噪、去模糊和低光增强等五项图像修复任务上,实验表明其在多种骨干网络下持续优于最先进蒸馏基线,相比朴素流匹配蒸馏训练方差降低约50%,同时恢复质量更优。
原文摘要 · Abstract (English)
Knowledge distillation for image restoration typically aligns intermediate features or relation matrices between teacher and student networks as static targets, ignoring the dynamic structure of the knowledge transfer process. In this paper, we propose Flow-Map Distillation on Relation Manifolds (FoRM), which reformulates relation-based knowledge transfer as a continuous flow mapping problem on the relation manifold. Rather than regressing a constant velocity field between student and teacher relation states, FoRM learns a flow map operator $\mathcal{F}_θ(\mathbf{z}, t, s)$ that directly predicts the relation state at any target time $s$ given the current state at time $t$, enabling richer trajectory-level supervision. To ensure global self-consistency of the learned flow map, we introduce a safe semigroup consistency constraint that enforces compositional agreement using ground-truth bridge states, eliminating phantom-state error accumulation. An endpoint anchoring loss further prevents the operator from drifting away from the teacher target. Extensive experiments on five image restoration tasks, including super-resolution, deraining, denoising, deblurring, and low-light enhancement, demonstrate consistent gains over state-of-the-art distillation baselines across multiple backbone architectures, reducing training variance by approximately 50\% compared to naive flow matching distillation while achieving superior restoration quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。