将风格建模与迁移解耦,实现高效多风格图像迁移。
Pluggable Style Representation Learning for Multi-Style Transfer
- 用紧凑表示学习风格信息,解耦风格建模与迁移过程。
- 无需增加推理开销即可支持多种风格,性能超越现有方法。
- 适合资源受限设备部署,兼顾精度与效率。
由于图像风格多样性高,扩展至多种风格在实际应用中至关重要。以往多风格迁移方法通过增大模型规模来适应多样风格,而任意风格迁移方法则依赖重型主干网络。然而,更多参数带来的额外计算开销限制了这些方法在资源受限设备上的部署。为此,本文提出一种解耦风格建模与迁移的风格迁移框架。针对风格建模,设计了一种风格表征学习方案,将风格信息编码为紧凑表示;针对风格迁移,构建了风格感知多风格迁移网络(SaMST),通过可插拔的风格表示适配多样化风格。该框架在不引入推理额外开销的前提下,实现对多种风格的灵活支持,保持高效性。实验表明,所学风格表示能准确提取风格信息;定性和定量结果均显示,本方法在精度与效率上达到当前最优水平。
原文摘要 · Abstract (English)
Due to the high diversity of image styles, the scalability to various styles plays a critical role in real-world applications. To accommodate a large amount of styles, previous multi-style transfer approaches rely on enlarging the model size while arbitrary-style transfer methods utilize heavy backbones. However, the additional computational cost introduced by more model parameters hinders these methods to be deployed on resource-limited devices. To address this challenge, in this paper, we develop a style transfer framework by decoupling the style modeling and transferring. Specifically, for style modeling, we propose a style representation learning scheme to encode the style information into a compact representation. Then, for style transferring, we develop a style-aware multi-style transfer network (SaMST) to adapt to diverse styles using pluggable style representations. In this way, our framework is able to accommodate diverse image styles in the learned style representations without introducing additional overhead during inference, thereby maintaining efficiency. Experiments show that our style representation can extract accurate style information. Moreover, qualitative and quantitative results demonstrate that our method achieves state-of-the-art performance in terms of both accuracy and efficiency. The codes are available in https://github.com/The-Learning-And-Vision-Atelier-LAVA/SaMST.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。