用RGB神经网络实现多光谱纹理合成，无需重新训练模型。
Multispectral Texture Synthesis using RGB Convolutional Neural Networks
- 通过随机光谱波段组合优化，适配多光谱数据
- 将多光谱像素投影到3维空间，保持纹理结构
- 适合需要跨光谱纹理生成的研究者
当前先进的RGB纹理合成方法依赖于深度特征的统计风格距离。这些特征由在大型RGB图像数据集上预训练的分类网络提取。将此类方法扩展至多光谱图像并不直接，因为预训练网络仅针对RGB图像设计。本文提出两种无需额外训练特征提取网络的方法：一是训练过程中优化随机三波段组合；二是将多光谱像素投影至三维空间。我们进一步探索在投影前进行颜色迁移，以避免投影导致的异常颜色分布。实验通过多种指标对比了不同方法性能，证明其可实现基于样本的纹理合成，视觉质量良好，在RGB波段上接近现有最优水平。
原文摘要 · Abstract (English)
State-of-the-art RGB texture synthesis algorithms rely on style distances that are computed through statistics of deep features. These deep features are extracted by classification neural networks that have been trained on large datasets of RGB images. Extending such synthesis methods to multispectral images is not straightforward, since the pre-trained networks are designed for and have been trained on RGB images. In this work, we propose two solutions to extend these methods to multispectral imaging. Neither of them require additional training of the neural network from which the second order neural statistics are extracted. The first one consists in optimizing over batches of random triplets of spectral bands throughout training. The second one projects multispectral pixels onto a 3 dimensional space. We further explore the benefit of a color transfer operation upstream of the projection to avoid the potentially abnormal color distributions induced by the projection. Our experiments compare the performances of the various methods through different metrics. We demonstrate that they can be used to perform exemplar-based texture synthesis, achieve good visual quality and comes close to state-of-the art methods on RGB bands.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。