让神经隐式表示学会通用特征,加速图像重建并提升质量
Learning Transferable Features for Implicit Neural Representations
- 共享编码器层,使多个隐式网络复用通用特征
- 新方法使重建质量提升约10dB,初期效果显著优于随机初始化
- 适合需要快速适应新图像或数据先验的场景
隐式神经表示(INRs)在逆问题和神经渲染等任务中表现优异,但通常仅针对单一信号训练,导致特征难以泛化。本文提出STRAINER框架,通过在多个INR间共享初始编码器层、独立解码器层,学习可迁移的神经特征。测试时,将学习到的编码器作为新INR的初始化,显著加快收敛速度并提升重建质量。实验表明,该方法在相同领域内图像拟合任务中能带来约10dB的信号质量增益,远超未训练的INR。同时,它为在INRs中嵌入数据驱动先验提供了简单途径。我们在多种域内与域外信号拟合任务及逆问题上验证了其有效性,并深入分析了特征的可迁移性。演示链接:https://kushalvyas.github.io/strainer.html。
原文摘要 · Abstract (English)
Implicit neural representations (INRs) have demonstrated success in a variety of applications, including inverse problems and neural rendering. An INR is typically trained to capture one signal of interest, resulting in learned neural features that are highly attuned to that signal. Assumed to be less generalizable, we explore the aspect of transferability of such learned neural features for fitting similar signals. We introduce a new INR training framework, STRAINER that learns transferrable features for fitting INRs to new signals from a given distribution, faster and with better reconstruction quality. Owing to the sequential layer-wise affine operations in an INR, we propose to learn transferable representations by sharing initial encoder layers across multiple INRs with independent decoder layers. At test time, the learned encoder representations are transferred as initialization for an otherwise randomly initialized INR. We find STRAINER to yield extremely powerful initialization for fitting images from the same domain and allow for $\approx +10dB$ gain in signal quality early on compared to an untrained INR itself. STRAINER also provides a simple way to encode data-driven priors in INRs. We evaluate STRAINER on multiple in-domain and out-of-domain signal fitting tasks and inverse problems and further provide detailed analysis and discussion on the transferability of STRAINER's features. Our demo can be accessed at https://kushalvyas.github.io/strainer.html .
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。