用局部神经场实现高分辨率图像重建，内存占用极低且泛化能力强。
LoFi: Neural Local Fields for Scalable Image Reconstruction
- 基于坐标分解的局部神经场框架，逐点独立重建图像。
- 1024x1024图像训练仅需<200MB内存，显著低于CNN和ViT。
- 支持极小数据集（<10样本）训练，无需正则化或早停。
神经场或隐式神经表示（INRs）因其基于坐标的高效图像与3D体数据表征，在计算机视觉与成像领域备受关注。本文提出一种基于坐标的成像逆问题求解框架——LoFi（Local Field）。不同于传统图像重建方法，LoFi通过多层感知机（MLPs）在每个坐标处独立处理局部信息，恢复该点的物体值。与INRs类似，LoFi可在任意连续坐标上重建图像，支持多分辨率输出。其性能与标准深度学习模型（如卷积神经网络CNN和视觉变压器ViT）相当甚至更优，且对分布外数据具有优异泛化能力，内存使用几乎与图像分辨率无关。值得注意的是，1024x1024图像训练仅需不足200MB内存，远低于典型CNN和ViT。此外，由于其局部设计，LoFi可在极小数据集（10样本或更少）上训练而不过拟合，无需显式正则化或早停策略。
原文摘要 · Abstract (English)
Neural fields or implicit neural representations (INRs) have attracted significant attention in computer vision and imaging due to their efficient coordinate-based representation of images and 3D volumes. In this work, we introduce a coordinate-based framework for solving imaging inverse problems, termed LoFi (Local Field). Unlike conventional methods for image reconstruction, LoFi processes local information at each coordinate separately by multi-layer perceptrons (MLPs), recovering the object at that specific coordinate. Similar to INRs, LoFi can recover images at any continuous coordinate, enabling image reconstruction at multiple resolutions. With comparable or better performance than standard deep learning models like convolutional neural networks (CNNs) and vision transformers (ViTs), LoFi achieves excellent generalization to out-of-distribution data with memory usage almost independent of image resolution. Remarkably, training on 1024x1024 images requires less than 200MB of memory -- much below standard CNNs and ViTs. Additionally, LoFi's local design allows it to train on extremely small datasets with 10 samples or fewer, without overfitting and without the need for explicit regularization or early stopping.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。