用层次与局部性先验实现通用信号的高效神经场编码
Neural Field Tokenizations with Hierarchy and Spatial Locality Priors
- 设计分层局部编码器，将坐标-值数据转为结构化令牌
- 内存降低42倍，批量大小提升133倍，仍保持高重建精度
- 适用于图像、3D形状、气候场等多模态数据，通用性强
神经场将数据建模为从坐标到数值的函数，为跨模态表征学习提供统一框架。现有方法多依赖每样本元学习，因内循环优化内存开销大，难以扩展。而前馈编码通常引入模态特异性假设，损害神经场的通用性。本文提出LH-NeF框架，通过注入局部性和层次性先验，在不牺牲模态无关性的前提下学习通用的令牌化表示。其层次局部编码器将原始坐标-值场观测映射为结构化令牌，并在训练中用于场重建。相比元学习的内循环，仅需单次前向传播，内存使用减少42倍，支持133倍更大的批量。在图像、3D形状和气候场任务上，该方法的重建性能及下游任务表现均达到或超过模态无关、模态特定及专用生成神经场基线。
原文摘要 · Abstract (English)
Neural fields parameterize data as functions from coordinates to values, providing a unified framework for representation learning across modalities. Existing approaches are dominated by per-sample meta-learning, which scales poorly due to memory-intensive inner-loop optimization. The natural alternative -- feed-forward encoding -- typically introduces modality-specific assumptions, sacrificing the generality that makes learning with neural fields attractive. We argue that locality and hierarchy are useful priors for learning field representations that can be injected without compromising modality-agnosticism. We propose LH-NeF, a framework to learn general-purpose tokenized representations of continuous signals. A locality-preserving hierarchical encoder maps raw coordinate-value field observations to structured tokens, from which the field is reconstructed during training. By replacing meta-learning's inner loop with a single forward pass, LH-NeF uses 42$\times$ less memory and supports 133$\times$ larger batches than the strongest modality-agnostic baseline. Across images, 3D shapes, and climate fields, our learned representations match or exceed performance of modality-agnostic, modality-specific, and specialized generative neural field baselines on both reconstruction and downstream tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。