无需重训，通用模型可高效提升视觉特征分辨率。
NAF: Zero-Shot Feature Upsampling via Neighborhood Attention Filtering
- 用邻域注意力与旋转位置编码自适应学习权重，实现零样本上采样。
- 在多任务上超越专有模型，2K分辨率下保持18帧/秒速度。
- 适合需要快速适配新模型的图像重建与像素级任务场景。
视觉基础模型（VFMs）提取空间下采样的特征表示，给像素级任务带来挑战。现有上采样方法存在根本性权衡：传统滤波器虽快速通用但依赖固定形式，现代上采样器通过可学习的、针对特定VFM的形式获得更优精度，却需为每个VFM重新训练。我们提出邻域注意力过滤（NAF），通过跨尺度邻域注意力和旋转位置编码（RoPE）自适应学习空间与内容权重，仅依赖高分辨率输入图像进行引导。NAF实现零样本运行：无需重训即可上采任意VFM的特征，是首个在性能上超越专用上采样器且具备广泛适用性的架构，在多个下游任务中达到最先进水平。它保持高效率，可处理2K特征图，在18帧/秒下重构中等分辨率特征图。此外，NAF在图像修复任务中也表现出色，彰显其通用性。代码与检查点见https://github.com/valeoai/NAF。
原文摘要 · Abstract (English)
Vision Foundation Models (VFMs) extract spatially downsampled representations, posing challenges for pixel-level tasks. Existing upsampling approaches face a fundamental trade-off: classical filters are fast and broadly applicable but rely on fixed forms, while modern upsamplers achieve superior accuracy through learnable, VFM-specific forms at the cost of retraining for each VFM. We introduce Neighborhood Attention Filtering (NAF), which bridges this gap by learning adaptive spatial-and-content weights through Cross-Scale Neighborhood Attention and Rotary Position Embeddings (RoPE), guided solely by the high-resolution input image. NAF operates zero-shot: it upsamples features from any VFM without retraining, making it the first VFM-agnostic architecture to outperform VFM-specific upsamplers and achieve state-of-the-art performance across multiple downstream tasks. It maintains high efficiency, scaling to 2K feature maps and reconstructing intermediate-resolution maps at 18 FPS. Beyond feature upsampling, NAF demonstrates strong performance on image restoration, highlighting its versatility. Code and checkpoints are available at https://github.com/valeoai/NAF.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。