用逆问题思路提升视觉模型特征分辨率,避免模糊。
Weighted Reverse Convolution for Feature Upsampling

- 将特征上采样看作带权重的反问题,自适应调节重建强度。
- 在分割、深度估计等任务上显著提升稠密特征质量。
- 计算高效可微,适合嵌入现有模型快速部署。
预训练视觉基础模型(VFMs)提供强大的语义表征,但其局部特征粒度粗糙,限制了在需要精细定位、密集预测和点对应任务中的表现。本文从逆问题视角重新审视特征上采样,提出加权逆卷积(WRC),一种空间自适应的逆算子,用于增强高层视觉描述符的密度。具体地,将特征上采样建模为加权Tikhonov正则化最小二乘问题,空间可变权重同时调节数据保真度与先验强度。这使WRC能根据空间位置调整重建策略,有效保留关键结构并抑制过度平滑。此外,WRC保持高效的闭式傅里叶变换解,具备完全可微性,是实用的即插即用上采样算子。集成至轻量级自监督稠密化框架后,WRC在多种下游任务中持续提升稠密特征质量,包括分割、深度估计、视频对象分割、目标发现和关键点对应,且计算效率高。
原文摘要 · Abstract (English)
Pre-trained vision foundation models (VFMs) provide strong semantic representations, yet their patch-level features are inherently coarse, limiting their effectiveness on tasks requiring fine-grained localization, dense prediction, and point-wise correspondence. In this work, we revisit feature upsampling for VFMs from the perspective of \textbf{\textit{inverse problem}} and propose Weighted Reverse Convolution (WRC), a spatially adaptive inverse operator for densifying high-level visual descriptors. Specifically, we formulate feature upsampling as a weighted Tikhonov-regularized least-squares problem, where spatially varying weights modulate both data fidelity and prior strength at each spatial location. This allows WRC to adapt the reconstruction to spatially varying feature characteristics, thereby preserving critical structures while mitigating over-smoothing. Moreover, WRC retains an efficient, fully differentiable closed-form FFT solution, making it a practical drop-in upsampling operator. Integrated into a lightweight self-supervised densification framework, WRC consistently improves dense feature quality across various downstream benchmarks, including segmentation, depth estimation, video object segmentation, object discovery, and keypoint correspondence, while maintaining high computational efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。