arXiv:2601.17950cs.CV2026-01中稿 · CVPR被引 2

提出轻量级像素密集特征上采样方法,效率更高且性能领先。

UPLiFT: Efficient Pixel-Dense Feature Upsampling with Local Attenders

  • 采用局部注意力机制替代全局交叉注意力,提升上采样效率。
  • 在保持稳定特征的同时,实现比现有方法更低的推理开销。
  • 适用于生成式任务,可替代先进变分自编码器上采样模型。

无任务特异性特征上采样已成为高效生成更稠密特征的前沿方向,通过学习将低分辨率特征映射到高分辨率版本,仅需极少成本即可获得密集特征。尽管早期方法依赖迭代上采样,近期工作转向基于交叉注意力的方法,但面临与骨干网络相同的效率瓶颈。本文表明,迭代方法仍具竞争力,且可通过新架构取得最佳性能并降低推理成本。我们提出UPLiFT——通用像素密集轻量特征变换框架,并引入高效的局部注意力算子(Local Attender),该算子采用全局部注意力池化形式,克服了传统迭代方法的局限性。实验显示,局部注意力使UPLiFT在上采样过程中保持特征稳定性,实现最优性能且推理成本低于现有像素密集上采样器。此外,将UPLiFT应用于生成式下游任务,在变分自编码器特征上采样中表现媲美最先进的耦合流匹配模型。总体而言,UPLiFT提供了一种灵活高效的稠密特征生成方案。

原文摘要 · Abstract (English)

The space of task-agnostic feature upsampling has emerged as a promising area of research to efficiently create denser features from pre-trained visual backbones. These methods act as a shortcut to achieve dense features for a fraction of the cost by learning to map low-resolution features to high-resolution versions. While early works in this space used iterative upsampling approaches, more recent works have switched to cross-attention-based methods, which risk falling into the same efficiency scaling problems of the backbones they are upsampling. In this work, we demonstrate that iterative upsampling methods can still compete with cross-attention-based methods; moreover, they can achieve state-of-the-art performance with lower inference costs. We propose UPLiFT, an architecture for Universal Pixel-dense Lightweight Feature Transforms. We also propose an efficient Local Attender operator to overcome the limitations of prior iterative feature upsampling methods. This operator uses an alternative attentional pooling formulation defined fully locally. We show that our Local Attender allows UPLiFT to maintain stable features throughout upsampling, enabling state-of-the-art performance with lower inference costs than existing pixel-dense feature upsamplers. In addition, we apply UPLiFT to generative downstream tasks and show that it achieves competitive performance with state-of-the-art Coupled Flow Matching models for VAE feature upsampling. Altogether, UPLiFT offers a versatile and efficient approach to creating denser features.

特征上采样轻量模型局部注意力

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。