根据图像复杂度动态调整分辨率和模型宽度，节省计算量。
ProgResViT: Progressive Resolution and Width for Adaptive Vision Transformers
- 逐轮提升分辨率与模型宽度，低复杂度图像提前终止推理。
- 在DeiT上实现84.9%准确率，计算量更低，优于现有自适应方法。
- 适用于分类、自监督表征与语义分割，适合资源受限场景。
视觉变压器（ViT）通常对所有图像使用固定输入分辨率和模型宽度，尽管许多图像可用更少计算量完成分类。我们提出ProgResViT，一种输入自适应的视觉变压器，通过多轮逐步推理实现高效推断：首轮以低分辨率和窄子网络处理图像，当预测置信度足够时终止；否则复用当前表示，升级为更高分辨率和更宽子网络继续优化。所有轮次共享同一主干，提出进度条件软门控（PSG），根据当前轮次、模块和输入分辨率调节令牌融合与层输出。在图像分类任务中，基于DeiT的ProgResViT在精度-计算权衡上优于自适应宽度、深度及动态令牌基线。结合知识蒸馏后，其达到84.9%的Top-1准确率，略高于相同设置下报告的DeiT-III-S表现。该设计同样在自监督DINO表征与下游语义分割任务中展现良好精度-计算平衡。代码已开源。
原文摘要 · Abstract (English)
Vision Transformers (ViTs) typically process every image using a fixed input resolution and model width, even though many images can be classified with substantially less computation. We introduce ProgResViT, an input-adaptive ViT that performs inference progressively across multiple rounds. The first round processes a low-resolution image with a narrow subnetwork. Inference terminates when the prediction is sufficiently confident; otherwise, the model reuses the representations produced in the current round and proceeds with a higher-resolution input and a wider subnetwork to refine its prediction. As all rounds share a single backbone, we propose Progress-Conditioned Soft Gating (PSG), which conditions token fusion and layer outputs on the current round, block, and input resolution. On image classification, applying ProgResViT to DeiT yields better accuracy-compute trade-offs than adaptive-width, adaptive-depth, and dynamic-token baselines. With knowledge distillation, a DeiT-based ProgResViT achieves 84.9% top-1 accuracy, slightly exceeding the reported DeiT-III-S accuracy under a comparable evaluation setting. We show that the same design also provides favorable accuracy-compute trade-offs for self-supervised DINO representations and downstream semantic segmentation. Code is available at https://github.com/ds-kiel/ProgResViT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。