arXiv:2603.01757cs.CV2026-03

通过结构纹理联合判断,实现视觉自回归模型高效剪枝。

StepVAR: Structure-Texture Guided Pruning for Visual Autoregressive Models

  • 用高通滤波和PCA分别捕捉纹理与结构信息
  • 在高分辨率下实现推理速度提升且保持生成质量
  • 无需训练,适用于多种自回归图像视频模型

基于下一尺度预测的视觉自回归(VAR)模型可实现高效的分层生成,但高分辨率下推理成本呈二次增长。我们观察到,计算开销大的后期尺度主要细化高频纹理,且存在显著空间冗余,而早期尺度决定全局结构布局。现有剪枝方法多聚焦高频特征提取进行标记选择,常忽略结构一致性,导致全局语义退化。为此,我们提出StepVAR——一种无需训练的令牌剪枝框架,通过联合考虑结构与纹理重要性加速VAR推理。具体地,采用轻量级高通滤波器捕获局部纹理细节,同时利用主成分分析(PCA)保留全局结构信息。该双准则设计使模型能保留对细节保真度和整体构图均关键的令牌。为在稀疏令牌下维持有效的下一尺度预测,进一步引入最近邻特征传播策略,从剪枝表示重建密集特征图。在当前最先进的文生图与文生视频VAR模型上的大量实验表明,StepVAR在保持生成质量的同时实现显著推理提速。定量与定性评估一致显示,本方法优于现有加速方案,验证了其有效性与跨多种VAR架构的普适性。

原文摘要 · Abstract (English)

Visual AutoRegressive (VAR) models based on next-scale prediction enable efficient hierarchical generation, yet the inference cost grows quadratically at high resolutions. We observe that the computationally intensive later scales predominantly refine high-frequency textures and exhibit substantial spatial redundancy, in contrast to earlier scales that determine the global structural layout. Existing pruning methods primarily focus on high-frequency detection for token selection, often overlooking structural coherence and consequently degrading global semantics. To address this limitation, we propose StepVAR, a training-free token pruning framework that accelerates VAR inference by jointly considering structural and textural importance. Specifically, we employ a lightweight high-pass filter to capture local texture details, while leveraging Principal Component Analysis (PCA) to preserve global structural information. This dual-criterion design enables the model to retain tokens critical for both fine-grained fidelity and overall composition. To maintain valid next-scale prediction under sparse tokens, we further introduce a nearest neighbor feature propagation strategy to reconstruct dense feature maps from pruned representations. Extensive experiments on state-of-the-art text-to-image and text-to-video VAR models demonstrate that StepVAR achieves substantial inference speedups while maintaining generation quality. Quantitative and qualitative evaluations consistently show that our method outperforms existing acceleration approaches, validating its effectiveness and general applicability across diverse VAR architectures.

自回归模型剪枝图像生成特征传播

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。