动态激活权重与令牌，提升视觉自回归生成效率
ActVAR: Activating Mixtures of Weights and Tokens for Efficient Visual Autoregressive Generation
- 通过可学习路由动态选择专家子网络，按内容激活不同权重
- 仅计算高更新潜力的令牌，降低21.2%计算量且性能损失小
- 适合追求高效图像生成的开发者，尤其对资源受限场景友好
视觉自回归（VAR）模型通过逐尺度预测实现高效图像生成，但序列长度增加时计算成本急剧上升。现有静态剪枝方法会永久移除权重或令牌，破坏预训练依赖关系。为此，我们提出ActVAR，一种在模型权重和令牌序列上引入双重稀疏性的动态激活框架，以提升效率而不损失容量。ActVAR将前馈网络（FFNs）分解为轻量级专家子网络，并使用可学习路由器根据内容动态选择特定令牌的专家子集。同时，门控令牌选择器识别具有高更新潜力的令牌进行计算，重构未选令牌以保持全局上下文与序列对齐。训练采用两阶段知识蒸馏策略，由原始VAR模型监督路由与门控策略的学习，使其与预训练知识对齐。在ImageNet $256 imes 256$基准上的实验表明，ActVAR实现了最高21.2%的浮点运算量（FLOPs）减少，且性能下降极小。
原文摘要 · Abstract (English)
Visual Autoregressive (VAR) models enable efficient image generation via next-scale prediction but face escalating computational costs as sequence length grows. Existing static pruning methods degrade performance by permanently removing weights or tokens, disrupting pretrained dependencies. To address this, we propose ActVAR, a dynamic activation framework that introduces dual sparsity across model weights and token sequences to enhance efficiency without sacrificing capacity. ActVAR decomposes feedforward networks (FFNs) into lightweight expert sub-networks and employs a learnable router to dynamically select token-specific expert subsets based on content. Simultaneously, a gated token selector identifies high-update-potential tokens for computation while reconstructing unselected tokens to preserve global context and sequence alignment. Training employs a two-stage knowledge distillation strategy, where the original VAR model supervises the learning of routing and gating policies to align with pretrained knowledge. Experiments on the ImageNet $256\times 256$ benchmark demonstrate that ActVAR achieves up to $21.2\%$ FLOPs reduction with minimal performance degradation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。