用类令牌控制生成，让注意力头自动剪枝，提升ViT自编码器效率与可控性。
ViTCAE: ViT-based Class-conditioned Autoencoder
- 将类令牌重用于生成全局潜在变量，指导局部细节合成。
- 基于共识动态理论设计温度调度，实现注意力头的自适应剪枝。
- 无需人工干预，显著提速训练，适合追求高效生成的开发者。
基于视觉变换器（ViT）的自编码器常未能充分利用全局类令牌，且采用静态注意力机制，限制了生成控制力和优化效率。本文提出ViTCAE框架，将类令牌重定位为生成核心。编码器将类令牌映射为全局潜在变量，决定局部补丁级潜在变量的先验分布，建立全局语义直接影响局部细节生成的强依赖关系。受意见动态启发，将每个注意力头视为寻求共识的动态系统。该视角催生一种收敛感知的温度调度策略，根据各头分布稳定性自适应调整其影响函数。此过程实现了基于理论诊断（如注意力演化距离、共识/聚类函数）的头冻结机制，训练中剪除已收敛的注意力头，大幅提高计算效率且不损失生成保真度。通过统一生成式类令牌与基于多智能体共识理论的自适应注意力机制，ViTCAE提供了一种更高效、更可控的Transformer生成新范式。
原文摘要 · Abstract (English)
Vision Transformer (ViT) based autoencoders often underutilize the global Class token and employ static attention mechanisms, limiting both generative control and optimization efficiency. This paper introduces ViTCAE, a framework that addresses these issues by re-purposing the Class token into a generative linchpin. In our architecture, the encoder maps the Class token to a global latent variable that dictates the prior distribution for local, patch-level latent variables, establishing a robust dependency where global semantics directly inform the synthesis of local details. Drawing inspiration from opinion dynamics, we treat each attention head as a dynamical system of interacting tokens seeking consensus. This perspective motivates a convergence-aware temperature scheduler that adaptively anneals each head's influence function based on its distributional stability. This process enables a principled head-freezing mechanism, guided by theoretically-grounded diagnostics like an attention evolution distance and a consensus/cluster functional. This technique prunes converged heads during training to significantly improve computational efficiency without sacrificing fidelity. By unifying a generative Class token with an adaptive attention mechanism rooted in multi-agent consensus theory, ViTCAE offers a more efficient and controllable approach to transformer-based generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。