无需重训即可替换视觉量化模块，大幅降低训练成本。
VQ-Transplant: Efficient VQ-Module Integration for Pre-trained Visual Tokenizers
- 将新量化模块插入预训练编码器-解码器，实现即插即用。
- 在VAR模型上实现接近顶尖的重建质量，训练成本降低95%。
- 适合资源有限的研究者快速验证新型量化方法。
向量量化（VQ）是现代离散视觉标记化的核心。然而，训练最先进的基于VQ的模型需要大量计算资源，这在实际中几乎阻碍了资源受限条件下新型先进VQ技术的发展。为此，我们提出VQ-Transplant，一种简单框架，可通过替换原生VQ模块，将新VQ模块无缝集成到冻结的预训练标记器中。关键在于，该移植过程保留所有编码器-解码器参数，避免了修改量化方法时昂贵的端到端重训练。为缓解解码器与量化空间之间的不匹配问题，我们引入轻量级解码器适应策略（仅在ImageNet-1k上训练5个周期），以对齐特征先验与新量化空间。在实验评估中，VQ-Transplant使行业级模型VAR获得接近顶尖的重建保真度，同时将训练成本降低95%。该方法推动了量化研究的普惠化，使资源高效地集成新型量化技术成为可能。
原文摘要 · Abstract (English)
Vector Quantization (VQ) underpins modern discrete visual tokenization. However, training quantization modules for state-of-the-art VQ-based models requires significant computational resources which, in practice, all but prevents the development of novel, cutting-edge VQ techniques under resource constraints. To address this limitation, we propose {\bf VQ-Transplant}, a simple framework that enables plug-and-play integration of new VQ modules into frozen, pre-trained tokenizers by replacing their native VQ modules. Crucially, the proposed transplantation process preserves all encoder-decoder parameters, obviating the need for costly end-to-end retraining when modifying the quantization method. To mitigate decoder-quantization mismatch, we introduce a lightweight decoder adaptation strategy (trained for only 5 epochs on ImageNet-1k) to align feature priors with the new quantization space. In our empirical evaluation, we find that VQ-Transplant allows obtaining near state-of-the-art reconstruction fidelity for industry-level models like VAR while reducing the training cost by 95\%. VQ-Transplant democratizes quantization research by enabling resource-efficient integration of novel VQ techniques while matching industry-level reconstruction performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。