用最优传输解决向量量化训练不稳问题，提升模型效率与重建质量。
Preventing Local Pitfalls in Vector Quantization via Optimal Transport
- 改用最优传输替代最近邻搜索，实现更全局的向量分配
- 在图像重建任务中达到100%码本利用率，性能超越现有先进方法
- 适合关注向量量化稳定性与训练效率的研究者
向量量化网络（VQNs）在多个任务中表现出色，但训练过程易受不稳定影响，常需精细初始化和模型蒸馏等技巧。本文指出局部极小值是导致不稳定的主因。为此，我们引入最优传输方法替代传统的最近邻搜索，实现更全局的向量分配。提出OptVQ，采用Sinkhorn算法求解最优传输问题，显著提升训练稳定性和效率。为缓解不同数据分布对Sinkhorn算法的影响，设计了一种简单有效的归一化策略。在图像重建任务上的大量实验表明，OptVQ实现了100%的码本利用率，并在重建质量上优于当前最先进的VQN方法。
原文摘要 · Abstract (English)
Vector-quantized networks (VQNs) have exhibited remarkable performance across various tasks, yet they are prone to training instability, which complicates the training process due to the necessity for techniques such as subtle initialization and model distillation. In this study, we identify the local minima issue as the primary cause of this instability. To address this, we integrate an optimal transport method in place of the nearest neighbor search to achieve a more globally informed assignment. We introduce OptVQ, a novel vector quantization method that employs the Sinkhorn algorithm to optimize the optimal transport problem, thereby enhancing the stability and efficiency of the training process. To mitigate the influence of diverse data distributions on the Sinkhorn algorithm, we implement a straightforward yet effective normalization strategy. Our comprehensive experiments on image reconstruction tasks demonstrate that OptVQ achieves 100% codebook utilization and surpasses current state-of-the-art VQNs in reconstruction quality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。