wav2tok 2.0实现高效语音检索的对齐感知分词，兼顾性能与可扩展性。
wav2tok 2.0: Scalable Audio Tokenization Maintaining Explicit Pairwise Token Alignment for Efficient Audio Retrieval
- 分阶段训练：先用对比学习提取说话人无关表征，再通过CTC和DTW优化逐帧对齐
- 在QbE-STD任务上超越BEST-STD和通用分词器，保持高效率与可扩展性
- 适合需要精准语音检索的工业级应用，如语音搜索与内容匹配
学习能保持变长语句间相似性的离散语音表示，是查询式语音术语检测（QbE-STD）的核心。尽管wav2tok引入基于CTC的序列对齐以保证分词一致性，但其聚类与对齐训练紧密耦合，限制了可扩展性。我们提出wav2tok 2.0，一种基于BEST-STD主干的可扩展对齐感知语音分词器。wav2tok 2.0采用分阶段训练：首先通过对比学习和向量量化学习判别性、说话人无关表示；随后利用CTC对齐损失和一种带自适应加权的DTW对齐帧级预测目标，强制实现成对分词一致性。实验表明，wav2tok 2.0在QbE-STD任务上持续优于BEST-STD和通用分词器，同时保持高效与可扩展。
原文摘要 · Abstract (English)
Learning discrete speech representations that preserve similarity across variable-length utterances is central to query-by-example spoken term detection (QbE-STD). While wav2tok introduced CTC-based sequence alignment to enforce token consistency, its tightly coupled clustering and alignment training recipe limits scalability. We propose wav2tok 2.0, a scalable alignment-aware speech tokenizer built on the BEST-STD backbone. wav2tok 2.0 employs staged training, first learning discriminative, speaker-invariant representations via contrastive learning and vector quantization, and then enforcing pairwise token consistency using a CTC alignment loss and a novel DTW-aligned framewise prediction objective with adaptive weighting. Experiments show that wav2tok 2.0 consistently outperforms BEST-STD and general-purpose tokenizers on QbE-STD while remaining efficient and scalable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。