arXiv:2605.17779cs.LG2026-05被引 1

为推荐生成设计可变长度编码,让热门商品用短码、冷门商品用长码。

Learning Variable-Length Tokenization for Generative Recommendation

论文配图:Learning Variable-Length Tokenization for Generative Recommendation
图 1 · 摘自论文原文
  • 根据物品流行度动态分配编码长度,用信息论证明最优长度与流行度负相关。
  • 在四个数据集上,新方法显著提升推荐准确率,训练和推理效率也更高。
  • 适合关注生成式推荐中编码效率与精度平衡的研究者或工程师。

生成式推荐将推荐任务重新定义为对离散语义标识符(ID)的下一个词预测。现有方法对所有物品采用固定长度编码,隐含假设其编码能力一致,但未被探索。通过在四个数据集上的系统实验,我们发现‘流行度-长度悖论’:热门物品在短ID下表现最佳,而尾部物品需要更长的编码以捕捉区分性语义。这揭示了关键错配——热门物品依赖丰富的协同信号,无需过多语义细节;尾部物品则因交互数据稀疏,必须依赖细粒度内容特征。为此,我们提出VarLenRec,一种学习可变长度编码的框架。我们设计了流行度加权信息预算分配(PIBA),从信息论角度证明最优ID长度应与流行度的负幂次成正比。直接实现可变长度面临两大挑战:标准欧几里得残差量化缺乏几何容量支持不同长度编码而不失真,且离散长度决策不可微。我们通过双曲残差量化解决前者,利用庞加莱球的指数体积增长自然分层编码容量;通过软长度控制器实现可微长度预测,其连续层保留概率受PIBA导出先验正则化。大量实验表明,VarLenRec在推荐准确率及训练/推理效率上均显著优于当前最先进方法,揭示了生成式推荐中自适应编码容量的重要性。

原文摘要 · Abstract (English)

Generative recommendation reformulates recommendation as next-token prediction over discrete semantic identifiers (IDs). A fundamental yet unexplored design choice is that existing methods employ fixed-length tokenization for all items, implicitly assuming uniform encoding capacity regardless of item characteristics. Through systematic experiments across four datasets, we discover the Popularity-Length Paradox: popular items achieve optimal performance with short IDs, while tail items require substantially longer codes to capture discriminative semantics. This reveals a critical mismatch where popular items benefit from abundant collaborative signals and require minimal semantic detail, whereas tail items must rely on fine-grained content features due to sparse interaction data. To address this, we propose VarLenRec, a framework for learning variable-length tokenization. We develop Popularity-Weighted Information Budget Allocation (PIBA), an information-theoretic framework proving that optimal ID length should scale as a negative power of popularity. Directly implementing variable-length allocation faces two technical challenges: standard Euclidean residual quantization lacks geometric capacity to support diverse code lengths without distortion, and discrete length decisions are non-differentiable. We address these through Hyperbolic Residual Quantization, which leverages the exponential volume growth of the Poincaré ball to naturally stratify encoding capacity, and a Soft Length Controller, which enables differentiable length prediction via continuous layer retention probabilities regularized by PIBA-derived priors. Extensive experiments demonstrate that VarLenRec achieves significant improvements over state-of-the-art methods in recommendation accuracy and training/inference efficiency, revealing the importance of adaptive encoding capacity in generative recommendation.

生成推荐可变编码信息论冷启动

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。