MERGE动态构建层级索引，解决推荐系统中物品分布不均问题。
MERGE: Next-Generation Item Indexing Paradigm for Large-Scale Streaming Recommendation
- 从零自适应建簇，动态监控占用率，分层合并形成细粒度索引
- 相比现有方法，分配准确率、簇均匀性与分离度显著提升
- 适合大规模实时推荐场景，尤其应对数据分布剧烈变化的工业系统
物品索引将大量物品映射为紧凑离散表示，对判别式与生成式推荐系统至关重要。然而，现有基于向量量化的方法在工业级流式推荐中面临高度偏斜且非平稳的物品分布，导致分配准确率低、簇占用不均、簇间分离不足。为此，我们提出MERGE，一种新一代物品索引范式：自适应从零构建簇，动态监测簇占用情况，并通过由细到粗的合并形成层次化索引结构。大量实验表明，相比现有方法，MERGE显著提升了分配准确率、簇均匀性与簇分离度；线上A/B测试也验证了其在关键业务指标上的显著提升，凸显其作为大规模推荐基础索引方案的潜力。代码已开源：https://github.com/baiyimeng/MERGE。
原文摘要 · Abstract (English)
Item indexing, which maps a large corpus of items into compact discrete representations, is critical for both discriminative and generative recommender systems, yet existing Vector Quantization (VQ)-based approaches struggle with the highly skewed and non-stationary item distributions common in industrial streaming recommenders, leading to poor assignment accuracy, imbalanced cluster occupancy, and insufficient cluster separation. To address these challenges, we propose MERGE, a next-generation item indexing paradigm that adaptively constructs clusters from scratch, dynamically monitors cluster occupancy, and forms hierarchical index structures via fine-to-coarse merging. Extensive experiments demonstrate that MERGE significantly improves assignment accuracy, cluster uniformity, and cluster separation compared with existing indexing methods, while online A/B tests show substantial gains in key business metrics, highlighting its potential as a foundational indexing approach for large-scale recommendation. Codes are available at https://github.com/baiyimeng/MERGE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。