用扩散模型生成文档相关性,提升排序效果
From Noise to Order: Learning to Rank via Denoising Diffusion
- 基于去噪扩散模型建模查询-文档联合分布
- 在4个标准数据集上优于传统判别式排序方法
- 适合关注生成式排序与扩散模型的科研人员
在信息检索中,传统学习排序(LTR)方法多采用判别式机器学习模型,仅建模给定查询-文档特征表示下文档相关的概率。本文提出一种基于去噪扩散的深度生成式方法——DiffusionRank,通过建模特征向量与相关性标签的完整联合分布来替代传统方法。我们扩展了现有用于表格数据的去噪扩散模型TabDiff,构建了经典点对式和成对式排序目标的生成式等价形式。在四个标准LTR数据集上的充分实验表明,DiffusionRank模型相比其判别式对应方法具有明显性能提升。本工作揭示了将扩散等深度生成模型进展应用于排序任务的巨大潜力,代码已开源。
原文摘要 · Abstract (English)
In information retrieval (IR), learning-to-rank (LTR) methods have traditionally limited themselves to discriminative machine learning approaches that model the probability of the document being relevant to the query given some feature representation of the query-document pair. In this work, we propose an alternative denoising diffusion-based deep generative approach to LTR that instead models the full joint distribution over feature vectors and relevance labels. While in the discriminative setting, an over-parameterized ranking model may find different ways to fit the training data, we hypothesize that candidate solutions that can explain the full data distribution under the generative setting are better equipped to estimate relevance. With this motivation, we propose DiffusionRank that extends TabDiff, an existing denoising diffusion-based generative model for tabular datasets, to create generative equivalents of classical discriminative pointwise and pairwise LTR objectives. We conduct thorough empirical evaluation on four standard LTR datasets to demonstrate improvements from DiffusionRank models over their discriminative counterparts. Our work points to a rich space for future research exploration on how we can leverage ongoing advancements in deep generative modeling approaches, such as diffusion, for LTR. We made our code publicly available at https://github.com/sadjadeb/DiffusionRank.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。