arXiv:2506.09454cs.LG2025-06

提出新型损失函数,让排序模型更快收敛且性能不降。

NDCG-Consistent Softmax Approximation with Accelerated Convergence

  • 用泰勒展开重构Softmax损失,设计出更高效的排名损失
  • 在真实数据集上收敛速度显著提升,排名效果接近甚至超越原方法
  • 适合需要高效排序的工业级相似性学习任务

排序任务是极端相似性学习框架的核心,其中大规模对象通过预定义的序关系建模。尽管Softmax(SM)损失因能处理全局负样本比较而广泛应用,但在大规模场景下存在计算开销大、可扩展性差的问题。本文提出两种与排序指标一致的新损失:基于泰勒展开的平方型RG²损失和交互型RG×损失,揭示了加权平方损失的内在机制,并建立采样与非采样损失范式的联系。进一步将新损失与高效的交替最小二乘(ALS)优化结合,提供泛化保证与收敛速率分析。实验证明,该方法在真实数据集上达到与SM损失相当或更优的排序性能,同时显著加速收敛。本框架为相似性学习提供了理论洞见与实用工具,适用于需平衡排序质量与计算效率的各类任务。

原文摘要 · Abstract (English)

Ranking tasks constitute fundamental components of extreme similarity learning frameworks, where extremely large corpora of objects are modeled through relative similarity relationships adhering to predefined ordinal structures. Among various ranking surrogates, Softmax (SM) Loss has been widely adopted due to its natural capability to handle listwise ranking via global negative comparisons, along with its flexibility across diverse application scenarios. However, despite its effectiveness, SM Loss often suffers from significant computational overhead and scalability limitations when applied to large-scale object spaces. To address this challenge, we propose novel loss formulations that align directly with ranking metrics: the Ranking-Generalizable \textbf{squared} (RG$^2$) Loss and the Ranking-Generalizable interactive (RG$^\times$) Loss, both derived through Taylor expansions of the SM Loss. Notably, RG$^2$ reveals the intrinsic mechanisms underlying weighted squared losses (WSL) in ranking methods and uncovers fundamental connections between sampling-based and non-sampling-based loss paradigms. Furthermore, we integrate the proposed RG losses with the highly efficient Alternating Least Squares (ALS) optimization method, providing both generalization guarantees and convergence rate analyses. Empirical evaluations on real-world datasets demonstrate that our approach achieves comparable or superior ranking performance relative to SM Loss, while significantly accelerating convergence. This framework offers the similarity learning community both theoretical insights and practically efficient tools, with methodologies applicable to a broad range of tasks where balancing ranking quality and computational efficiency is essential.

排序学习损失函数高效优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。