arXiv:2503.13051cs.LGcs.CV2025-03被引 1

用仅N个参数实现高效可微排序,突破传统方法内存瓶颈

Permutation Learning with Only N Parameters: From SoftSort to Self-Organizing Gaussians

  • 通过迭代打乱索引并结合少量SoftSort步骤,实现低参量可微排序
  • 在多维数据上排序质量显著优于纯SoftSort,且内存占用仅为原方法的1%
  • 适合大规模优化任务,如自组织高斯模型等需要高效排列学习的场景

排序与排列学习在优化和机器学习中至关重要,尤其用于将高维数据组织为有意义的空间布局。Gumbel-Sinkhorn方法虽有效,但需N×N参数表示完整排列矩阵,计算开销大。低秩矩阵分解可降至2NM参数(M<<N),但仍难应对超大规模问题。SoftSort提供了一维可微排序的连续松弛,但在多维数据和复杂排列上表现不足。本文提出一种仅需N个参数的新型排列学习方法,显著降低存储成本。该方法扩展SoftSort,通过迭代打乱元素索引并每轮执行少量SoftSort优化步骤,显著提升多维数据排序质量与复杂目标下的性能,优于纯SoftSort。相比现有方法,本方法具备更高内存效率与可扩展性,特别适用于自组织高斯模型等对高效、可扩展排列学习至关重要的大规模优化任务。

原文摘要 · Abstract (English)

Sorting and permutation learning are key concepts in optimization and machine learning, especially when organizing high-dimensional data into meaningful spatial layouts. The Gumbel-Sinkhorn method, while effective, requires N*N parameters to determine a full permutation matrix, making it computationally expensive for large datasets. Low-rank matrix factorization approximations reduce memory requirements to 2NM (with M << N), but they still struggle with very large problems. SoftSort, by providing a continuous relaxation of the argsort operator, allows differentiable 1D sorting, but it faces challenges with multidimensional data and complex permutations. In this paper, we present a novel method for learning permutations using only N parameters, which dramatically reduces storage costs. Our method extends SoftSort by iteratively shuffling the N indices of the elements and applying a few SoftSort optimization steps per iteration. This modification significantly improves sorting quality, especially for multidimensional data and complex optimization criteria, and outperforms pure SoftSort. Our method offers improved memory efficiency and scalability compared to existing approaches, while maintaining high-quality permutation learning. Its dramatically reduced memory requirements make it particularly well-suited for large-scale optimization tasks, such as "Self-Organizing Gaussians", where efficient and scalable permutation learning is critical.

可微排序低参数排列学习自组织

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。