提出排序不变嵌入的定量边界,提升节点顺序无关图学习的精度与稳定性。
Quantitative Bounds for Sorting-Based Permutation-Invariant Embeddings
- 通过排序一维投影构造置换不变嵌入,实现输入点集的唯一映射。
- 证明嵌入畸变与点数平方成正比,且与维度无关,优于已有理论上限。
- 适用于图神经网络等需节点顺序不变性的场景,尤其关注高维稀疏数据建模。
我们研究了 $d$-维点集的置换不变嵌入,该嵌入通过排序 $D$ 个独立的一维投影定义,广泛应用于图深度学习中要求输出对节点排列不变的场景。先前工作表明,在足够大的 $D$ 且投影处于一般位置时,该映射是单射,并满足双李普希茨条件。然而仍有两大空白:一是最优嵌入维数 $D$ 尚未明确;二是映射的双李普希茨常数缺乏估计。本文在两方面取得实质性进展:首先,改进了保证单射性所需的 $D$ 的上界,并给出了最小单射维数的下界;其次,构造出投影矩阵,使得映射的双李普希茨畸变随点数 $n$ 的平方增长,且完全与维度 $d$ 无关。同时证明,对任意投影向量选择,畸变下限始终不低于 $\sqrt{n}$ 的量级。最后,我们还证明,即使对映射施加线性投影以降维,仍可保持类似理论保证。
原文摘要 · Abstract (English)
We study permutation-invariant embeddings of $d$-dimensional point sets, which are defined by sorting $D$ independent one-dimensional projections of the input. Such embeddings arise in graph deep learning where outputs should be invariant to permutations of graph nodes. Previous work showed that for large enough $D$ and projections in general position, this mapping is injective, and moreover satisfies a bi-Lipschitz condition. However, two gaps remain: firstly, the optimal size $D$ required for injectivity is not yet known, and secondly, no estimates of the bi-Lipschitz constants of the mapping are known. In this paper, we make substantial progress in addressing both of these gaps. Regarding the first gap, we improve upon the best known upper bounds for the embedding dimension $D$ necessary for injectivity, and also provide a lower bound on the minimal injectivity dimension. Regarding the second gap, we construct matrices of projection vectors, so that the bi-Lipschitz distortion of the mapping depends quadratically on the number of points $n$, and is completely independent of the dimension $d$. We also show that for any choice of projection vectors, the distortion of the mapping will never be better than a bound proportional to the square root of $n$. Finally, we show that similar guarantees can be provided even when linear projections are applied to the mapping to reduce its dimension.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。