提出任意维度模型的通用性理论,解决模型对变尺寸输入的泛化难题。
Any-Dimensional Invariant Universality
- 将变尺寸输入映射到无限维极限空间,统一建模不同规模输入
- 证明在特定拓扑下,任意维度模型可实现通用逼近
- 揭示现有架构缺陷并给出简单修复方案,适合图/点云等场景
许多机器学习模型可处理任意尺寸输入,如节点数不同的图或点数不同的点云。然而,这类任意维度模型的通用性理论尚不清晰,因传统通用性研究针对固定尺寸输入,定义于其定义域的紧子集上。与之相反,任意维度模型可视为一系列定义在不断增长输入上的函数序列,其通用性在何种意义上成立仍不明确。本文提出系统方法建立任意维度通用性:将任意维度函数与一个唯一函数关联,该函数定义在包含所有有限尺寸输入及其极限的合适无限维极限空间上。利用输入的对称性及不同尺寸输入间的关联关系,我们证明该极限空间具有自然拓扑,并存在丰富的紧集,可在其上建立任意维度通用性。通过实例说明若干现有架构无法实现通用性,并提出简单修改使其恢复通用性。
原文摘要 · Abstract (English)
Several machine learning models are defined for inputs of any size, such as graphs with different numbers of nodes and point clouds containing varying numbers of points. The universality properties of such any-dimensional models remain poorly understood, as universality is traditionally studied for models accepting inputs of a fixed size, defined on a compact subset of their domain. In sharp contrast, any-dimensional models can be viewed as sequences of functions defined on growing-sized inputs, and it is not clear in which sense they can be universal. We develop a systematic approach to establish any-dimensional universality, by identifying any-dimensional functions with a unique function taking inputs in a suitable infinite-dimensional limit space containing inputs of all finite sizes as well as their limits. Using the symmetries of these inputs and relations between inputs of different sizes, we show that this limit space admits a natural topology with rich families of compact sets on which any-dimensional universality can be established. We illustrate our approach by showing that several existing architectures fail to be universal, and we propose simple modifications that restore universality.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。