arXiv:2603.03135cs.LG2026-03

将深度学习嵌入改为环面拓扑,提升低功耗设备部署效率。

Torus embeddings

  • 用归一化方法实现环面拓扑的嵌入表示。
  • 训练稳定且性能接近标准球面归一化。
  • 适合追求极致高效嵌入部署的TinyML场景。

多数数据表示为连续值向量,深度学习嵌入通常位于欧几里得空间或超球面。这些表示可量化为整数以支持大规模高效使用。然而,现有计算机最基础且高效的数值表示是带溢出的整数,其向量对应环面(超环面)拓扑,而非前述两种空间。这种不匹配导致表示容量浪费。本文表明,仅通过简单修改主流深度学习框架,即可实现具有天然环面拓扑的嵌入表示。我们比较了两种策略,发现基于归一化的方案在训练中表现出良好稳定性与性能,与标准超球面L2归一化相当。同时,环面嵌入保持了优良的量化特性。虽然环面嵌入总体性能不优于超球面嵌入,但表现相当,并为深度嵌入提供了极简路径以实现高效‘TinyML’部署。

原文摘要 · Abstract (English)

Many data representations are vectors of continuous values. In particular, deep learning embeddings are data-driven representations, typically either unconstrained in Euclidean space, or constrained to a hypersphere. These may also be translated into integer representations (quantised) for efficient large-scale use. However, the fundamental (and most efficient) numeric representation in the overwhelming majority of existing computers is integers with overflow -- and vectors of these integers do not correspond to either of these spaces, but instead to the topology of a (hyper)torus. This mismatch can lead to wasted representation capacity. Here we show that common deep learning frameworks can be adapted, quite simply, to create representations with inherent toroidal topology. We investigate two alternative strategies, demonstrating that a normalisation-based strategy leads to training with desirable stability and performance properties, comparable to a standard hyperspherical L2 normalisation. We also demonstrate that a torus embedding maintains desirable quantisation properties. The torus embedding does not outperform hypersphere embeddings in general, but is comparable, and opens the possibility to train deep embeddings which have an extremely simple pathway to efficient `TinyML' embedded implementation.

嵌入环面TinyML量化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。