将句子嵌入约束在球面、环面等流形上,提升语义区分与结构表达能力。
Manifold-Constrained Sentence Embeddings via Triplet Loss: Projecting Semantics onto Spheres, Tori, and Möbius Strips
- 用三元组损失将嵌入投影到球面、环面和莫比乌斯带等流形空间。
- 在AG News和MBTI数据集上,聚类轮廓系数和分类准确率显著优于传统方法。
- 适合关注几何结构与语义关系建模的NLP研究者,尤其对嵌入空间设计感兴趣者。
表示学习的最新进展强调了嵌入几何对捕捉语义结构的重要性。传统的句子嵌入通常位于无约束的欧氏空间中,可能限制其对语言复杂关系的表达能力。本文提出一种新框架,通过三元组损失将句子嵌入约束在连续流形上——具体为单位球面、环面和莫比乌斯带。通过在输出空间施加微分几何约束,该方法促使嵌入同时具备判别性与拓扑结构性。我们在基准数据集AG News和MBTI上评估该方法,并与TF-IDF、Word2Vec及无约束的Keras嵌入基线进行比较。结果表明,特别是投影到球面和莫比乌斯带的嵌入,在聚类质量(轮廓系数)和分类性能(准确率)上均显著优于传统方法。这些发现凸显了在流形空间中进行嵌入的价值——拓扑结构与语义分离相辅相成,为NLP中的几何表示学习提供了新的数学基础方向。
原文摘要 · Abstract (English)
Recent advances in representation learning have emphasized the role of embedding geometry in capturing semantic structure. Traditional sentence embeddings typically reside in unconstrained Euclidean spaces, which may limit their ability to reflect complex relationships in language. In this work, we introduce a novel framework that constrains sentence embeddings to lie on continuous manifolds -- specifically the unit sphere, torus, and Möbius strip -- using triplet loss as the core training objective. By enforcing differential geometric constraints on the output space, our approach encourages the learning of embeddings that are both discriminative and topologically structured. We evaluate our method on benchmark datasets (AG News and MBTI) and compare it to classical baselines including TF-IDF, Word2Vec, and unconstrained Keras-derived embeddings. Our results demonstrate that manifold-constrained embeddings, particularly those projected onto spheres and Möbius strips, significantly outperform traditional approaches in both clustering quality (Silhouette Score) and classification performance (Accuracy). These findings highlight the value of embedding in manifold space -- where topological structure complements semantic separation -- offering a new and mathematically grounded direction for geometric representation learning in NLP.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。