用子空间代替向量，更好表达概念层级与组合关系。
Native Hierarchical and Compositional Representations with Subspace Embeddings
- 将概念表示为线性子空间，通过维度和包含关系建模层次结构。
- 在层级推理和自然语言蕴含任务上达到顶尖性能，优于传统向量方法。
- 无需额外标注即可自然处理逻辑否定，适合需要语义组合的场景。
传统嵌入将数据点表示为向量，便于计算相似度，但难以捕捉层级结构与组合性。本文提出新范式：将概念表示为线性子空间。通过占据多个维度，子空间可建模更广义的概念，并在其内部嵌套更具体的概念。这种几何结构天然通过维度体现泛化性、通过包含关系体现层级性，并可通过线性代数运算实现组合性。为使该范式可训练，我们引入基于软投影矩阵的可微子空间参数化方法，使每个子空间的有效维度可学习。实验表明，该方法在层级推理与自然语言蕴含基准测试中达到最优性能，同时提供几何可解释的蕴含模型。进一步证明，当标准向量嵌入在否定查询上退化至近随机水平时，子空间嵌入能原生支持逻辑组合，且保持与高效欧氏向量搜索的兼容性。
原文摘要 · Abstract (English)
Traditional embeddings represent datapoints as vectors, which makes similarity easy to compute but limits how well they capture hierarchies and compositionality. We propose a fundamentally different approach: representing concepts as linear subspaces. By spanning multiple dimensions, subspaces can model broader concepts with higher-dimensional regions and nest more specific concepts within them. This geometry naturally captures generality through dimension, hierarchy through inclusion, and enables an emergent structure for composition via linear algebraic operations. To make this paradigm trainable, we introduce a differentiable subspace parameterization via soft projection matrices, allowing the effective dimension of each subspace to be learned. Our method not only achieves state-of-the-art performance on hierarchical and natural language inference benchmarks but also provides a geometrically-grounded model of entailment. Further, we demonstrate that while standard vector embeddings degrade to near-random performance on negated queries, subspace embeddings natively capture logical composition without explicit supervision, while preserving compatibility with efficient Euclidean vector search.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。