arXiv:2605.01073cs.CL2026-05

研究句子嵌入空间中近义句的局部几何结构,提出非线性建模方法。

Controlled Paraphrase Geometry in Sentence Embedding Space: Local Manifold Modeling and Latent Probing

论文配图:Controlled Paraphrase Geometry in Sentence Embedding Space: Local Manifold Modeling and Latent Probing
图 1 · 摘自论文原文
  • 用仿射、二次、三次模型拟合近义句的局部几何结构
  • 非线性模型比线性模型更准确描述嵌入云分布
  • 适用于需要理解嵌入空间几何性质的研究者

本文研究由受控的语义相近句子集合所诱导的嵌入云的局部几何结构。核心问题是:受控的改写式语义变化在句子嵌入空间中如何组织?是否可用低阶拟合曲面显式建模?我们提出了基于仿射、二次和三次拟合模型的局部几何建模方案,并采用基于曲面的潜在探查方法,在降维后的局部主成分分析(PCA)空间中构建合成潜在点。该方法用于离线表示空间分析、局部流形建模与几何感知的潜在探查。生成的潜在点通过表面一致性、邻域结构保持度、经验分布吻合度、基于海森矩阵的二阶形状描述符稳定性以及拟合系数稳定性等标准进行评估。在受控的语义相近句子集上的实验表明,非线性局部模型比仿射模型更准确地描述嵌入云。基于曲面的生成方法表现出强几何保真性,包括表面一致性、海森形状一致性与系数一致性。下游实验显示,合成点的几何有效性并不自动带来分类性能提升。结果支持对句子嵌入空间进行显式局部几何建模,并强调几何有效性与判别效用需区分。作为资源贡献,我们推出了 extbf{CoPaGE-300K},一个基于模板的受控语义相近句子变体数据集,包含槽级标注与预计算的句子嵌入。

原文摘要 · Abstract (English)

The paper studies the local geometry of embedding clouds induced by \emph{controlled local classes of semantically close sentences}. The central question is how controlled paraphrase-like semantic variation is organized in sentence embedding space and whether this local structure can be explicitly modeled by low-degree fitted carriers. We introduce a local geometric modeling scheme based on affine, quadratic, and cubic fitted models. We also use a surface-based latent probing procedure that constructs synthetic latent points in a reduced local PCA space with respect to the fitted carrier. The procedure is intended as an offline method for representation-space analysis, local manifold modeling, and geometry-aware latent probing. Generated latent points are evaluated using criteria that measure consistency with the fitted surface, preservation of neighborhood structure, agreement with the empirical distribution, stability of Hessian-based second-order shape descriptors, and stability of fitted-model coefficients. Experiments on controlled sets of semantically close sentences show that nonlinear local models describe embedding clouds more accurately than affine models. Surface-based generation provides strong fitted-geometry fidelity, including surface consistency, Hessian-based shape consistency, and coefficient consistency. Downstream experiments show that geometric validity of synthetic latent points does not automatically translate into improved classification performance. The results support explicit local geometric modeling of sentence embedding space and highlight the need to distinguish geometric validity from discriminative utility. As a resource contribution, we introduce \textbf{CoPaGE-300K}, a controlled template-based dataset of semantically close sentence variants with slot-level annotations and precomputed sentence embeddings.

句子嵌入局部几何流形建模数据集

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。