提出新方法精准比较词语语义广度,避免方向差异干扰结果。
Accurate and Efficient Statistical Testing for Word Semantic Breadth

- 用霍尔黑德反射对齐词向量均值方向,分离方向与发散差异。
- 实验显示误报率降低32.5%,且在真实差异下仍保持敏感性。
- 支持GPU批量计算,速度比CPU快23倍,适合大规模分析。
随着上下文嵌入表示的兴起,衡量词语语义广度(即其在不同语境中的分布范围)已成为可能。一个词类可被表示为一组词向量构成的云点,其分散度统计量可作为上下文多样性的代理指标(Nagata and Tanaka-Ishii, ACL2025)。这些测量对构建同义词典和领域词典时判断语义区分是否恰当具有价值。然而,直接对分散度进行假设检验可能存在误导:语义方向差异可能伪装成分散度差异,导致第一类错误膨胀,在无真实广度差异时仍出现“显著”结果。这在小差异区间尤为危险,因显著性测试应能区分真实效应与偶然波动。本文提出一种霍尔黑德对齐的置换检验方法:先通过单个霍尔黑德反射对齐两词类的均值方向,再对对齐后的词向量云执行置换检验,从而获得校准的非参数p值。为提升实用性,引入面向GPU的实现,支持批量置换与线性代数运算。实验证明,该方法将第一类错误降低32.5%,同时保留对真实广度差异的检测能力,并实现相比CPU基线23倍的速度提升。
原文摘要 · Abstract (English)
Measuring the breadth of a word's meaning, or its spread across contexts, has become feasible with contextualized token embeddings. A word type can be represented as a cloud of token vectors, with dispersion-based statistics serving as proxies for contextual diversity (Nagata and Tanaka-Ishii, ACL2025). These measurements are useful for deciding appropriate sense distinctions when constructing thesauri and domain-specific dictionaries. However, when comparing the breadth of two word types, naive hypothesis testing on dispersion can be misleading: differences in semantic direction can masquerade as dispersion differences, inflating Type-I error and yielding "statistically significant" outcomes even when there is no true breadth difference. This is problematic because significance testing should distinguish genuine effects from incidental fluctuations in small-difference regimes. We propose a Householder-aligned permutation test to isolate dispersion differences from directional differences. Our method applies a single Householder reflection to align the mean directions of the two word types and then performs a permutation test on the aligned token clouds, yielding calibrated, non-parametric p-values. For practicality, we introduce a GPU-oriented implementation that batches permutations and linear algebra operations. Empirically, our alignment reduced Type-I error by 32.5% while preserving sensitivity to genuine breadth differences, and achieved a 23x speedup over the CPU baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。