arXiv:2605.07409cs.CLcs.LG2026-05ACL被引 1

用新方法把语言嵌入转化为可信的社会测量工具

The Proxy Presumption: From Semantic Embeddings to Valid Social Measures

  • 提出构造效度协议,系统验证嵌入是否真反映目标概念
  • 通过反事实中性化消除主题、风格等干扰因素影响
  • 适合做社会科学研究的学者使用,尤其关注测量可靠性

自然语言处理正成为计算社会科学的核心工具,研究者常使用词向量等嵌入表示来衡量新颖性、创造力和偏见等隐变量。然而,这一做法面临根本性挑战:即‘代理假定’——依赖几何距离(如余弦相似度)作为社会概念的直接度量。我们指出,在缺乏明确验证的情况下,无监督表示会混杂目标构念(C)与话题、风格、作者等混淆变量(Z)。为此,本文提出构造效度协议(CVP),结合因果表示学习与心理测量学,构建从概念定义到量化验证的严格流程。进一步提出基于大模型的反事实中性化方法,以减少嵌入空间中的混淆。通过提供包含区分效度、增量效度和预测效度的标准化效度套件,本工作为社区提供了将启发式代理转化为科学可辩护测量工具的完整方案。

原文摘要 · Abstract (English)

Natural Language Processing is rapidly evolving into a primary instrument for Computational Social Science, with researchers increasingly using embeddings to measure latent constructs such as novelty, creativity, and bias. However, this transition faces a fundamental validity challenge: the ''Proxy Presumption,'' or the reliance on geometric properties (e.g., cosine distance) as direct measures of social concepts. We argue that without explicit validation, unsupervised representations remain entangled mixtures of the target construct ($C$) and confounding attributes ($Z$) like topic, style, and authorship. To bridge the gap between semantic embeddings and valid social measures, we introduce the Construct Validity Protocol (CVP). Drawing on causal representation learning and psychometrics, the CVP offers a rigorous pipeline from conceptualization to quantitative verification. We further propose Counterfactual Neutralization, a novel method using LLMs to reduce confounding in embedding space. By providing a standardized Validity Suite -- including tests for discriminant, incremental, and predictive validity -- this work offers the community a toolkit to transform heuristic proxies into robust, scientifically defensible instruments.

嵌入验证社会测量效度评估

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。