让文本嵌入捕捉隐含语义,而非仅表面意思。
Position: Text Embeddings Should Capture Implicit Semantics, Not Just Surface Meaning
- 主张将隐含语义作为嵌入模型的核心目标
- 现有效果仅略优于简单词典基线
- 适合关注语用与社会语境的NLP研究者
本文主张文本嵌入研究应超越表面语义,将隐含语义作为核心建模目标。当前嵌入模型多聚焦表层语义,而语言学理论指出人类意义的很大一部分是隐含的,受语用、说话人意图及社会文化背景影响。现有训练数据缺乏深度,评估基准也偏向表面相似性,导致模型在需要解释性推理、立场识别或社会语境理解的任务中表现不佳。我们的初步研究表明,即使是最先进的嵌入模型,在探测隐含语义的任务上,效果也仅比基础词汇基线略有提升。因此我们呼吁范式转变:使用更丰富、语言学导向的训练数据,构建能探测深层语义理解的新基准,并将隐含意义视为嵌入建模的核心目标,以更好地匹配真实语言的复杂性。代码已公开于 http://github.com/dukesun99/Implicit-Embeddings。
原文摘要 · Abstract (English)
This position paper argues that text embedding research should move beyond surface meaning and embrace implicit semantics as a central modeling objective. Text embeddings are a foundational component of modern NLP, underpinning a wide range of applications and driving sustained research progress. Despite rapid progress, most embedding models remain narrowly focused on surface-level semantics, whereas linguistic theory emphasizes that much of human meaning is implicit, shaped by pragmatics, speaker intent, and sociocultural context. Current models are typically trained on datasets that lack such depth and evaluated using benchmarks that reward surface similarity. As a result, they struggle with tasks that require interpretive reasoning, stance recognition, or socially grounded understanding. Our pilot study makes this limitation explicit, showing that even state-of-the-art embeddings achieve only marginal improvements over simple lexical baselines on tasks probing implicit semantics. We therefore call for a paradigm shift: embedding research should prioritize linguistically grounded and diverse training data, develop benchmarks that probe deeper semantic understanding, and treat implicit meaning as a core modeling objective to better align embeddings with real-world language complexity. The code is available at http://github.com/dukesun99/Implicit-Embeddings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。