arXiv:2609.05721cs.AIcs.IR2026-09

用投影法从语言模型嵌入中提取时间与地理信息,无需模型权重。

Recovering Temporal and Geographic Signals from Language Model Embeddings

论文配图:Recovering Temporal and Geographic Signals from Language Model Embeddings
图 1 · 摘自论文原文
  • 通过少量种子样本定义嵌入空间中的坐标轴,实现对文本的时序与空间排序。
  • 在多个数据集上成功恢复出有意义的时间与地理结构。
  • 黑箱适用,适合分析仅提供API的现代语言模型。

理解语言模型嵌入是否包含结构化的现实世界信息,对表征分析和信息检索至关重要。本文采用一种基于投影的简单方法,直接作用于输出嵌入,研究其对时间与地理信号的编码能力。给定少量种子示例,该方法在嵌入空间中定义一条轴,并根据文本或实体在该轴上的投影进行排序。该方法完全为黑箱且模型无关,仅需嵌入向量,无需模型权重、内部激活、辅助探测器或额外训练。因此适用于仅通过API提供的现代嵌入模型,提供了一种轻量级分析工具,可判断时间与空间维度是否存在于其表示空间中。我们在时间与地理数据集上应用该方法,发现嵌入投影能有效恢复出有意义的时序与空间结构。结果表明输出嵌入确实编码了与时空相关的信息,同时为可解释性分析及下游任务(如时间排序、地理排名与标注)提供了实用工具。

原文摘要 · Abstract (English)

Understanding whether language-model embeddings encode structured real-world information is important for both representation analysis and information retrieval. We study this question for temporal and geographic signals using a simple projection-based method that operates directly on output embeddings. Given a small set of seed examples, the method defines an axis in embedding space and ranks texts or entities by their projection onto that axis. Our approach is fully black-box and model-agnostic: it requires only embeddings, without access to model weights, internal activations, auxiliary probes, or additional training. This makes it applicable to modern embedding models available only through APIs and provides a lightweight way to analyze whether temporal and spatial dimensions are present in their representation spaces. We apply the method to temporal and geographic datasets and find that embedding projections recover meaningful chronological and spatial structure. These results provide evidence that output embeddings encode signals relevant to time and space, while also offering a practical tool for interpretability and for downstream temporal and geographic information retrieval tasks, such as temporal ordering, geographic ranking, and tagging.

嵌入分析时序建模地理信息

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。