1000亿参数语言模型能编码夜空地图，且可从残差流中解码。
Sky sphere representation in language models
- 从残差流中解码夜空坐标表示，利用主成分分析提取特征。
- LOO测试显示解释方差达65%-85%（R²），平均角度误差12°-21°。
- 发现首个高维曲面不可约特征流形，适合对空间认知感兴趣的读者。
我们分析了规模约为1000亿参数的语言模型是否在残差流中存在可解码的夜空地图表征。结果表明，大多数开源模型确实具备此类表征，且在询问“此物体附近有哪些天体”等提示下，该表征常出现在前几主成分中。除一个模型外，所有模型在留一法（LOO）测试中均表现出显著得分，解释方差高达65%-85%（R²），中位角误差低至12°-21°。我们验证了该表征并非来自相关平面表示的泄露。据我们所知，这是首个被发现的高维弯曲不可约特征流形。论文代码已公开于https://github.com/l3erdnik/Decodable-sky。
原文摘要 · Abstract (English)
We analyze whether language models of size ~100B have a representation of the night sky map that is decodable from their residual stream. We find that most of the considered open-source models do have such a representation, and it often even surfaces to the top principal components on prompts that ask questions like ``what is close to this object in the night sky''. In all but one model this representation showed significant scores in LOO testing, containing up to 65-85% of variance ($R^2$-score) and having median angular error down to $12^\circ-21^\circ$. We verify that our representation is not a simple leak from a correlated flat representation. To our knowledge, this representation is the first example of a curved high-dimensional irreducible feature manifold. Codes used in the paper are published at https://github.com/l3erdnik/Decodable-sky
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。