测试简单转换在不同文本嵌入模型间的迁移能力
How Far Do Simple Transformations Translate Across Text Embedding Models?
- 用线性映射等轻量级方法尝试跨模型转换表示
- 九种模型中部分组合可成功迁移，部分完全失败
- 模型架构和训练目标共同决定转换兼容性
我们研究了简单变换是否能在异构文本嵌入模型间实现表示迁移。理解独立训练的模型如何组织语义信息，是实现AI间无需解码为人类可读文本的潜在通信的关键。聚焦于线性映射等轻量级翻译器，在超越简化基准的真实文本场景下检验了隐空间普遍性的文献假设。在九种架构、池化策略和训练目标各异的嵌入模型上，通过核中心化相关性（CKA）、下游迁移、保真度和检索任务评估兼容性。结果表明，简单翻译器能恢复有意义的共享结构，并支持部分兼容模型对的迁移，但在其他情况下则急剧失效。兼容性由架构、训练目标、池化方式和数据分布共同决定。总体而言，异构嵌入空间并非如某些文献所言普遍由简单映射关联。
原文摘要 · Abstract (English)
We investigate whether simple transformations can translate representations across heterogeneous text embedding models. Understanding how independently trained models organize semantic information is an enabler for AI-to-AI latent communication without decoding into human-readable text. Focusing on lightweight translators such as linear mappings, we test the literature hypothesis of latent universality in a realistic text setting beyond simplified benchmarks. Across nine embedding models differing in architecture, pooling strategy, and training objective, we evaluate compatibility using CKA, downstream transfer, fidelity, and retrieval. Simple translators recover meaningful shared structure and support transfer for some compatible pairs, but fail sharply for others. Compatibility depends jointly on architecture, training objective, pooling, and data distribution. Overall, the results show that heterogeneous embedding spaces are not universally related by simple mappings as often suggested in some literature.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。