用拓扑方法分析模型语义空间，提升跨语言理解测试可信度
Semantic Alignment of AI Models: Concept Collapse, Checkpoint Dynamics, and Cross-Lingual Transfer
- 用拓扑工具对比模型嵌入空间与知识图谱的结构相似性
- 发现模型在训练中存在概念坍缩现象，影响跨语言迁移能力
- 适合关注模型内在表征与多语言理解的研究者
语言模型评估面临挑战：仅靠输出结果无法检验模型对语言的概念化程度，且主流开源基准很快被模型吸收或饱和。本研究强调，除评估输出外，通过刻画语义结构可更深入理解模型如何关联抽象概念。针对高维嵌入空间难以解读的问题，本文展示如何利用拓扑方法，严格比较这些空间与低维可解释基线（如本体、人工构建的知识图谱）的结构一致性。多模态对齐测试使追踪模型演进过程、验证跨语言短语理解成为可能。
原文摘要 · Abstract (English)
Language model benchmarking is a difficult task. Outcome reasoning alone does not test the model's conceptualization of language and popular open-source benchmarks are quickly saturated or ingested as training data. It is important to test the model's output, but augmenting these tests by characterizing semantic structure gives more insight to how models relate abstract concepts. However, the high dimensional embedding spaces are not easy to interpret. This work demonstrates how topological methods can be used to rigorously compare these spaces to low dimensional and interpretable baselines like ontologies and curated knowledge graphs. These multi-modal alignment tests make it possible to track model adaptations and test phrase understanding across multiple languages.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。