发现大模型答题时存在几何转变关键层，与准确率高度相关。
Isotropy Cliffs: The Geometric Signature of Decision-Making in Large Language Models
- 通过各向同性分析揭示模型答题时的几何转变层。
- 该转变层与任务相关聚类出现同步，准确率相关性达0.84。
- 转变对提示变化鲁棒，反映通用决策机制，适合模型解释研究者。
我们从各向同性的角度研究多选题问答中大模型的决策几何特征。分析五种开源模型在多个数据集上的表现，发现决策关键层具有各向同性突变，伴随表征显著变化和任务相关聚类的涌现。该同步几何行为与下游准确率高度相关（r≈0.84），表明其对成功决策的重要意义。此外，该转变对提示变化保持鲁棒，暗示其反映了模型行为的普遍机制。
原文摘要 · Abstract (English)
We investigate the geometry of decision-making in Multiple Choice Question Answering (MCQA) through the lens of isotropy. Analyzing five open-weight models across diverse datasets, we identify decision-critical transition layers characterized by a shift in isotropy, coinciding with a major representational change and the emergence of task-relevant clusters. We demonstrate that this synchronized geometric behavior is strongly correlated with downstream accuracy ($r\approx0.84$), displaying its relevance for successful decision-making. Furthermore, we show that this transition is robust to prompt variations, suggesting that it reflects a general mechanism of model behavior.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。