arXiv:2608.27020stat.MLcs.LG2026-08

发现主流方法会受隐藏坐标选择影响,导致测量结果不可靠。

Representation Measurements Under Function-Preserving Reparameterizations

论文配图:Representation Measurements Under Function-Preserving Reparameterizations
图 1 · 摘自论文原文
  • 提出函数保持重参数化不变性检验标准
  • 实测五模型三领域下组件数中位差异达0.79
  • 建议用正交不变评分替代传统平行分析

语言模型的隐层坐标不唯一由输入-输出函数决定,因此基于表示的测量应保持函数保持重参数化的不变性。本研究发现,列置换并行分析违反该不变性:在模型函数与观测协方差谱不变的情况下,其参考分布和选定成分数仍可改变。更普遍地,数据内参考过程无法同时满足每坐标边际分布保持、正交等变性及消除跨坐标协方差。在五个模型、三个检索领域、75次变换下,中位成分数分歧为0.79,中位阈值决策分歧为0.26。仅中心化控制实验显示,1141/1200个成分数发生变化,而独立并行分析种子变化为零。相比之下,正交不变比较分数数值稳定且判别能力相当。结果表明,并行分析得出的成分数与决策可能反映隐藏坐标选择,而非模型的确定属性。

原文摘要 · Abstract (English)

Hidden coordinates are not uniquely determined by a language model's input--output function, so representation-derived measurements should be invariant to function-preserving changes of basis. This study shows that column-permutation parallel analysis violates function-preserving reparameterization invariance because its reference distribution and selected component count can change while the model function and observed covariance spectrum remain fixed. More generally, a data-internal reference procedure cannot simultaneously preserve every coordinate marginal, remain orthogonally equivariant, and remove cross-coordinate covariance. Empirically, across five models, three retrieval domains, and 75 transformations, median component-count disagreement is 0.79 and median fixed-threshold decision disagreement is 0.26. A centering-only control isolates the reference-driven effect, with 1,141 of 1,200 component counts changing despite an unchanged observed spectrum, whereas independent parallel analysis seeds change none of the corresponding decisions. By contrast, orthogonally invariant comparator scores remain numerically stable with similar held-out discrimination. Together, these results show that parallel analysis-derived component counts and decisions can reflect hidden-coordinate choice rather than a well-defined property of the model.

表示测量模型不变性并行分析语言模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。