提出新方法分离动态丰富性与表征性能，实现无误差的动态分析。
Decoupling Dynamical Richness from Representation Learning: Towards Practical Measurement
- 基于低秩偏差设计不依赖准确率的动态丰富性度量
- 可稳定捕捉从懒惰到丰富的过渡现象（如grokking）
- 适合研究训练策略如何影响模型动态与表征
动态特征变换（丰富态）并不总与预测性能（优质表征）一致，但现有研究常以准确率为丰富性的代理指标，限制了二者关系的深入分析。本文提出一种计算高效、独立于性能的丰富性度量，其理论基础为丰富动态中的低秩偏差，可还原神经坍缩作为特例。该度量在实验中比现有方法更稳定，无需依赖准确率即可捕捉已知的懒惰到丰富过渡现象（如grokking）。进一步用于分析训练因素（如学习率）对丰富性的影响，验证了已有假设并揭示新发现（如批量归一化促进丰富动态）。同时引入基于特征分解的可视化方法，提升可解释性，共同构成研究训练因素、动态行为与表征关系的诊断工具。
原文摘要 · Abstract (English)
Dynamic feature transformation (the rich regime) does not always align with predictive performance (better representation), yet accuracy is often used as a proxy for richness, limiting analysis of their relationship. We propose a computationally efficient, performance-independent metric of richness grounded in the low-rank bias of rich dynamics, which recovers neural collapse as a special case. The metric is empirically more stable than existing alternatives and captures known lazy-torich transitions (e.g., grokking) without relying on accuracy. We further use it to examine how training factors (e.g., learning rate) relate to richness, confirming recognized assumptions and highlighting new observations (e.g., batch normalization promotes rich dynamics). An eigendecomposition-based visualization is also introduced to support interpretability, together providing a diagnostic tool for studying the relationship between training factors, dynamics, and representations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。