警惕过度解读深度学习现象，应聚焦真正有价值的理论探索。
Not All Explanations for Deep Learning Phenomena Are Equally Valuable
- 反对为孤立现象定制解释，主张回归通用原理研究。
- 多数反直觉现象在真实场景中罕见，研究投入需更审慎。
- 现象可作为打磨通用理论的实验场，适合长期主义者。
近年来，理解深度学习中的反直觉现象（如双下降、隐性学习、彩票定理等）成为研究重点。现有工作多针对个别现象提出临时假设，缺乏普适性。本文指出，许多现象在真实应用中并不常见，此类研究可能低效且偏离领域主流目标。因此，不应将它们视为需单独破解的谜题。但这些现象仍具价值，可作为检验和优化深层学习通用理论的特殊实验环境。通过分析近期典型案例的研究成果，本文反思当前学术范式，并提出未来研究建议，旨在使现象研究与整个深度学习领域的实际进展保持一致。
原文摘要 · Abstract (English)
Developing a better understanding of surprising or counterintuitive phenomena has constituted a significant portion of deep learning research in recent years. These include double descent, grokking, and the lottery ticket hypothesis -- among many others. Works in this area often develop ad hoc hypotheses attempting to explain these observed phenomena on an isolated, case-by-case basis. This position paper asserts that, in many prominent cases, there is little evidence to suggest that these phenomena appear in real-world applications and these efforts may be inefficient in driving progress in the broader field. Consequently, we argue against viewing them as isolated puzzles that require bespoke resolutions or explanations. However, despite this, we suggest that deep learning phenomena do still offer research value by providing unique settings in which we can refine our broad explanatory theories of more general deep learning principles. This position is reinforced by analyzing the research outcomes of several prominent examples of these phenomena from the recent literature. We revisit the current norms in the research community in approaching these problems and propose practical recommendations for future research, aiming to ensure that progress on deep learning phenomena is well aligned with the ultimate pragmatic goal of progress in the broader field of deep learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。