arXiv:2412.04858cs.AI2024-12中稿 · WWW 2025 for a tut…被引 2

重提数据表示中的不变性,为深度学习提供新思路。

Rethink Deep Learning with Invariance in Data Representation

  • 以对称性为先验,设计具备不变性的数据表示
  • 指出传统CNN忽视不变性导致鲁棒性等瓶颈
  • 适合关注模型可解释性与高效设计的研究者

将不变性融入数据表示是智能系统与网络应用中的基本原则。表示在系统构建中起根本作用,系统基于数字输入的有意义表示(而非原始数据)运行。恰当的设计或学习这些表示依赖于特定任务的先验知识。其中,来自埃尔朗根纲领的对称性概念是最有成效的先验——一个系统的对称性是使系统某性质保持不变的变换。对称性先验无处不在,例如物体分类中的平移对称性:物体类别在平移下保持不变。对不变性的追求与模式识别和数据挖掘的历史一样悠久。在深度学习之前的时代,不变性设计是多种表示方法的基石,如SIFT。进入深度学习早期,不变性原则被数据驱动范式(如CNN)取代,但很快暴露出鲁棒性、可解释性、效率等方面的瓶颈。如今,在重新思考深度学习的背景下,不变性原则回归,并催生了几何深度学习(GDL)这一新领域。本文从历史视角回顾数据表示中的不变性,重点识别研究困境、有前景的工作、未来方向及网络应用。

原文摘要 · Abstract (English)

Integrating invariance into data representations is a principled design in intelligent systems and web applications. Representations play a fundamental role, where systems and applications are both built on meaningful representations of digital inputs (rather than the raw data). In fact, the proper design/learning of such representations relies on priors w.r.t. the task of interest. Here, the concept of symmetry from the Erlangen Program may be the most fruitful prior -- informally, a symmetry of a system is a transformation that leaves a certain property of the system invariant. Symmetry priors are ubiquitous, e.g., translation as a symmetry of the object classification, where object category is invariant under translation. The quest for invariance is as old as pattern recognition and data mining itself. Invariant design has been the cornerstone of various representations in the era before deep learning, such as the SIFT. As we enter the early era of deep learning, the invariance principle is largely ignored and replaced by a data-driven paradigm, such as the CNN. However, this neglect did not last long before they encountered bottlenecks regarding robustness, interpretability, efficiency, and so on. The invariance principle has returned in the era of rethinking deep learning, forming a new field known as Geometric Deep Learning (GDL). In this tutorial, we will give a historical perspective of the invariance in data representations. More importantly, we will identify those research dilemmas, promising works, future directions, and web applications.

不变性几何深度学习表示学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。