arXiv:2605.20635cs.LGmath.ST2026-05

提出统一框架,用局部核与局部均值解释多种机器学习模型

The General Theory of Localization Methods

论文配图:The General Theory of Localization Methods
图 1 · 摘自论文原文
  • 基于局部核和局部均值构建通用学习框架
  • 揭示Transformer等模型可由分层局部模型构造
  • 适合研究模型统一理论与新型自适应系统设计者

本文提出一种名为定位方法的通用机器学习框架,其核心基于两个基本概念:定位核与局部均值——这正是自注意力机制的关键组成部分。为建立严谨理论基础,该框架通过两个支柱正式定义:局部(化)模型表述与定位技巧。系统性探讨了该方法与众多现有机器学习模型/方法的关联,包括核方法、懒学习、MeanShift算法、松弛标记、霍普菲尔德网络、局部线性嵌入(LLE)、模糊推理及去噪自编码器(DAEs)。通过剖析这些关系,阐明了定位方法的广泛理论意义,并验证其在多样化机器学习任务中的实际应用能力。进一步探索框架的高级扩展,如自适应核、分层局部模型与非局部模型。特别地,表明Transformer——现代序列建模的基石——可由分层局部模型构建,揭示该方法统一并泛化前沿架构的能力。本工作不仅为重新诠释现有模型提供统一理论视角,也为设计灵活、数据自适应的学习系统提供新方法工具。

原文摘要 · Abstract (English)

This paper proposes a general machine learning framework called the localization method, which is fundamentally built on two core concepts: localization kernels and local means -- key components that underpin the self-attention mechanism. To establish a rigorous theoretical foundation, the framework is formally defined through two essential pillars: the formulation of the local(-ized) model and the localization trick. We systematically investigate the connections between the localization method and a wide range of existing machine learning models/methods, including (but not limited to) kernel methods, lazy learning, the MeanShift algorithm, relaxation labeling, Hopfield networks, local linear embedding (LLE), fuzzy inference, and denoising autoencoders (DAEs). By dissecting these relationships, we clarify the broader theoretical significance of the localization method and demonstrate its practical applicability across diverse machine learning tasks. Furthermore, we explore advanced extensions of the framework, such as adaptive kernels, hierarchical local models, and non-local models. Notably, we show that the Transformer -- a cornerstone of modern sequence modeling -- can be constructed using hierarchical local models, revealing the ability of the localization method to unify and generalize state-of-the-art architectures. This work not only provides a unified theoretical lens to reinterpret existing models but also offers new methodological tools for designing flexible, data-adaptive learning systems.

理论统一自注意力模型解析框架设计

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。