用几何优化提升ViT特征表达，让模型看懂数据间的全局关系。
Proximal Vision Transformer: Enhancing Feature Representation through Two-Stage Manifold Geometry
- 将自注意力机制视为流形的切空间构造，从多视角提取几何特征。
- 通过近端迭代实现特征在流形上的对齐与优化，分类准确率显著提升。
- 适合研究视觉表征学习与几何深度学习的学者参考。
视觉变换器（ViT）在计算机视觉中广泛应用，凭借其自注意力机制在各类任务中取得显著成果。然而，现有ViT的优化仍局限于单张图像内的局部关系建模，难以捕捉数据点间的全局几何结构。为此，本文提出一种新框架，将ViT与近端工具结合，实现统一的几何优化，以增强特征表示与分类性能。该框架利用ViT的自注意力机制构建流形的切丛，每个注意力头对应一个切空间，从不同局部视角提供几何表示。随后引入近端迭代，在切丛中定义截面，并将数据从切空间投影回基空间，实现全局特征对齐与优化。实验结果表明，所提方法在分类精度与数据分布上均优于传统ViT。
原文摘要 · Abstract (English)
The Vision Transformer (ViT) architecture has become widely recognized in computer vision, leveraging its self-attention mechanism to achieve remarkable success across various tasks. Despite its strengths, ViT's optimization remains confined to modeling local relationships within individual images, limiting its ability to capture the global geometric relationships between data points. To address this limitation, this paper proposes a novel framework that integrates ViT with the proximal tools, enabling a unified geometric optimization approach to enhance feature representation and classification performance. In this framework, ViT constructs the tangent bundle of the manifold through its self-attention mechanism, where each attention head corresponds to a tangent space, offering geometric representations from diverse local perspectives. Proximal iterations are then introduced to define sections within the tangent bundle and project data from tangent spaces onto the base space, achieving global feature alignment and optimization. Experimental results confirm that the proposed method outperforms traditional ViT in terms of classification accuracy and data distribution.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。