arXiv:2603.23976cs.CV2026-03

用语言模型思路提升步态识别,让二值轮廓更像自然语言。

SilLang: Improving Gait Recognition with Silhouette Language Encoding

  • 将步态轮廓转为离散符号序列,类比自然语言处理。
  • 在SUSTech1K等三个数据集上超越现有最优方法。
  • 适合关注步态识别与大模型融合的研究者。

步态轮廓可编码为二值步态码,广泛用于表征行人运动模式。现有方法多依赖视觉主干网络提取连续视觉特征,忽略了二值轮廓固有的离散编码特性,而该特性与自然语言共享同一离散空间。大语言模型(LLMs)在处理离散序列、建模长程依赖方面表现出色,有望捕捉细微运动变化。受此启发,我们探索在二值编码空间中连接步态轮廓与自然语言。然而,文本词元与步态轮廓的编码空间存在频率与密度差异,导致对齐困难。为此,我们提出轮廓-速度分词器(Contour-Velocity Tokenizer),通过重塑轮廓分布使其更贴近文本词元空间。进而构建双分支框架Silhouette Language Model,通过融合来自LLM的离散语言嵌入来增强视觉轮廓表示。在主流步态主干网络上实现,SilLang在SUSTech1K、GREW和Gait3D三个基准上持续提升当前最优性能。

原文摘要 · Abstract (English)

Gait silhouettes, which can be encoded into binary gait codes, are widely adopted to representing motion patterns of pedestrian. Recent approaches commonly leverage visual backbones to encode gait silhouettes, achieving successful performance. However, they primarily focus on continuous visual features, overlooking the discrete nature of binary silhouettes that inherently share a discrete encoding space with natural language. Large Language Models (LLMs) have demonstrated exceptional capability in extracting discriminative features from discrete sequences and modeling long-range dependencies, highlighting their potential to capture temporal motion patterns by identifying subtle variations. Motivated by these observations, we explore bridging binary gait silhouettes and natural language within a binary encoding space. However, the encoding spaces of text tokens and binary gait silhouettes remain misaligned, primarily due to differences in token frequency and density. To address this issue, we propose the Contour-Velocity Tokenizer, which encodes binary gait silhouettes while reshaping their distribution to better align with the text token space. We then establish a dual-branch framework termed Silhouette Language Model, which enhances visual silhouettes by integrating discrete linguistic embeddings derived from LLMs. Implemented on mainstream gait backbones, SilLang consistently improves state-of-the-art methods across SUSTech1K, GREW, and Gait3D.

步态识别语言模型离散编码轮廓分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。