轻量级模型分离行人动作与外观,提升异常行为搜索精度
LightAIR: Lightweight Action Inversion and Riemannian Rectification for Text-based Person Anomaly Search

- 用文本语义引导动作特征提取,避免外观干扰
- 通过正交投影实现动作与外观严格解耦
- 在黎曼流形上优化梯度,防止学习捷径
传统文本驱动行人搜索仅匹配静态外观特征,忽略动态行为信息。文本驱动行人异常搜索(TPAS)需同时定位微观异常行为并匹配宏观外观,但现有方法受限于外部姿态估计器在复杂场景中脆弱,以及隐式学习时像素级纠缠导致的视觉解耦失败,使主导外观信息淹没细微动作特征。此外,在欧氏空间中对困难负样本(相同外观、不同动作)进行对比优化会引发严重捷径学习。为此,提出轻量级动作逆向与黎曼校正网络(LightAIR)。首先,通过轻量动作逆向算子引入文本语义先验作为锚点,提取纯净动作特征,克服视觉固有耦合;其次,采用正交零空间投影,将外观特征约束在动作特征正交补空间内,确保严格前向解耦;最后设计梯度校正模块,计算黎曼梯度以约束反向传播路径,强制梯度沿保持解耦性质的切空间更新,切断有害捷径。在主流TPAS和TIPR数据集上的大量实验表明,LightAIR显著优于现有最先进方法。代码已开源。
原文摘要 · Abstract (English)
Traditional Text-based Person Search (TPS) is typically limited to matching static appearance attributes, severely neglecting dynamic action information. The Text-based Person Anomaly Search (TPAS) task bridges this gap, requiring models to locate micro-level specific abnormal behaviors while matching macro-level appearance of pedestrians. However, current TPAS methods face fundamental limitations: external explicit pose estimators are fragile in unconstrained surveillance scenarios, and implicit learning encounters visual decoupling failure under pixel-level entanglement, causing dominant appearance information to easily swallow and contaminate subtle action features. Furthermore, performing contrastive optimization on hard negative samples (``same appearance, different actions'') in conventional Euclidean spaces induces severe shortcut learning. To address these, we propose the Lightweight Action Inversion and Riemannian rectification network (LightAIR). First, it introduces textual semantic priors as anchors via a lightweight action inversion operator to extract pure action features, thereby overcoming visual-inherent coupling. Subsequently, it employs orthogonal null-space projection to constrain appearance features within the orthogonal complement space of action features, guaranteeing strict forward decoupling. Finally, we designed a gradient rectification module that computes the Riemannian gradient to constrain the backpropagation trajectory, forcing the gradient flow to update strictly along the tangent space that preserves decoupling properties, thereby cutting off harmful shortcuts. Extensive experiments on the widely used TPAS and TIPR datasets demonstrate that LightAIR significantly outperforms existing state-of-the-art methods. Codes are available at https://github.com/rainy-london/LightAIR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。