arXiv:2605.10732cs.CVcs.AI2026-05被引 1

融合视觉与骨骼信息,提升地铁支付动作识别准确率

iPay: Integrated Payment Action Recognition via Multimodal Networks and Adaptive Spatial Prior Learning

论文配图:iPay: Integrated Payment Action Recognition via Multimodal Networks and Adaptive Spatial Prior Learning
图 1 · 摘自论文原文
  • 采用多模态专家混合架构,分别处理图像与骨骼数据
  • 在真实场景数据上达到83.45%识别准确率,适合边缘部署
  • 引入相对运动建模,增强支付动作的判别能力

自动化公共交通支付分析对可扩展的票务审计和乘客行为分析至关重要,但实际仍依赖有限的人工检查。以往基于视觉和骨骼的方法在嘈杂的车载监控下表现脆弱,且常依赖泛化能力差的手工特征。借鉴图卷积网络在动作识别中的成功经验,我们发现骨骼特征擅长建模全局时空依赖,但忽略细微局部相对运动;而RGB特征保留精细空间细节,却在监控视频中常缺乏可靠的时间连续性。为此,我们提出iPay——一种面向车载监控系统的集成支付动作识别框架。iPay采用四流紧密耦合的多模态专家混合架构:(1) 基于区域聚焦计算的RGB专家流,强调局部证据;(2) 基于图卷积主干的骨骼专家流,建模关节运动;(3) 双注意力融合流,实现骨骼到图像的时间传递与图像到骨骼的空间增强;(4) 先验驱动的空间差异判别器(SDD),显式建模手与锚点的相对运动以提升任务判别力。我们还与本地交通机构合作,采集超过55小时的真实车载监控视频,生成500+支付片段。实验表明,iPay优于已有方法,在保持良好计算效率的同时达到83.45%的识别准确率,适合边缘部署。代码已开源。

原文摘要 · Abstract (English)

Automated transit payment analysis is vital for scalable fare auditing and passenger analytics, yet practice still relies on limited manual inspection. Prior vision- and skeleton-based methods remain brittle under noisy onboard surveillance and often depend on poorly generalizable handcrafted features. Building on the success of graph convolutional networks in human action recognition, we observe that skeleton features excel at modeling global spatiotemporal dependencies but tend to underemphasize the subtle local relative motions that distinguish payment actions. In contrast, RGB features preserve fine-grained spatial details yet often lack reliable temporal continuity in surveillance footage. To bridge both system-level deployment needs and model-level design challenges, we present iPay, an integrated payment action recognition framework for onboard transit surveillance system. iPay adopts a multimodal mixture-of-experts architecture with four tightly coupled streams: (1) an RGB expert stream emphasizing local evidence via region-focused computation; (2) a skeleton expert stream modeling articulated motion with a graph convolutional backbone; (3) a dual-attention fusion stream enabling skeleton-to-RGB temporal transfer and RGB-to-skeleton spatial enhancement; and (4) a prior-driven Spatial Difference Discriminator (SDD) that explicitly models hand-to-anchor relative motion to improve task-specific discriminability. We also collaborate with local transit agencies to collect over 55 hours of real onboard surveillance footage, yielding 500+ payment clips. Experiments show that iPay outperforms prior methods and achieves 83.45\% recognition accuracy with competitive computational efficiency, making it suitable for edge deployment. Code is available at https://github.com/ccoopq/iPay.

动作识别多模态交通分析边缘计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。