用多尺度注意力捕捉鼠类社交行为动态,自动识别更准更快。
MSGL-Transformer: A Multi-Scale Global-Local Transformer for Rodent Social Behavior Recognition
- 设计多尺度并行注意力结构,同时捕捉短、中、长时序行为特征。
- 在两个数据集上准确率超87%,较现有方法提升超10%。
- 轻量级架构可跨数据集通用,适合神经行为学研究者使用。
鼠类行为识别对理解神经与行为机制至关重要。传统人工评分耗时且易出错。本文提出MSGL-Transformer,一种基于姿态时序序列的多尺度全局-局部变换器,用于识别鼠类社交行为。模型采用轻量级变压器编码器,结合多尺度注意力,捕捉不同时间尺度上的运动动态。架构包含并行的短程、中程和全局注意力分支,显式建模多时序层次的行为动态。我们引入受SE网络启发的行为感知调制(BAM)模块,通过调节时间嵌入,强化行为相关特征。在两个数据集上评估:RatSI(5类行为,12维姿态输入)和CalMS21(4类行为,28维姿态输入)。在RatSI上,平均准确率达75.4%,F1分数为0.745,优于TCN、LSTM和Bi-LSTM;在CalMS21上,准确率87.1%,F1分数0.8745,较HSTWFormer提升10.7%,优于ST-GCN、MS-G3D、CTR-GCN和STGAT。相同架构仅需调整输入维度与类别数即可跨数据集泛化。
原文摘要 · Abstract (English)
Recognition of rodent behavior is important for understanding neural and behavioral mechanisms. Traditional manual scoring is time-consuming and prone to human error. We propose MSGL-Transformer, a Multi-Scale Global-Local Transformer for recognizing rodent social behaviors from pose-based temporal sequences. The model employs a lightweight transformer encoder with multi-scale attention to capture motion dynamics across different temporal scales. The architecture integrates parallel short-range, medium-range, and global attention branches to explicitly capture behavior dynamics at multiple temporal scales. We also introduce a Behavior-Aware Modulation (BAM) block, inspired by SE-Networks, which modulates temporal embeddings to emphasize behavior-relevant features prior to attention. We evaluate on two datasets: RatSI (5 behavior classes, 12D pose inputs) and CalMS21 (4 behavior classes, 28D pose inputs). On RatSI, MSGL-Transformer achieves 75.4% mean accuracy and F1-score of 0.745 across nine cross-validation splits, outperforming TCN, LSTM, and Bi-LSTM. On CalMS21, it achieves 87.1% accuracy and F1-score of 0.8745, a +10.7% improvement over HSTWFormer, and outperforms ST-GCN, MS-G3D, CTR-GCN, and STGAT. The same architecture generalizes across both datasets with only input dimensionality and number of classes adjusted.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。