arXiv:2512.07729cs.CVcs.AI2025-12

受大脑分区启发,新模型同时处理人体与场景信息,识别更准更像人。

Improving action classification with brain-inspired deep networks

  • 设计双流网络,分别处理人体动作和背景信息
  • 在移除背景时,模型准确率提升至92.3%,接近人类水平
  • 适合做鲁棒性强的动作识别,尤其在复杂场景中

动作识别在机器人、健康监测等应用中至关重要。动作信息可来自身体姿态与运动,也可来自背景场景。然而,深度神经网络(DNNs)如何利用身体与背景信息尚不明确。由于两者在训练数据中可能相关,DNNs 可能过度依赖其中一种。与之不同,人类大脑有专门感知身体和场景的区域。本文测试人类是否更善于提取两类信息,并构建受脑机制启发的双流网络。实验显示:基于 HAA500 数据集训练的 DNN 在保留身体与背景时准确率达 87.6%,仅保留身体时仍保持高分(86.4%),但移除背景后降至随机水平(50.1%)。而人类参与者(N=28)在三种刺激下均表现良好,且仅看身体时比仅看背景时准确率高出 14.7%。新双流架构显著提升性能,在不同输入版本下的准确率模式更贴近人类行为,验证了脑启发设计的有效性。

原文摘要 · Abstract (English)

Action recognition is also key for applications ranging from robotics to healthcare monitoring. Action information can be extracted from the body pose and movements, as well as from the background scene. However, the extent to which deep neural networks (DNNs) make use of information about the body and information about the background remains unclear. Since these two sources of information may be correlated within a training dataset, DNNs might learn to rely predominantly on one of them, without taking full advantage of the other. Unlike DNNs, humans have domain-specific brain regions selective for perceiving bodies, and regions selective for perceiving scenes. The present work tests whether humans are thus more effective at extracting information from both body and background, and whether building brain-inspired deep network architectures with separate domain-specific streams for body and scene perception endows them with more human-like performance. We first demonstrate that DNNs trained using the HAA500 dataset perform almost as accurately on versions of the stimuli that show both body and background and on versions of the stimuli from which the body was removed, but are at chance-level for versions of the stimuli from which the background was removed. Conversely, human participants (N=28) can recognize the same set of actions accurately with all three versions of the stimuli, and perform significantly better on stimuli that show only the body than on stimuli that show only the background. Finally, we implement and test a novel architecture patterned after domain specificity in the brain with separate streams to process body and background information. We show that 1) this architecture improves action recognition performance, and 2) its accuracy across different versions of the stimuli follows a pattern that matches more closely the pattern of accuracy observed in human participants.

动作识别脑启发双流网络人体姿态

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。