模拟人脑视觉通路,让机器同时理解明暗和复杂运动模式。
Machine Learning Modeling for Multi-order Human Visual Motion Processing
- 构建双通路模型,模仿大脑V1-MT区域处理运动信号的机制。
- 在非朗伯材质视频上训练后,模型自然学会感知第二类运动。
- 适合研究生物视觉、计算机视觉与神经启发模型的学者。
本研究旨在开发能像人类一样感知视觉运动的机器。尽管当前计算机视觉中的深度神经网络已能准确估计自然图像中的光流,但在架构与行为上仍与生物视觉系统存在显著差异。人类具备感知高阶图像特征运动(第二类运动)的能力,而多数视觉模型因依赖亮度守恒定律而无法捕捉此类信息。为此,我们设计了模仿皮层V1-MT运动处理通路的模型架构,包含可训练的运动能量传感器阵列与递归图网络。通过多样自然视频的监督学习,模型成功复现了第一类(基于亮度)运动感知的心理物理与生理学现象。针对第二类运动,受神经科学启发,引入非线性预处理路径,使用简单3D多层CNN块实现。研究发现,在非朗伯材质物体运动数据集上训练时,模型自然获得感知第二类运动的能力——这正是人类在光泽表面等光学波动环境中识别稳定物体运动的关键能力。最终模型不仅符合生物系统特性,还能泛化至自然场景中的一阶与二阶运动现象。
原文摘要 · Abstract (English)
Our research aims to develop machines that learn to perceive visual motion as do humans. While recent advances in computer vision (CV) have enabled DNN-based models to accurately estimate optical flow in naturalistic images, a significant disparity remains between CV models and the biological visual system in both architecture and behavior. This disparity includes humans' ability to perceive the motion of higher-order image features (second-order motion), which many CV models fail to capture because of their reliance on the intensity conservation law. Our model architecture mimics the cortical V1-MT motion processing pathway, utilizing a trainable motion energy sensor bank and a recurrent graph network. Supervised learning employing diverse naturalistic videos allows the model to replicate psychophysical and physiological findings about first-order (luminance-based) motion perception. For second-order motion, inspired by neuroscientific findings, the model includes an additional sensing pathway with nonlinear preprocessing before motion energy sensing, implemented using a simple multilayer 3D CNN block. When exploring how the brain acquired the ability to perceive second-order motion in natural environments, in which pure second-order signals are rare, we hypothesized that second-order mechanisms were critical when estimating robust object motion amidst optical fluctuations, such as highlights on glossy surfaces. We trained our dual-pathway model on novel motion datasets with varying material properties of moving objects. We found that training to estimate object motion from non-Lambertian materials naturally endowed the model with the capacity to perceive second-order motion, as can humans. The resulting model effectively aligns with biological systems while generalizing to both first- and second-order motion phenomena in natural scenes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。