用事件相机实现低光下快速上身人形机器人远程操控
Event-Based Upper-Body Humanoid Teleoperation Under Challenging Illumination
- 采用事件相机与惯性数据融合，实现实时感知
- 在5流明以下暗光环境仍保持23-34毫秒端到端延迟
- 适合快速动作或光照恶劣场景下的机器人遥操作
我们提出一种基于神经形态事件视觉的实时上身人形机器人运动模仿框架。针对传统帧式RGB传感器在高动态范围场景和快速运动中因固定积分时间导致的性能瓶颈，本工作利用Prophesee EVK4事件相机，其具备超过120 dB动态范围和高时间分辨率，支持在严重逆光及低于5流明的极暗环境下稳定追踪。系统架构包含低延迟感知模块（优化事件累积与重力对齐惯性融合）和因果运动模块（TWIST），实现在线运动重映射。在嵌入式NVIDIA Booster T1平台与18自由度人形上身系统上验证，端到端光子到动作延迟为23-34毫秒，在实验条件下优于RGB基线。结果表明：事件相机在快速或低光环境下更优，而光照充足且静态场景可能更适合使用RGB或混合传感。
原文摘要 · Abstract (English)
We present a real-time upper-body human-to-humanoid motion imitation framework driven by neuromorphic event-based vision. This work addresses practical perceptual bottlenecks of conventional frame-based RGB sensors, specifically their difficulty in high dynamic range (HDR) scenes and rapid motions due to fixed integration times. By leveraging the Prophesee EVK4 event camera, which operates asynchronously with high temporal resolution and a dynamic range exceeding 120 dB, our system supports stable tracking in conditions where standard vision pipelines degrade, such as severe backlighting and very low light environments below 5 lux. The architecture integrates a low-latency Perception Module, utilizing optimized event accumulation and gravity-aligned inertial fusion, with a causal Motion Module (TWIST) that performs online kinematic retargeting. We validate the system on an embedded NVIDIA Booster T1 platform and an 18-DoF humanoid upper-body setup, demonstrating an end-to-end photon-to-action latency of 23-34 ms and advantages over RGB baselines under our experimental setup. The results indicate a practical trade-off: events can be preferable for fast or poorly lit upper-body teleoperation, whereas well-lit static scenes may favor RGB or hybrid sensing.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。