让边缘设备具备主动感知能力,实时规划视角适应动态环境。
Towards smart and adaptive agents for active sensing on edge devices
- 基于主动推理构建轻量级智能体,支持边缘端感知与决策
- 仅需300MB内存即可实现实时动态规划与感知
- 适用于无人机、安防等需要快速响应的边缘场景
TinyML使在低功耗边缘设备上部署深度学习模型成为可能,为资源受限环境中的实时感知开辟新机遇。然而,现有深度学习方法的适应性仅限于数据漂移,缺乏对环境动态性和不确定性建模的能力。当模型被缩放到边缘设备时,深度学习的扩展规律(如大规模数据和模型)无法适用,反而加剧了性能瓶颈。本文提出一种创新的智能体系统,可在设备端实现感知与规划,支持边缘上的主动感知。通过引入主动推理机制,该系统超越传统深度学习能力,能在动态环境中实时规划,并保持极小内存占用(最低仅300 MB)。我们通过在NVIDIA Jetson嵌入式设备上部署一个连接可俯仰/旋转摄像头的“扫视智能体”验证方案,该智能体依据主动推理原则生成最优控制策略,模拟人类眼动的扫视行为,适用于监控与机器人应用。
原文摘要 · Abstract (English)
TinyML has made deploying deep learning models on low-power edge devices feasible, creating new opportunities for real-time perception in constrained environments. However, the adaptability of such deep learning methods remains limited to data drift adaptation, lacking broader capabilities that account for the environment's underlying dynamics and inherent uncertainty. Deep learning's scaling laws, which counterbalance this limitation by massively up-scaling data and model size, cannot be applied when deploying on the Edge, where deep learning limitations are further amplified as models are scaled down for deployment on resource-constrained devices. This paper presents an innovative agentic system capable of performing on-device perception and planning, enabling active sensing on the edge. By incorporating active inference into our solution, our approach extends beyond deep learning capabilities, allowing the system to plan in dynamic environments while operating in real-time with a compact memory footprint of as little as 300 MB. We showcase our proposed system by creating and deploying a saccade agent connected to an IoT camera with pan and tilt capabilities on an NVIDIA Jetson embedded device. The saccade agent controls the camera's field of view following optimal policies derived from the active inference principles, simulating human-like saccadic motion for surveillance and robotics applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。