机器人用语义通信协作，让传输内容直接助力任务执行。
When Robots Exchange Meaning: A Demo of Goal-Oriented Semantic Communications for Collaborative Robotics
- 机器人将视觉信息压缩为语义令牌，边缘节点重建并生成语义地图。
- 320×240图像仅需5400字节传输，相比原始数据压缩42.67倍。
- 适合研究语义通信、具身智能与6G协同机器人系统的人群。
协作机器人是6G的典型任务导向应用场景，通信质量应体现在任务完成度、环境理解与闭环操作上，而非仅关注包送达率。本文展示了一个集成机器人端视觉压缩、边缘端语义建图与可视化交互的语义通信（SemCom）测试平台。搭载RGB-D传感器和LiDAR的移动机器人运行ROS 2，Jetson Orin边缘节点负责重建、RTAB-Map建图、语义对象处理及浏览器可视化。作为初步概念验证，机器人端使用ONNX Runtime编码器将RGB帧转为VQ-VAE令牌，边缘端用PyTorch解码器重建。320×240图像被表示为80×60的令牌网格，打包后仅5400字节，相较模型输入的原始RGB数据减少42.67倍。重建后的视觉流与深度、位姿及3D地图信息融合，生成可用于下游机器人的语义地图。该演示完整呈现了从语义视觉传输到物体级地图交互的全流程，为未来任务感知型6G网络研究提供了实用平台，涵盖语义通信、具身人工智能与物理智能驱动的机器人技术交汇领域。演示视频见：https://tinyurl.com/Tos09
原文摘要 · Abstract (English)
Collaborative robotics is a representative task-oriented 6G use-case, where communication quality should be reflected in mission execution, environment understanding, and closed-loop operation rather than packet delivery alone. This demo paper presents a robot-edge semantic communication (SemCom) testbed integrating robot-side visual compression, edge-side semantic mapping, and dashboard-based mission interaction. A mobile robot equipped with RGB-D sensing and LiDAR runs ROS 2, while a Jetson Orin edge node performs reconstruction, RTAB-Map mapping, semantic object handling, and browserbased visualization. As an initial proof of concept, RGB frames are encoded on the robot into VQ-VAE tokens using an ONNX Runtime encoder and reconstructed on the edge using a PyTorch decoder. A 320 X 240 image is represented by an 80 X 60 token grid with a packed payload of 5400 bytes, corresponding to a 42.67X reduction relative to model-input RGB bytes. The reconstructed visual stream is further associated with depth, pose, and 3D mapping information to generate a semantic map for downstream robotic applications. The demo exposes the full path from semantic visual transport to object-level map interaction, and provides a practical platform for future task-aware 6G networking studies at the intersection of SemCom, embodied AI, and physical AI-enabled robotics. A video of the demo is available at https://tinyurl.com/Tos09
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。