arXiv:2503.05398cs.ROcs.CV2025-03中稿 · IJRR被引 3

用3D高斯点云实现机器人高保真自建模,支持纹理与运动联合学习。

Learning High-Fidelity Robot Self-Model with Articulated 3D Gaussian Splatting

  • 以3D高斯点云表示机器人形态与纹理,通过神经椭球骨骼建模关节运动。
  • 仅需关节角度+多视角图像(无深度)训练,即可实现跨视角图像渲染。
  • 模型可直接用于运动规划与逆运动学,适合自主机器人系统构建。

自建模使机器人能基于自动采集的数据,在无需人工干预和先验知识的前提下,构建任务无关的形态与运动学模型,从而提升机器智能。近期研究显示数据驱动技术在机器人建模中具有潜力,但现有方法或建模质量低,或数据采集成本过高。除形态与运动学外,纹理也是机器人的重要组成部分,却难以建模且未被充分探索。本文提出一种高质量、带纹理感知、逐链接的机器人自建模方法。采用三维(3D)高斯表示机器人的静态形态与纹理,并聚类3D高斯构建神经椭球骨骼,其形变由运动学神经网络生成的变换矩阵控制。3D高斯与运动学神经网络通过包含关节角度、相机参数和多视图图像(无深度信息)的数据对进行训练。通过输入关节角度,可利用已训练模型在链接级别描述对应形态、运动学与纹理,并借助3D高斯泼溅从不同视角渲染机器人图像。此外,我们证明该模型可用于运动规划与逆运动学等下游任务。

原文摘要 · Abstract (English)

Self-modeling enables robots to build task-agnostic models of their morphology and kinematics based on data that can be automatically collected, with minimal human intervention and prior information, thereby enhancing machine intelligence. Recent research has highlighted the potential of data-driven technology in modeling the morphology and kinematics of robots. However, existing self-modeling methods suffer from either low modeling quality or excessive data acquisition costs. Beyond morphology and kinematics, texture is also a crucial component of robots, which is challenging to model and remains unexplored. In this work, a high-quality, texture-aware, and link-level method is proposed for robot self-modeling. We utilize three-dimensional (3D) Gaussians to represent the static morphology and texture of robots, and cluster the 3D Gaussians to construct neural ellipsoid bones, whose deformations are controlled by the transformation matrices generated by a kinematic neural network. The 3D Gaussians and kinematic neural network are trained using data pairs composed of joint angles, camera parameters and multi-view images without depth information. By feeding the kinematic neural network with joint angles, we can utilize the well-trained model to describe the corresponding morphology, kinematics and texture of robots at the link level, and render robot images from different perspectives with the aid of 3D Gaussian splatting. Furthermore, we demonstrate that the established model can be exploited to perform downstream tasks such as motion planning and inverse kinematics.

机器人建模3D高斯自建模运动学

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。