用双臂机器人转动物体，360度扫描生成高精度数字孪生模型。
Omni-Scan: Creating Visually-Accurate Digital Twin Object Models Using a Bimanual Robot with Handover and Gaussian Splat Merging
- 双机械臂交替抓握，暴露被遮挡表面，实现全景拍摄。
- 在12类工业与家用物品上检测缺陷，平均准确率达83%。
- 适合需要高精度三维建模的制造、质检与虚拟仿真场景。
3D Gaussian Splats（3DGS）是从多视角图像中生成的3D物体模型，可用于仿真、虚拟现实、营销、机器人策略微调及零件检测等。传统3D扫描常依赖多相机阵列、精密激光扫描仪或腕装摄像头，受限于工作空间。本文提出Omni-Scan，一种利用双臂机器人抓取物体并相对于固定相机旋转，通过第二机械臂重新抓握以暴露被遮挡表面的扫描流程。结合DepthAny-thing、Segment Anything和RAFT光流模型，实现对机器人抓握物的识别与背景/夹具去除。改进3DGS训练流程，支持包含夹具遮挡的拼接数据集，生成360度全向物体模型。应用于零件缺陷检测，在12种不同工业与家用物体上实现平均83%的检测准确率。交互式3DGS模型视频见https://berkeleyautomation.github.io/omni-scan/
原文摘要 · Abstract (English)
3D Gaussian Splats (3DGSs) are 3D object models derived from multi-view images. Such "digital twins" are useful for simulations, virtual reality, marketing, robot policy fine-tuning, and part inspection. 3D object scanning usually requires multi-camera arrays, precise laser scanners, or robot wrist-mounted cameras, which have restricted workspaces. We propose Omni-Scan, a pipeline for producing high-quality 3D Gaussian Splat models using a bi-manual robot that grasps an object with one gripper and rotates the object with respect to a stationary camera. The object is then re-grasped by a second gripper to expose surfaces that were occluded by the first gripper. We present the Omni-Scan robot pipeline using DepthAny-thing, Segment Anything, as well as RAFT optical flow models to identify and isolate objects held by a robot gripper while removing the gripper and the background. We then modify the 3DGS training pipeline to support concatenated datasets with gripper occlusion, producing an omni-directional (360 degree view) model of the object. We apply Omni-Scan to part defect inspection, finding that it can identify visual or geometric defects in 12 different industrial and household objects with an average accuracy of 83%. Interactive videos of Omni-Scan 3DGS models can be found at https://berkeleyautomation.github.io/omni-scan/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。