用分层语义场实现物体姿态感知的机器人操作，提升精准控制能力。
HeRO: Hierarchical 3D Semantic Representation for Pose-aware Object Manipulation
- 通过扩散模型融合几何与细粒度语义，生成空间一致的特征表示。
- 在6个姿态敏感任务上平均提升6.5%，放置双鞋任务成功率提高12.3%。
- 适合需要精细部件识别的机器人抓取与操纵场景。
机器人操作的模仿学习已从2D图像策略发展到显式编码几何信息的3D表示。然而，纯几何策略往往缺乏显式的部件级语义，而这对姿态感知操作（如区分鞋子的前掌与后跟）至关重要。本文提出HeRO，一种基于扩散模型的策略，通过分层语义场耦合几何与语义信息。HeRO采用密集语义提升，将DINOv2的判别性、几何敏感特征与Stable Diffusion的平滑全局对应关系融合，生成兼具细粒度与空间一致性的密集特征。这些特征被处理并划分为全局场与一组局部场。分层条件模块使用置换不变网络架构，对全局与局部场进行条件化，避免顺序敏感偏差，从而生成连贯的姿态感知控制策略。在多项测试中，HeRO达到新基准，放置双鞋任务成功率提升12.3%，六个挑战性姿态感知任务平均提升6.5%。代码已公开于https://github.com/Chongyang-99/HeRO。
原文摘要 · Abstract (English)
Imitation learning for robotic manipulation has progressed from 2D image policies to 3D representations that explicitly encode geometry. Yet purely geometric policies often lack explicit part-level semantics, which are critical for pose-aware manipulation (e.g., distinguishing a shoe's toe from heel). In this paper, we present HeRO, a diffusion-based policy that couples geometry and semantics via hierarchical semantic fields. HeRO employs dense semantics lifting to fuse discriminative, geometry-sensitive features from DINOv2 with the smooth, globally coherent correspondences from Stable Diffusion, yielding dense features that are both fine-grained and spatially consistent. These features are processed and partitioned to construct a global field and a set of local fields. A hierarchical conditioning module conditions the generative denoiser on global and local fields using permutation-invariant network architecture, thereby avoiding order-sensitive bias and producing a coherent control policy for pose-aware manipulation. In various tests, HeRO establishes a new state-of-the-art, improving success on Place Dual Shoes by 12.3% and averaging 6.5% gains across six challenging pose-aware tasks. Code is available at https://github.com/Chongyang-99/HeRO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。