arXiv:2510.14874cs.CV2025-10被引 11

用文字控制生成多样且真实的自由形态手物交互。

TOUCH: Text-guided Controllable Generation of Free-Form Hand-Object Interactions

  • 分三阶段的扩散模型,通过接触建模实现细粒度语义控制。
  • 在4.4千个独特交互上验证,支持92种意图与610类物体。
  • 适合做复杂人机交互、虚拟现实和具身智能的研究者。

手物交互(HOI)是人类表达意图的基础。现有研究多局限于固定抓握模式,控制依赖物理先验或通用指令,即使使用复杂语言也难以突破稳定抓握的强归纳偏置,无法捕捉日常交互多样性。为此,本文提出自由形态手物交互生成,旨在基于细粒度意图生成可控、多样且物理合理的非抓握交互,如推、戳、旋转等。为此构建了野拍数据集WildO2,包含4.4千个独特交互,覆盖92种意图和610类物体,附带详细语义标注。在此基础上提出TOUCH框架,采用多层级扩散模型,通过显式接触建模进行条件控制,并结合接触一致性与物理约束提升真实性。大量实验表明,该方法能生成代表日常活动的可控、多样且真实的手部交互。

原文摘要 · Abstract (English)

Hand-object interaction (HOI) is fundamental for humans to express intent. Existing HOI generation research is predominantly confined to fixed grasping patterns, where control is tied to physical priors such as force closure or generic intent instructions, even when expressed through elaborate language. Such an overly general conditioning imposes a strong inductive bias for stable grasps, thus failing to capture the diversity of daily HOI. To address these limitations, we introduce Free-Form HOI Generation, which aims to generate controllable, diverse, and physically plausible HOI conditioned on fine-grained intent, extending HOI from grasping to free-form interactions, like pushing, poking, and rotating. To support this task, we construct WildO2, an in-the-wild diverse 3D HOI dataset, which includes diverse HOI derived from internet videos. Specifically, it contains 4.4k unique interactions across 92 intents and 610 object categories, each with detailed semantic annotations. Building on this dataset, we propose TOUCH, a three-stage framework centered on a multi-level diffusion model that facilitates fine-grained semantic control to generate versatile hand poses beyond grasping priors. This process leverages explicit contact modeling for conditioning and is subsequently refined with contact consistency and physical constraints to ensure realism. Comprehensive experiments demonstrate our method's ability to generate controllable, diverse, and physically plausible hand interactions representative of daily activities. The project page is $\href{https://guangyid.github.io/hoi123touch}{here}$.

手物交互扩散模型可控生成3D生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。