用扩散模型生成符合文本描述的触觉振动,提升真实感与设计效率。
HapticLDM: A Diffusion Model for Text-to-Vibrotactile Generation

- 基于潜空间扩散模型,全局去噪确保振动时序稳定
- 在30人用户研究中显著提升振动真实感与语义对齐度
- 适合元宇宙、游戏等需要精细触觉反馈的场景
文本到振动生成将自然语言转换为触觉反馈,帮助振动设计师更高效地生成情境适配的振动效果,在元宇宙、游戏和影视等领域具有广阔应用前景。该领域核心挑战在于如何根据文本语义生成准确、一致且完整的振动。近期自回归方法(如HapticGen)受限于序列建模本质和数据约束,难以捕捉全局依赖关系。本文提出HapticLDM,首个基于潜空间扩散模型的文本到振动生成模型。首先,设计强调动态特性的文本处理策略,构建高质量数据对以实现细粒度动态建模;其次,引入全局去噪机制,调控时序包络的连贯性与稳定性。我们进行了广泛评估,包括与当前最优基线的A/B测试及30名参与者的用户研究。结果表明,本模型显著提升振动的真实感与语义对齐度。定性反馈显示,HapticLDM简化了触觉设计流程,可生成多样、细微且物理精确的振动。
原文摘要 · Abstract (English)
Text-to-vibration generation converts natural language into haptic feedback, enabling vibration-effect designers to get scenarios-fitted vibrations more efficiently, which shows great potentials in application fields such as metaverse, games, and film to enrich the user experience in interactive scenarios. The core challenge in this field is how to generate accurate, consistent, and complete vibrations according to textual semantics. Very recent autoregressive (AR) approaches (e.g., HapticGen) exhibit limited capacity in fully capturing global dependencies, owing to the inherent sequential nature of their modeling and prevailing data constraints. In this paper, we proposed HapticLDM, the first text-to-vibration generative model built upon Latent Diffusion Models (LDMs). Firstly, with respect to the data, we introduced a text-processing strategy that emphasizes dynamic characteristics to curate high-quality data pairs for fine-grained dynamic modeling. Secondly, HapticLDM incorporates a global denoising mechanism that regulates coherent and stable variations in the temporal envelope. Furthermore, we conduct extensive evaluations, including A/B testing against the state-of-the-art baseline and a user study involving 30 participants. The results demonstrate that our model enhances realism and semantic alignment. Qualitative feedback further indicates that HapticLDM simplifies the haptic design workflow while generating diverse, subtle, and physically precise vibrations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。