用扩散模型让合成数据直接搞定床上人体建模，省下真实数据采集
DiSRT-In-Bed: Diffusion-Based Sim-to-Real Transfer Framework for In-Bed Human Mesh Recovery
- 用扩散模型从合成深度图中学习真实场景的人体网格
- 在无真实数据时仍保持高精度，跨不同被褥和环境泛化能力强
- 适合医疗健康领域做睡眠监测、压疮预防等实时人体姿态分析
床上人体网格恢复对睡眠模式监测、康复支持和压疮预防等医疗应用至关重要。然而，由于隐私和成本限制，该领域难以获取大规模真实视觉数据，制约了深度学习模型的训练与部署。现有方法高度依赖真实数据，泛化能力受限于不同覆盖物和环境变化。为此，我们提出一种基于扩散模型的仿真到真实迁移框架，利用大规模合成数据结合少量甚至无需真实样本，实现从俯视深度图像中恢复人体网格。所提扩散模型有效弥合合成与真实数据之间的域差距，在多种医疗场景下显著提升模型鲁棒性与适应性。大量实验与消融研究验证了框架的有效性。
原文摘要 · Abstract (English)
In-bed human mesh recovery can be crucial and enabling for several healthcare applications, including sleep pattern monitoring, rehabilitation support, and pressure ulcer prevention. However, it is difficult to collect large real-world visual datasets in this domain, in part due to privacy and expense constraints, which in turn presents significant challenges for training and deploying deep learning models. Existing in-bed human mesh estimation methods often rely heavily on real-world data, limiting their ability to generalize across different in-bed scenarios, such as varying coverings and environmental settings. To address this, we propose a Sim-to-Real Transfer Framework for in-bed human mesh recovery from overhead depth images, which leverages large-scale synthetic data alongside limited or no real-world samples. We introduce a diffusion model that bridges the gap between synthetic data and real data to support generalization in real-world in-bed pose and body inference scenarios. Extensive experiments and ablation studies validate the effectiveness of our framework, demonstrating significant improvements in robustness and adaptability across diverse healthcare scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。