从单张图片精准预测脚与地面接触,无视鞋款差异和地面单调性。
Shoe Style-Invariant and Ground-Aware Learning for Dense Foot Contact Estimation
- 通过对抗训练让模型忽略鞋款外观差异,提取通用接触特征。
- 设计地面上下文特征提取器,增强对单调地面的感知能力。
- 适合动作分析、虚拟试穿等需要精细足部交互的应用场景。
足部接触在人类与环境交互中至关重要,但现有方法多依赖零速度假设并仅关注关节级接触,难以捕捉足部与地面的详细互动。从单张RGB图像进行密集足部接触估计仍属未充分探索领域。主要挑战在于:鞋款外观差异大导致模型泛化困难;地面常呈单调纹理,难以提取有效特征。为此,本文提出FECO框架,实现鞋款无关与地面感知的密集足部接触估计。通过鞋款风格对抗训练,强制模型学习与鞋款无关的接触特征;引入基于空间上下文的地面特征提取器,有效利用地面信息。实验表明,该方法在不同鞋款和复杂地面条件下均能实现鲁棒的接触估计。代码已开源。
原文摘要 · Abstract (English)
Foot contact plays a critical role in human interaction with the world, and thus exploring foot contact can advance our understanding of human movement and physical interaction. Despite its importance, existing methods often approximate foot contact using a zero-velocity constraint and focus on joint-level contact, failing to capture the detailed interaction between the foot and the world. Dense estimation of foot contact is crucial for accurately modeling this interaction, yet predicting dense foot contact from a single RGB image remains largely underexplored. There are two main challenges for learning dense foot contact estimation. First, shoes exhibit highly diverse appearances, making it difficult for models to generalize across different styles. Second, ground often has a monotonous appearance, making it difficult to extract informative features. To tackle these issues, we present a FEet COntact estimation (FECO) framework that learns dense foot contact with shoe style-invariant and ground-aware learning. To overcome the challenge of shoe appearance diversity, our approach incorporates shoe style adversarial training that enforces shoe style-invariant features for contact estimation. To effectively utilize ground information, we introduce a ground feature extractor that captures ground properties based on spatial context. As a result, our proposed method achieves robust foot contact estimation regardless of shoe appearance and effectively leverages ground information. The codes are available at https://github.com/dqj5182/FECO_RELEASE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。