用扩散模型生成人与商品互动的带货视频,效果更自然精准。
AnchorCrafter: Animate Cyber-Anchors Selling Your Products via Human-Object Interacting Video Generation
- 通过视角感知和运动注入,让虚拟人物与商品自然互动。
- 物体外观保留提升7.5%,定位准确率翻倍,动作连贯性更强。
- 适合电商、广告创意人员快速生成高质量推广视频。
锚点风格的商品推广视频生成在电子商务、广告和用户参与中具有广阔前景。尽管姿态引导的人体视频生成已取得进展,但创作商品推广视频仍面临挑战。我们识别出将人-物交互(HOI)融入姿态引导人体视频生成是核心问题。为此,提出AnchorCrafter,一种基于扩散模型的新系统,可生成包含目标人物和定制物品的2D视频,实现高视觉保真度和可控交互。具体提出两项关键创新:HOI外观感知,从任意多视角增强物体外观识别并解耦人与物外观;HOI运动注入,克服物体轨迹条件设置与遮挡管理难题,实现复杂人物交互。大量实验表明,本系统在物体外观保留上提升7.5%,定位准确率翻倍,且优于现有最优方法,在保持人体动作一致性与生成高质量视频方面表现更佳。项目页面含数据、代码及HuggingFace演示:https://github.com/cangcz/AnchorCrafter。
原文摘要 · Abstract (English)
The generation of anchor-style product promotion videos presents promising opportunities in e-commerce, advertising, and consumer engagement. Despite advancements in pose-guided human video generation, creating product promotion videos remains challenging. In addressing this challenge, we identify the integration of human-object interactions (HOI) into pose-guided human video generation as a core issue. To this end, we introduce AnchorCrafter, a novel diffusion-based system designed to generate 2D videos featuring a target human and a customized object, achieving high visual fidelity and controllable interactions. Specifically, we propose two key innovations: the HOI-appearance perception, which enhances object appearance recognition from arbitrary multi-view perspectives and disentangles object and human appearance, and the HOI-motion injection, which enables complex human-object interactions by overcoming challenges in object trajectory conditioning and inter-occlusion management. Extensive experiments show that our system improves object appearance preservation by 7.5\% and doubles the object localization accuracy compared to existing state-of-the-art approaches. It also outperforms existing approaches in maintaining human motion consistency and high-quality video generation. Project page including data, code, and Huggingface demo: https://github.com/cangcz/AnchorCrafter.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。