arXiv:2605.24176cs.CV2026-05

Loki用解耦表示实现高效人脸动画,无需跨身份训练数据。

Loki: Representation over Architecture for Diffusion-Based Portrait Animation

论文配图:Loki: Representation over Architecture for Diffusion-Based Portrait Animation
图 1 · 摘自论文原文
  • 用身份正交的参数化人脸模型分离表情与姿态,避免RGB空间混杂
  • 仅需43%参数、1496倍少样本训练,仍保持高质量动画生成
  • 支持零样本跨身份重演,适合低资源场景下的实时动画应用

人脸动画将驱动视频中的面部表情和头部姿态迁移至单张参考图像,同时保留参考身份。现有扩散模型通过堆叠表情、姿态、身份模块实现,导致可训练参数多、依赖专有数据集,并存在各控制轴间的残余耦合。根源在于从RGB图像中学习表情与姿态,使身份、姿态、表情在表征上不可分。Loki在条件路径上跳出RGB,使用参数轴身份正交的人脸模型编码驱动表情与姿态,将其栅格化为扩散主干可原生接收的空间图;身份则通过轻量级键值注入,经扩散主干预训练特征传递。因参数表示显式解耦身份与表情/姿态,跨身份重演仅需推理时替换系数,无需跨身份训练数据。Loki推理参数量比领先基线减少约43%,训练视频样本减少1496倍。我们定义两个直接衡量生成头姿轨迹与表情是否跟随驱动者的指标,Loki在两项指标上均领先或并列领先。

原文摘要 · Abstract (English)

Portrait animation transfers a driver clip's facial expression and head pose onto a single reference image while preserving the reference's identity. State-of-the-art diffusion systems address this by stacking trained modules for expression, pose, and identity in turn, paying for it in trainable parameters, proprietary corpora, and residual entanglement between the very axes the system is meant to control independently. This complexity compensates for an upstream choice -- learning facial expression and head pose from RGB, a representation in which identity, pose, and expression are inseparable without being learned apart. Loki steps out of RGB on the conditioning path. Driver expression and head pose are encoded by a face model whose parameter axes are identity-orthogonal by construction, then rasterised into a spatial map that the diffusion backbone consumes natively. Identity is routed separately through the diffusion backbone's own pretrained features via lightweight key-value injection. Because the parametric representation factorises identity from expression and pose, cross ID reenactment reduces to a coefficient substitution at inference, requiring no cross ID training data. Loki requires ~43% fewer inference parameters than leading diffusion baselines and trained on 1496x less video samples. We define two metrics that directly measure whether the generated head pose trajectory and facial expression followed the driver's -- the questions portrait animation actually asks; Loki leads or co-leads on both.

人脸动画扩散模型解耦表示

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。