FAME通过注意力调制消除视频编辑中的职业性别偏见。
FAME: Fairness-aware Attention-modulated Video Editing
- 用公平性嵌入软注入文本编码器,缓解偏见
- 在自注意力和跨注意力中引入公平性调制,保持时序一致性
- 在新基准FairVE上优于现有方法,兼顾公平与语义保真
训练无关的视频编辑模型在处理职业相关提示时往往沿用性别刻板印象。本文提出FAME(Fairness-aware Attention-modulated Video Editing),在保持提示对齐和时序一致性的前提下,缓解职业相关的性别偏见。FAME通过从现有少数群体表征中提取公平性嵌入,并以软注入方式引入文本编码器。同时,FAME将公平性调制集成到时序自注意力和提示-区域交叉注意力中,以减轻直接引入公平性线索导致的动作失真和时序不一致。对于自注意力,FAME采用区域约束注意力掩码结合时间衰减权重,增强区域内一致性并抑制无关区域间交互;对于交叉注意力,通过引入基于去偏提示嵌入的敏感相似性掩码,重新加权令牌到区域的匹配分数。这些调制机制使敏感语义始终关联正确视觉区域,防止帧间时序漂移。在新的视频编辑公平性基准FairVE上的大量实验表明,FAME实现了更强的公平性对齐和语义保真度,超越现有视频编辑基线。
原文摘要 · Abstract (English)
Training-free video editing (VE) models tend to fall back on gender stereotypes when rendering profession-related prompts. We propose \textbf{FAME} for \textit{Fairness-aware Attention-modulated Video Editing} that mitigates profession-related gender biases while preserving prompt alignment and temporal consistency for coherent VE. We derive fairness embeddings from existing minority representations by softly injecting debiasing tokens into the text encoder. Simultaneously, FAME integrates fairness modulation into both temporal self attention and prompt-to-region cross attention to mitigate the motion corruption and temporal inconsistency caused by directly introducing fairness cues. For temporal self attention, FAME introduces a region constrained attention mask combined with time decay weighting, which enhances intra-region coherence while suppressing irrelevant inter-region interactions. For cross attention, it reweights tokens to region matching scores by incorporating fairness sensitive similarity masks derived from debiasing prompt embeddings. Together, these modulations keep fairness-sensitive semantics tied to the right visual regions and prevent temporal drift across frames. Extensive experiments on new VE fairness-oriented benchmark \textit{FairVE} demonstrate that FAME achieves stronger fairness alignment and semantic fidelity, surpassing existing VE baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。