无需微调即可自定义视觉特效,实现高质量可控生成。
EffectMaker: Unifying Reasoning and Generation for Customized Visual Effect Creation
- 用大模型理解特效语义并推理适配目标主体
- 通过参考视频捕捉细节,保持特效一致性
- 适合影视创作、设计人员快速生成定制特效
视觉效果(VFX)对提升视频表达力与创造力至关重要,但高质量特效制作通常依赖专家知识和昂贵流程。现有AIGC系统在特效生成上面临特效数据稀缺、超自然或风格化效果难建模等问题,且常需针对每种特效微调,严重限制可扩展性与泛化能力。本文提出EffectMaker,一种统一的推理-生成框架,支持基于参考的特效定制。该框架利用多模态大语言模型解析高层特效语义并推理其在目标主体上的适配方式,同时采用扩散变压器通过上下文学习从参考视频中捕捉精细视觉线索。两者构成语义-视觉双路径引导机制,实现无需每特效微调的精准、可控、一致合成。此外,构建了包含130,000个视频、覆盖3,000类VFX的EffectData数据集,显著提升泛化与可扩展性。实验表明,EffectMaker在视觉质量与特效一致性上优于当前最优基线,为定制化VFX生成提供高效灵活的新范式。
原文摘要 · Abstract (English)
Visual effects (VFX) are essential for enhancing the expressiveness and creativity of video content, yet producing high-quality effects typically requires expert knowledge and costly production pipelines. Existing AIGC systems face significant challenges in VFX generation due to the scarcity of effect-specific data and the inherent difficulty of modeling supernatural or stylized effects. Moreover, these approaches often require per-effect fine-tuning, which severely limits their scalability and generalization to novel VFX. In this work, we present EffectMaker, a unified reasoning-generation framework that enables reference-based VFX customization. EffectMaker employs a multimodal large language model to interpret high-level effect semantics and reason about how they should adapt to a target subject, while a diffusion transformer leverages in-context learning to capture fine-grained visual cues from reference videos. These two components form a semantic-visual dual-path guidance mechanism that enables accurate, controllable, and effect-consistent synthesis without per-effect fine-tuning. Furthermore, we construct EffectData, the largest high-quality synthetic dataset containing 130k videos across 3k VFX categories, to improve generalization and scalability. Experiments show that EffectMaker achieves superior visual quality and effect consistency over state-of-the-art baselines, offering a scalable and flexible paradigm for customized VFX generation. Project page: https://effectmaker.github.io
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。