无需训练即可生成特定视觉效果的视频模型，3.3秒完成参数合成。
Prompt2Effect: Training-Free Image-to-Video Model Specialization via LoRA Generation
- 基于基模型权重设计超网络，直接生成专属LoRA参数。
- 合成效果与微调相当，视频质量更优，计算成本降低94%以上。
- 适合需要快速试错特效的创作者和交互式视频生成场景。
尽管个性化图像到视频扩散模型以生成特定视觉效果的需求日益增长，但现有方法需为每种效果单独训练低秩适配（LoRA）模块，带来高昂的数据准备与迭代优化成本，阻碍了交互式控制。本文提出Prompt2Effect，一种权重驱动的超网络，通过单次前向传播直接合成特定效果的LoRA权重，实现按效果训练的开销分摊。不同于仅依赖语义回归的早期超网络，Prompt2Effect显式依赖冻结的基模型权重，使预测结果锚定于各层结构几何。此外，我们引入SVD规范化的参数化方式，解决因子分解歧义性并稳定大规模合成。大量实验表明，Prompt2Effect在视频质量和效果对齐方面达到或优于传统LoRA微调，将计算成本从56个GPU小时降至3.3秒的超网络推理。当作为后续微调的初始化时，其预测权重可进一步提升最终性能，并加速优化约10倍。
原文摘要 · Abstract (English)
While personalizing Image-to-Video (I2V) diffusion models with specific visual effects is increasingly demanded for high-end generation, current practice requires training a separate Low-Rank Adaptation (LoRA) module for each effect, incurring substantial data curation and iterative optimization costs that hinder interactive control. We present Prompt2Effect, a weight-driven hypernetwork that amortizes per-effect training by directly synthesizing effect-specific LoRA weights in a single forward pass. Unlike prior hypernetworks that regress adapter weights purely from semantics, Prompt2Effect is explicitly conditioned on the frozen base model weights, grounding prediction in the structural geometry of each layer. Furthermore, instead of predicting raw LoRA matrices, we introduce an SVD-canonicalized parameterization that resolves factorization ambiguity and stabilizes large-scale synthesis. Extensive experiments demonstrate that Prompt2Effect achieves on-par or superior video quality and effect alignment compared to conventional LoRA fine-tuning, while reducing the computational cost from 56 GPU training hours to 3.3 seconds of hypernetwork inference. When used as initialization for subsequent fine-tuning, our predicted weights further improve final performance and accelerate optimization by approximately 10x.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。