arXiv:2512.15420cs.LG2025-12被引 1

用共享隐空间和双向流实现高效任意模态生成

FlowBind: Efficient Any-to-Any Generation with Bidirectional Flows

  • 通过共享隐空间与模态专属可逆流构建统一生成框架
  • 参数量减少6倍,训练速度提升10倍,质量相当
  • 适合需要灵活跨模态生成且资源有限的场景

任意模态生成旨在实现任意模态子集间的自由转换,支持灵活的跨模态合成。尽管近期取得进展,现有基于流的方法仍面临效率低下问题:需大规模数据集且配对约束严格,建模联合分布计算成本高,并依赖复杂的多阶段训练。本文提出FlowBind,一种高效的任意模态生成框架。其核心思想是学习一个共享隐空间以捕捉跨模态信息,再通过模态特定的可逆流将该隐空间映射至各模态。两个组件在单一流匹配目标下联合优化,推理时可逆流作为编码器与解码器,实现模态间直接转换。通过共享隐空间分解模态交互,FlowBind自然支持任意模态子集训练,在显著降低数据需求与计算成本的同时,保持竞争力的生成质量。在文本、图像、音频上的实验表明,FlowBind在生成质量相当的情况下,参数量最多减少6倍,训练速度提升10倍。

原文摘要 · Abstract (English)

Any-to-any generation seeks to translate between arbitrary subsets of modalities, enabling flexible cross-modal synthesis. Despite recent success, existing flow-based approaches are challenged by their inefficiency, as they require large-scale datasets often with restrictive pairing constraints, incur high computational cost from modeling joint distribution, and rely on complex multi-stage training. We propose FlowBind, an efficient framework for any-to-any generation. Our approach is distinguished by its simplicity: it learns a shared latent space capturing cross-modal information, with modality-specific invertible flows bridging this latent to each modality. Both components are optimized jointly under a single flow-matching objective, and at inference the invertible flows act as encoders and decoders for direct translation across modalities. By factorizing interactions through the shared latent, FlowBind naturally leverages arbitrary subsets of modalities for training, and achieves competitive generation quality while substantially reducing data requirements and computational cost. Experiments on text, image, and audio demonstrate that FlowBind attains comparable quality while requiring up to 6x fewer parameters and training 10x faster than prior methods. The project page with code is available at https://yeonwoo378.github.io/official_flowbind.

跨模态生成可逆流高效训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。