用分层扩散模型精准生成弯曲结构图像,支持多领域控制。
CSGen: A Multi-Domain Curvilinear Structure Generation Model via Hierarchical Multimodal Diffusion

- 分阶段注入信号,分离拓扑与视觉信息,防止语义漂移。
- 在5个领域24000+样本上训练,结构精度显著提升。
- 专为细长稀疏结构设计损失重加权,适合医学/遥感等场景。
弯曲结构分析是多媒体领域的基础任务,但精确生成带可控弯曲结构的图像仍具挑战。为此,我们提出CSGen,一种基于分层多模态扩散的统一生成模型,可精准对齐多种控制条件生成高保真图像。核心创新包括:1)构建包含5个领域、7类标注、超24,000样本的多领域多模态数据集;2)提出分阶段信号注入的层次化渐进控制策略,解耦拓扑线索与视觉上下文,缓解语义漂移并保障稀疏结构的拓扑完整性;3)设计稀疏感知损失重加权机制,增强优化过程中对细长脆弱结构的关注。大量实验表明,CSGen在结构准确性与视觉真实感方面表现优异,显著提升下游分割性能,且对多样化提示保持鲁棒性。结果验证了其在多元多媒体应用中分析复杂弯曲结构的可扩展性与数据驱动范式。代码与数据集见https://github.com/ShanZard/CSGen。
原文摘要 · Abstract (English)
Curvilinear structure analysis is an important and fundamental task in multimedia. However, the controllable generation of images with precise curvilinear structure objects remains an open challenge. To address this, we propose CSGen, a hierarchical multimodal diffusion model that synthesizes high-fidelity images precisely aligned with multiple control conditions. The CSGen is built upon three key innovations: 1) We construct a multi-domain and multimodal dataset, including over 24K samples from 5 domains and 7 different types of annotations, to train the unified generation model. 2) We propose a novel hierarchical progressive control strategy that decouples topology clues from visual context by a phased signal injection, mitigating semantic drift while ensuring the topological integrity of sparse structures. 3) We design a sparsity-aware loss re-weighting mechanism to address the extreme sparsity of curvilinear structures, significantly enhancing the attention on thin and fragile structures during optimization. Extensive experiments demonstrate that CSGen generates images with superior structure accuracy and visual realism, significantly improving downstream segmentation performance while maintaining robustness across diverse prompts. Our results confirm CSGen as a scalable, data-centric paradigm for the analysis of complex curvilinear structures in diverse multimedia applications. Code and dataset are available at https://github.com/ShanZard/CSGen.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。