用多模态大模型从UML图自动生成代码，提升开发效率
Unified Modeling Language Code Generation from Diagram Images Using Multimodal Large Language Models
- 基于多模态大模型，将UML图转为可执行代码
- 最佳模型在序列图上达到0.779的BLEU和0.942的SSIM
- 适合需要自动化重构或现代化遗留系统的开发者
统一建模语言（UML）是广泛用于软件系统设计建模与文档化的标准化视觉语言。尽管已有工具可从UML代码生成UML图，但从图像形式的UML图生成可执行代码仍具挑战性。本文提出一种新方法，利用大型多模态语言模型自动完成该任务。我们构建了合成的UML活动图和序列图数据集用于训练与测试。通过对比标准微调与LoRA技术优化基础模型，实验评估了不同模型规模与训练策略下的代码生成准确率。结果表明，领域适配的多模态大模型在自动化生成UML代码方面表现优异，在最优模型下，序列图的BLEU得分为0.779，SSIM得分为0.942。该方法有助于实现遗留系统的现代化，并降低软件开发流程中的手动工作量。
原文摘要 · Abstract (English)
The Unified Modeling Language is a standardized visual language widely used for modeling and documenting the design of software systems. Although many tools generate UML diagrams from UML code, generating executable UML code from image-based UML diagrams remains challenging. This paper proposes a new approach to generate UML code using a large multimodal language model automatically. Synthetic UML activity and sequence diagram datasets were created to train and test the model. We compared standard fine-tuning with LoRA techniques to optimize base models. The experiments measured code generation accuracy across different model sizes and training strategies. These results demonstrated that domain-adapted MM-LLMs perform for UML code generation automation, whereby, at the best model, it achieved BLEU and SSIM scores of 0.779 and 0.942 on sequence diagrams. This will enable the modernization of legacy systems and decrease the manual effort in software development workflows.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。