用多模态大模型解析汽车软件设计图，提升工程理解与效率
Multi-modal Summarization in Model-Based Engineering: Automotive Software Development Case Study
- 结合UML/EMF图的文本与结构信息，用多模态大模型自动提取关键内容
- 在汽车软件开发中验证了模型对关系与功能识别的有效性，提升理解效率
- 适合从事模型驱动工程、系统设计的开发者和研究人员参考
多模态摘要融合多种数据模态信息，为理解复杂流程中的信息提供了有前景的解决方案。然而，在模型驱动工程（MBE）领域，该技术尚未受到足够关注。MBE已成为复杂系统设计与开发的核心，利用形式化模型提升全生命周期的理解、验证与自动化水平。UML与EMF图包含大量多模态信息及复杂关联数据。本研究探索多模态大语言模型在MBE领域的应用，评估其对嵌入于UML与EMF图中的关系、特征与功能的理解与识别能力。我们旨在展示多模态摘要在提升MBE实践中的生产力与准确性的变革潜力及其局限性。所提方法在汽车软件开发场景中进行评估，并对比了多项先进模型。
原文摘要 · Abstract (English)
Multimodal summarization integrating information from diverse data modalities presents a promising solution to aid the understanding of information within various processes. However, the application and advantages of multimodal summarization have not received much attention in model-based engineering (MBE), where it has become a cornerstone in the design and development of complex systems, leveraging formal models to improve understanding, validation and automation throughout the engineering lifecycle. UML and EMF diagrams in model-based engineering contain a large amount of multimodal information and intricate relational data. Hence, our study explores the application of multimodal large language models within the domain of model-based engineering to evaluate their capacity for understanding and identifying relationships, features, and functionalities embedded in UML and EMF diagrams. We aim to demonstrate the transformative potential benefits and limitations of multimodal summarization in improving productivity and accuracy in MBE practices. The proposed approach is evaluated within the context of automotive software development, while many promising state-of-art models were taken into account.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。