arXiv:2504.11459cs.AIcs.CL2025-04

将文本数据转化为多模态表达,构建可复用的知识模型。

From Conceptual Data Models to Multimodal Representation

  • 用概念网络建模语义关系,支持跨模态内容组织
  • 通过动态模型实现音视频数据的多形式重用与发布
  • 适合信息设计、数字人文及跨媒体内容开发者

本文探讨信息设计中的两大实践:定义文本数据集的意义及其视觉或多媒体表达。基于结构符号学与语言学传统,强调领域语义内容与表达模态之间的根本区别。提出通过概念网络或图结构进行语义建模,以刻画概念间的关系、使用情境与目标,克服动态适应性模型在术语库与互操作本体集成中的挑战。在实际应用中,依托OKAPI系统分析、发布与再利用音视频资料,探索视觉叙事与文档重构等方法,推动数字内容在多场景下的协作化智能传播,增强系统的互操作性与灵活性。相关内容于2018年6月21日在波尔多蒙桑日大学(MICA实验室)举办的“信息设计符号学”研讨会上展示。

原文摘要 · Abstract (English)

1) Introduction and Conceptual Framework: This document explores the concept of information design by dividing it into two major practices: defining the meaning of a corpus of textual data and its visual or multimodal representation. It draws on expertise in enriching textual corpora, particularly audiovisual ones, and transforming them into multiple narrative formats. The text highlights a crucial distinction between the semantic content of a domain and the modalities of its graphic expression, illustrating this approach with concepts rooted in structural semiotics and linguistics traditions. 2) Modeling and Conceptual Design: The article emphasizes the importance of semantic modeling, often achieved through conceptual networks or graphs. These tools enable the structuring of knowledge within a domain by accounting for relationships between concepts, contexts of use, and specific objectives. Stockinger also highlights the constraints and challenges involved in creating dynamic and adaptable models, integrating elements such as thesauri or interoperable ontologies to facilitate the analysis and publication of complex corpora. 3) Applications and Multimodal Visualization: The text concludes by examining the practical application of these models in work environments like OKAPI, developed to analyze, publish, and reuse audiovisual data. It also discusses innovative approaches such as visual storytelling and document reengineering, which involve transforming existing content into new resources tailored to various contexts. These methods emphasize interoperability, flexibility, and the intelligence of communication systems, paving the way for richer and more collaborative use of digital data. The content of this document was presented during the "Semiotics of Information Design" Day organized by Anne Beyaert-Geslin of the University of Bordeaux Montaigne (MICA laboratory) on June 21, 2018, in Bordeaux.

信息设计多模态语义建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。