arXiv:2501.09884cs.CVcs.IR2025-01中稿 · oral presentation …被引 3

用图像生成历史叙事,让老照片自己讲故事

Semi-Supervised Image-Based Narrative Extraction: A Case Study with Historical Photographic Records

  • 将文本叙事图算法移植到图像数据,用深度学习提取视觉特征
  • 在10张以上图片的长叙事中,效果显著优于随机采样(p<0.05)
  • 适合历史学者、档案馆员分析大规模历史影像资料

本文提出一种半监督方法,通过改进叙事图算法,从历史照片中提取叙事。将原用于文本的无监督方法拓展至图像数据,利用深度学习进行视觉特征提取与相似性计算。实验基于罗杰(ROGER)数据集,包含1928年玻利维亚萨坎巴亚远征中罗伯特·格斯特曼拍摄的照片。将算法生成的视觉叙事与专家构建的5至30张图像的参考时间线进行对比,采用动态时间规整(DTW)算法匹配。专家对代表性案例进行定性评估,结果表明:在10张及以上图片的长叙事中,该方法显著优于随机采样(p<0.05),且生成叙事具备历史准确性和连贯性。本研究为视觉文化遗产的计算分析提供新工具,助力历史学家、档案员及数字人文学者探索大规模图像集合。

原文摘要 · Abstract (English)

This paper presents a semi-supervised approach to extracting narratives from historical photographic records using an adaptation of the narrative maps algorithm. We extend the original unsupervised text-based method to work with image data, leveraging deep learning techniques for visual feature extraction and similarity computation. Our method is applied to the ROGER dataset, a collection of photographs from the 1928 Sacambaya Expedition in Bolivia captured by Robert Gerstmann. We compare our algorithmically extracted visual narratives with expert-curated timelines of varying lengths (5 to 30 images) to evaluate the effectiveness of our approach. In particular, we use the Dynamic Time Warping (DTW) algorithm to match the extracted narratives with the expert-curated baseline. In addition, we asked an expert on the topic to qualitatively evaluate a representative example of the resulting narratives. Our findings show that the narrative maps approach generally outperforms random sampling for longer timelines (10+ images, p < 0.05), with expert evaluation confirming the historical accuracy and coherence of the extracted narratives. This research contributes to the field of computational analysis of visual cultural heritage, offering new tools for historians, archivists, and digital humanities scholars to explore and understand large-scale image collections. The method's ability to generate meaningful narratives from visual data opens up new possibilities for the study and interpretation of historical events through photographic evidence.

图像叙事历史影像半监督学习数字人文

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。