用最小代价多割法自动识别图像中的视觉主题，提升分析效率。
I Spy With My Little Eye: A Minimum Cost Multicut Investigation of Dataset Frames
- 将图像聚类建模为最小代价多割问题，基于成对相似性推断最优分组。
- DINOv2在宏观主题识别上表现更优，ConvNeXt V2能捕捉细粒度差异如演讲与抗议。
- 适用于需要高效自动化视觉主题分析的研究者，尤其适合气候传播研究。
视觉框架分析是社会科学中识别话语核心主题的重要方法。为减少人工标注负担，图像聚类可显著加速这一过程。本文将聚类任务建模为最小代价多割问题（Minimum Cost Multicut Problem, MP），其解能仅基于图像对间属于同一簇的局部概率，最大化后验概率。我们评估了多种嵌入空间在检测视觉框架上的效果，发现DINOv2在宏观框架识别上表现优异，而ConvNeXt V2则产生更多包含细微差别的簇（如演讲与抗议）。结合最优聚类结果与嵌入空间差异的洞察，本研究推动了自动化视觉框架检测的发展。代码已开源：https://github.com/KathPra/MP4VisualFrameDetection。
原文摘要 · Abstract (English)
Visual framing analysis is a key method in social sciences for determining common themes and concepts in a given discourse. To reduce manual effort, image clustering can significantly speed up the annotation process. In this work, we phrase the clustering task as a Minimum Cost Multicut Problem [MP]. Solutions to the MP have been shown to provide clusterings that maximize the posterior probability, solely from provided local, pairwise probabilities of two images belonging to the same cluster. We discuss the efficacy of numerous embedding spaces to detect visual frames and show its superiority over other clustering methods. To this end, we employ the climate change dataset \textit{ClimateTV} which contains images commonly used for visual frame analysis. For broad visual frames, DINOv2 is a suitable embedding space, while ConvNeXt V2 returns a larger number of clusters which contain fine-grain differences, i.e. speech and protest. Our insights into embedding space differences in combination with the optimal clustering - by definition - advances automated visual frame detection. Our code can be found at https://github.com/KathPra/MP4VisualFrameDetection.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。