arXiv:2502.16790cs.CL2025-02被引 3

用大模型清理图像描述文本,效果有限但值得探索。

Are Large Language Models Good Data Preprocessors?

  • 用LLaMA、GPT-4等大模型重写BLIP/GIT生成的图文描述。
  • 清理后数据在情感识别任务上性能提升,但多数不显著。
  • 适合研究大模型在复杂数据清洗中的实际作用。

高质量文本训练数据对多模态任务至关重要,但BLIP和GIT等图像描述模型输出常含错误与异常,规则方法难以修复。现有工作多在简单公开数据集上用GPT进行预处理,缺乏对更广泛LLM及更具挑战性数据集的探索。本研究评估了LLaMA 3.1 70B、GPT-4 Turbo和Sonnet 3.5 v2等多款大模型对BLIP/GIT输出的优化能力,并通过下游任务(SemEval 2024 多标签讽刺检测)对比清洗前后数据的效果。实验显示,使用大模型清洗后的数据在部分任务上表现提升,但统计检验表明多数改进不显著。这表明大模型虽具潜力,其有效性受应用场景、任务复杂度及文本噪声水平制约。研究强调需进一步探究大模型在数据预处理中的边界,为集成大模型提供实证依据。

原文摘要 · Abstract (English)

High-quality textual training data is essential for the success of multimodal data processing tasks, yet outputs from image captioning models like BLIP and GIT often contain errors and anomalies that are difficult to rectify using rule-based methods. While recent work addressing this issue has predominantly focused on using GPT models for data preprocessing on relatively simple public datasets, there is a need to explore a broader range of Large Language Models (LLMs) and tackle more challenging and diverse datasets. In this study, we investigate the use of multiple LLMs, including LLaMA 3.1 70B, GPT-4 Turbo, and Sonnet 3.5 v2, to refine and clean the textual outputs of BLIP and GIT. We assess the impact of LLM-assisted data cleaning by comparing downstream-task (SemEval 2024 Subtask "Multilabel Persuasion Detection in Memes") models trained on cleaned versus non-cleaned data. While our experimental results show improvements when using LLM-cleaned captions, statistical tests reveal that most of these improvements are not significant. This suggests that while LLMs have the potential to enhance data cleaning and repairing, their effectiveness may be limited depending on the context they are applied to, the complexity of the task, and the level of noise in the text. Our findings highlight the need for further research into the capabilities and limitations of LLMs in data preprocessing pipelines, especially when dealing with challenging datasets, contributing empirical evidence to the ongoing discussion about integrating LLMs into data preprocessing pipelines.

大模型数据清洗多模态图像描述

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。