通过分步评估增强训练，提升轻量多模态大模型的推理自检能力。
Cascaded Self-Evaluation Augmented Training for Lightweight Multimodal LLMs
- 将长推理链拆分为多个短任务，分层评估以增强自检能力。
- 在多数据集上平均提升22.16%性能，显著改善推理准确性。
- 适合需要高可靠性推理的轻量级多模态应用开发者。
高效多模态大语言模型（EMLLMs）可通过思维链（CoT）推理提升性能，但在推理过程中自检能力较弱，这是由于其简化推理流程以及下游微调导致自检能力退化所致。为此，我们提出直观的自评估增强训练（SEAT），利用更强大的EMLLMs对CoT推理数据进行评估，并用评估结果训练EMLLMs。然而，由于EMLLMs处理长序列输入输出存在困难，且自检能力在CoT推理中持续退化，该方法未能充分发挥作用。因此，我们进一步提出级联式自评估增强训练（Cas-SEAT），将长提示转换为多个聚焦特定任务的短提示。同时，混合使用CoT推理与自评估数据，在保留推理能力的同时增强自检能力。我们还引入双层数据过滤（DDF），包括源数据和标注数据过滤，结合人工筛选与多模态大模型自动过滤。Cas-SEAT与DDF协同提升EMLLM性能。实验表明，Cas-SEAT在多个数据集上平均提升22.16%，且DDF显著降低训练资源消耗。
原文摘要 · Abstract (English)
Efficient Multimodal Large Language Models (EMLLMs) can improve performance through Chain-of-Thought (CoT) reasoning, but they have poor self-evaluation capabilities during the CoT reasoning process. This is due to their tendency to simplify the reasoning process and the degradation of self-evaluation ability during downstream task fine-tuning. To address this, we intuitively propose \textit{Self-Evaluation Augmented Training (SEAT)}, which uses more powerful EMLLMs to evaluate CoT reasoning data. The evaluation data is then used to train EMLLMs. However, due to the difficulties EMLLMs face with processing long token input-output sequences, and the degradation of self-evaluation ability as a basis for CoT reasoning, the SEAT method is not fully adapted. Therefore, we further propose \textit{Cascaded Self-Evaluation Augmented Training (Cas-SEAT)}, which converts long prompts into cascaded short prompts, each focusing on a specific task. Additionally, we mix CoT reasoning and self-evaluation data to preserve its CoT reasoning ability while enhancing the self-evaluation capability of EMLLMs. We also conduct \textit{Double-level Data Filtering (DDF)}, which includes source data filtering and labeled data filtering, using both manual selection and MLLMs for filtering. Cas-SEAT and DDF work together to improve the performance of EMLLMs. Experiments show that Cas-SEAT achieves an average improvement of 22.16% across multiple datasets, and DDF significantly reduces the resource consumption of training
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。