用样本内一致性筛选视觉指令数据，提升模型性能且节省训练资源。
DICS: Exploring Data Intrinsic Consistency for Visual Instruction Selection
- 基于视觉与指令、响应间的内在一致性设计评分机制
- 仅用25%数据达到全量训练94.5%性能，显著优于现有方法
- 适合大规模视觉语言模型训练的数据筛选，尤其关注效率与质量
视觉指令微调对提升视觉-语言模型（VLMs）的对齐能力与指令遵循能力至关重要。然而，在数据快速膨胀背景下，如何在固定比例约束下选取最优子集仍是一大瓶颈。现有方法多依赖分布多样性或启发式过滤，常忽略单个样本内部的一致性。为此，我们提出数据内在一致性（DIC），一种用于量化样本级组件间一致性的自评分指标，包含两个模块：视觉信息一致性（VIC），评估视觉内容与指令的对齐程度；响应信息一致性（RIC），衡量回答相对于指令的连贯性。基于DIC，我们提出数据内在一致性选择（DICS），一种适应性数据选择方法，在不同数据预算下权衡高样本内一致性与全局分布多样性。大量实验表明，DICS在多种数据规模与模型架构下持续优于当前最佳方法，使用仅25%的LLaVA-1.5-665K数据即超越全量微调效果。我们进一步构建了包含600万样本的DICS-6M多模态指令语料库，实现了迄今最大规模的视觉指令选择研究；尤为突出的是，DICS仅用其报告训练数据的25%以下，便达到官方InternVL3-8B-Instruct 94.52%的性能。
原文摘要 · Abstract (English)
Visual instruction tuning is crucial for advancing the vision-language alignment and instruction-following capabilities of Vision-Language Models (VLMs). However, identifying optimal subsets under a fixed ratio constraint from rapidly expanding datasets remains a significant bottleneck. While existing methods largely depend on distribution diversity or heuristic filtering, they often overlook the internal coherence within individual samples. To bridge this gap, we propose Data Intrinsic Consistency (DIC), a self-scoring metric designed to quantify the sample-level inter-component consistency. DIC consists of two modules: Visual Information Consistency (VIC), evaluating the alignment between visual content and instructions, and Response Information Consistency (RIC), assessing response coherence relative to the instruction. Building upon DIC, we introduce Data Intrinsic Consistency Selection (DICS), an adaptive data selection method that optimizes the trade-off between high intra-sample consistency and global distributional diversity under varying data budgets. Extensive experiments demonstrate that DICS consistently outperforms state-of-the-art methods across diverse dataset scales and model architectures, surpassing full-dataset fine-tuning while using only 25% of the LLaVA-1.5-665K data. We further curate DICS-6M, a 6M-sample multi-modal instruction corpus that enables the largest-scale visual instruction selection study to date; remarkably, DICS reaches 94.52\% of the official InternVL3-8B-Instruct performance using less than 25\% of its reported training data. Code can be seen at https://github.com/cqu-student/DICS
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。