用熵增最大化选子集，高效评估图表理解训练数据的效果。
EXaMCaP: Subset Selection with Entropy Gain Maximization for Probing Capability Gains of Large Chart Understanding Training Sets
- 通过最大化样本熵增，迭代选择高多样性子集。
- 子集仅需少量样本即可逼近全集的模型能力提升效果。
- 适用于各类多模态大模型，加速图表数据集的迭代优化。
近期研究致力于合成图表理解（ChartU）训练集，以向多模态大语言模型（MLLMs）注入高级图表知识，通常通过微调后评估来量化能力提升。然而，对全集进行微调耗时巨大，阻碍了图表数据集的迭代优化。我们发现，子集可有效探测全集微调带来的能力增益。由于数据多样性对提升模型性能至关重要，且熵能反映多样性特征，本文提出EXaMCaP，通过熵增最大化策略选择子集。为获得高多样性子集，EXaMCaP从大规模ChartU数据集中选取最大熵子集。由于枚举所有子集不可行，该方法迭代选择使集合熵增最大的样本，近似逼近全集的最大熵子集。实验表明，EXaMCaP在探测ChartU训练集能力增益方面优于基线方法，且在不同子集规模下均表现稳健，并兼容多种MLLM架构。
原文摘要 · Abstract (English)
Recent works focus on synthesizing Chart Understanding (ChartU) training sets to inject advanced chart knowledge into Multimodal Large Language Models (MLLMs), where the sufficiency of the knowledge is typically verified by quantifying capability gains via the fine-tune-then-evaluate paradigm. However, full-set fine-tuning MLLMs to assess such gains incurs significant time costs, hindering the iterative refinement cycles of the ChartU dataset. Reviewing the ChartU dataset synthesis and data selection domains, we find that subsets can potentially probe the MLLMs' capability gains from full-set fine-tuning. Given that data diversity is vital for boosting MLLMs' performance and entropy reflects this feature, we propose EXaMCaP, which uses entropy gain maximization to select a subset. To obtain a high-diversity subset, EXaMCaP chooses the maximum-entropy subset from the large ChartU dataset. As enumerating all possible subsets is impractical, EXaMCaP iteratively selects samples to maximize the gain in set entropy relative to the current set, approximating the maximum-entropy subset of the full dataset. Experiments show that EXaMCaP outperforms baselines in probing the capability gains of the ChartU training set, along with its strong effectiveness across diverse subset sizes and compatibility with various MLLM architectures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。