用轻量代理模型高效筛选多模态训练数据,省时43%且性能更优。
Efficient Data Selection for Multimodal Models via Incremental Optimization Utility

- 通过单步更新模拟估算样本边际价值,替代传统语义规则
- 选前50%数据可降43%成本,性能超基线1.8分
- 适合资源有限但追求高效率的多模态模型训练者
大型多模态模型(LMMs)的扩展受限于合成数据的质量-数量权衡。现有方法如LLM-as-a-Judge虽有效,但计算开销大且不可解释。本文提出One-Step-Train(OST),将数据选择重构为增量优化效用排序问题。不依赖语义启发式,而是通过轻量级代理模型模拟单步更新来估计每条样本的边际效用。在Qwen系列多模态数学推理基准上的实验表明,OST实现帕累托最优效率:选取前50%数据可降低43%训练成本(总耗时减少17%),性能优于强基线LLM-as-a-Judge 1.8分;在固定算力预算下,仅用前20%数据即比LLM-as-a-Judge提升5.6分,超越DEITA等启发式基线,并较Full-SFT提升8.8分。值得注意的是,当Full-SFT因噪声导致性能下降时,本方法基于优化机制成功识别有毒样本,有效逆转复杂推理任务中常见的负迁移现象。
原文摘要 · Abstract (English)
The scaling of Large Multimodal Models (LMMs) is constrained by the quality-quantity trade-off inherent in synthetic data. Previous approaches, such as LLM-as-a-Judge, have proven their effectiveness in addressing this but suffer from prohibitive computational costs and lack of interpretability. To bridge this gap, we propose One-Step-Train (OST), a framework that reformulates data selection as an incremental optimization utility ranking problem. Instead of relying on semantic heuristics, OST estimates the marginal utility of each sample via a simulated single-step update on a lightweight proxy. Experiments on the Qwen series across multimodal mathematical reasoning benchmarks demonstrate that OST achieves Pareto-optimal efficiency. By selecting the top-50 subset, OST reduces training costs by 43% (and total time consumption by 17) while surpassing the strong LLM-as-a-Judge baseline by 1.8 points. Furthermore, under a fixed compute budget, our method using only the top-20 subset achieves a 5.6 point gain over LLM-as-a-Judge, improves upon heuristic scoring baselines like DEITA, and outperforms the Full-SFT baseline by 8.8 points. Notably, while Full-SFT suffers from performance degradation due to noise, our optimization-grounded approach effectively identifies toxic samples, successfully reversing the negative transfer frequently observed in complex reasoning tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。