用智能搜索找最危险又真实的模型漏洞，效率提升两倍
TESTNAV: Pareto-Guided Search for Compositional Robustness Testing
- 将鲁棒性测试转为双目标优化，兼顾破坏力与输入真实性
- 在四个任务上比基线快2.15倍，仅用不到九成的扰动组合
- 适合需要高效发现真实场景下模型弱点的研究者
深度学习模型对真实世界输入扰动仍易出错，尤其当多种损坏同时出现时（如亮度变化与运动模糊）。组合式测试可揭示这些交互效应，但面临两大挑战：扰动空间随维度和严重程度指数级增长；大量组合产生不现实的劣化输入，诊断价值低。本文提出TESTNAV，一种基于帕累托优化的鲁棒性测试框架，可在有限评估次数下高效探索离散组合扰动空间。它将鲁棒性测试建模为双目标优化问题：最大化性能下降，同时保持输入保真度（视觉用SSIM、KID；语言与代码用chrF、BERT-F1）。采用NSGA-II算法逼近帕累托前沿。在涵盖视觉、自然语言和代码生成的四个基准上，TESTNAV比基于搜索的基线快2.15倍，仅使用4个扰动维度、每维6级共定义空间的35.8%~89.3%。
原文摘要 · Abstract (English)
Deep learning models remain vulnerable to real-world input perturbations, especially when multiple corruptions co-occur in the same input (e.g., brightness shifts and motion blur). Compositional testing reveals these interaction effects but introduces two challenges: combinatorial growth of the perturbation space as dimensions and severity levels increase, and uneven diagnostic value-many combinations yield unrealistically degraded inputs with limited practical relevance. We present TESTNAV, 1 a Pareto-guided robustness testing framework for efficiently exploring discrete, compositional perturbation spaces when only a limited number of perturbation configurations can be evaluated. TESTNAV prioritises severe yet realistic failures by formulating robustness testing as bi-objective optimisation: maximise performance degradation while preserving input fidelity measured by modality-specific metrics (e.g., SSIM and KID for vision; chrF and BERT-F1 for language and code). It uses NSGA-II to approximate the bi-objective Pareto front. Across four benchmarks spanning vision, natural language, and code generation, TESTNAV recovers Pareto fronts up to 2.15x faster than search-based baselines, using 35.8%-89.3% of the discrete perturbation space defined by four perturbation dimensions with six levels each.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。