arXiv:2506.01584cs.LGcs.AI2025-06

VirnyFlow同时优化模型精度、公平性和稳定性,支持大规模分布式训练。

VirnyFlow: Optimizing ML Pipelines for Accuracy, Fairness, and Stability at Scale

  • 采用多目标贝叶斯优化与成本感知带权选择,联合优化真实管道
  • 在6个真实数据集上超越或持平主流AutoML系统,128节点下提速7倍
  • 支持人类介入调整目标,适合需兼顾公平与鲁棒性的实际部署场景

构建面向真实部署的机器学习系统需权衡精度、公平性、稳定性等多重目标。现有AutoML框架虽高效,但固定优化目标、缺乏开发者干预且难以扩展。我们提出VirnyFlow,可在大规模下联合优化精度、公平性与稳定性。用户自定义评估协议(含二元及交叉群体公平性)驱动优化全过程:多目标贝叶斯优化物理管道、成本感知带权选择逻辑管道、多准则剪枝。系统结合Apache Kafka异步执行与数据库支撑的实验管理,实现细粒度并行、容错能力与交互式权衡分析。在六个真实数据集上,VirnyFlow在相同资源约束下表现优于或持平auto-sklearn、Alpine Meadow、FLAML;支持跨四节点128个工作进程,处理最大达260万条记录的数据集,相比最佳单节点基线提速最高达7倍,且并行增加时精度与公平性保持稳定。临床案例研究与经伦理审批的用户研究显示,数据科学家可动态定义、检查并迭代优化目标,避免单一“最优”模型输出,更贴合实际部署需求。

原文摘要 · Abstract (English)

Developing machine learning (ML) systems for real-world deployment requires navigating context-dependent trade-offs among accuracy, fairness, stability, and other objectives. Existing AutoML frameworks optimize pipelines efficiently, but they fix the optimization objective up front, leave it outside the developer's control during search, and rarely scale beyond a single node. We present VirnyFlow, a system that optimizes ML pipelines jointly for accuracy, fairness, and stability at scale. A user-defined evaluation protocol, with fairness measured over binary and intersectional groups, drives every layer of the optimizer: multi-objective Bayesian optimization of physical pipelines, cost-aware bandit selection of logical pipelines, and multi-criterion pruning. The architecture combines asynchronous execution over Apache Kafka with database-backed experiment management, providing fine-grained parallelism, fault tolerance, and interactive inspection of trade-offs. On six real-world datasets, VirnyFlow achieves competitive or superior performance compared to state-of-the-art AutoML systems (auto-sklearn, Alpine Meadow, FLAML) under identical resource constraints, scales to 128 workers across four nodes on datasets of up to 2.6M records, and achieves up to 7x higher speedup than the best-scaling single-node baseline, while maintaining stable accuracy and fairness as parallelism increases. A clinical case study on distribution shift and an IRB-approved user study demonstrate human-in-the-loop navigation of trade-offs in practice: rather than returning a single "best" model, VirnyFlow lets data scientists define, inspect, and iteratively refine the objectives of the search to fit their deployment context.

AutoML公平性分布式多目标优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。