提出一套让机器学习系统更契合用户需求的工程框架
From Failure to Alignment: A Requirements Engineering Framework for Machine Learning Systems
- 从数据、模型到系统整体，整合多维度需求
- 用失败案例反推优化需求，避免系统出错
- 支持可追溯迭代，适合需要高可信度的开发场景
设计、开发和部署机器学习系统（MLS）的组织需要确保系统的可信性，并向不同利益相关方（如用户、工程师、公众）清晰传达。需求工程以利益相关方为中心，是推动可信MLS设计的重要途径。然而，仍缺乏一种系统化方法，能够结合利益相关方需求与开发约束来建模和推理MLS需求。本文提出名为REAL（Requirements Engineering for mAchines that Learn - and Fail）的框架，通过需求工程方法帮助构建与利益相关方需求对齐的MLS。该基于模型的框架遵循三大原则：第一，融合数据、模型与系统整体的需求；第二，利用失败案例驱动替代需求的探索；第三，实现需求的迭代与可追溯细化。通过自动驾驶领域的实例验证，REAL能有效提升系统与利益相关方需求的一致性。复现代码包已公开。
原文摘要 · Abstract (English)
Organisations designing, developing, and deploying machine learning systems (MLS) need to be able to check that these systems are trustworthy, and communicate this clearly to their stakeholders, be they different categories of users, engineers, or wider society. By focusing on stakeholders, Requirements Engineering is well positioned to drive the design and engineering of MLS that align with the needs of their stakeholders. Yet, we still need a systematic process for modelling and reasoning about requirements for MLS that is driven both by stakeholders' needs and constraints for MLS development. This paper proposes a framework entitled REAL (Requirements Engineering for mAchines that Learn - and Fail) to help develop MLS that align with stakeholders' needs by adopting a requirements engineering approach. This model-based framework is based on three principles. First, weaving together requirements for data, models, and the system as a whole. Second, using failure to drive the exploration of alternative requirements. Third, iterative and traceable refinement of MLS requirements. We demonstrate the proposed framework using an example from autonomous driving and show that REAL supports the development of MLS that better align with stakeholders' requirements. A replication package is available online.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。