给多阶段AI模型划分责任,谁贡献了成功或失败。
Who Gets Credit or Blame? Attributing Accountability in Modern AI Systems
- 通过反事实分析追踪每阶段对模型行为的影响。
- 无需重训练即可量化各阶段贡献,准确识别关键影响。
- 适合模型开发者、审计人员用于责任追溯与风险控制。
现代AI系统通常经历多个阶段:预训练、微调及后续适应或对齐,每个阶段在前一阶段基础上进行更新。当部署的模型表现成功或失败时,应由哪个阶段负责?本文提出可解释性框架,回答“若某一阶段未发生更新,模型行为会如何变化”这一反事实问题。该框架引入高效估计器,在不重新训练模型的前提下,量化各阶段影响,同时考虑数据分布与优化动态(如学习率调度、动量、权重衰减)。实验表明,该方法能准确衡量各阶段对模型行为的贡献,并识别出图像分类和文本毒性检测任务中跨阶段产生的虚假相关性,实现针对性消除。本方法为模型分析提供实用工具,推动可问责的AI开发进程。
原文摘要 · Abstract (English)
Modern AI systems are typically developed through multiple stages-pretraining, fine-tuning rounds, and subsequent adaptation or alignment, where each stage builds on the previous ones and updates the model in distinct ways. This raises a critical question of accountability: when a deployed model succeeds or fails, which stage is responsible, and to what extent? We pose the accountability attribution problem for tracing model behavior back to specific stages of the model development process. To address this challenge, we propose a general framework that answers counterfactual questions about stage effects: how would the model's behavior have changed if the updates from a particular stage had not occurred? Within this framework, we introduce estimators that efficiently quantify stage effects without retraining the model, accounting for both the data and key aspects of model optimization dynamics, including learning rate schedules, momentum, and weight decay. We demonstrate that our approach successfully quantifies the accountability of each stage to the model's behavior. Based on the attribution results, our method can identify and remove spurious correlations learned during image classification and text toxicity detection tasks that were developed across multiple stages. Our approach provides a practical tool for model analysis and represents a significant step toward more accountable AI development.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。