AI系统更新导致用户难懂变化原因,提出可解释的治理框架
Update Opacity: Epistemic Accessibility and Governance Under AI System Change
- 用信任度画像和阈值披露区分关键更新
- 结合欧盟AI法案与机器学习运维实现动态监管
- 适合医疗等高风险领域长期监管与透明披露
嵌入部署系统的机器学习模型需定期更新以维持正常运行,但更新可能引发更新不透明:用户无法理解相同输入为何产生不同输出。我们指出,更新不透明本质是历时性认知可及性的失败——重要变化未能以支持理解、合理依赖和适时行动的形式对人类用户保持可及。这构成治理难题。并非所有变化都同等重要,披露全部更新反而会造成信息过载。为此,我们结合欧盟《人工智能法案》界定规范性变更边界,以及机器学习运维(MLOps)提供追踪与比较工具,提出一个基于信任度等级与信任度画像的系统变更建模框架,并采用阈值触发披露机制,向不同利益相关方分时揭示实质性内部变更。通过医疗AI案例说明该方法,推导出生命周期文档、上市后监测与更新披露的实际应用建议。
原文摘要 · Abstract (English)
Machine learning models embedded in deployed AI systems are routinely updated to maintain correct functioning over time. Yet such updates can generate update opacity: users may not be able to understand why the same input now yields a different output. We argue that update opacity is best understood as a diachronic failure of epistemic accessibility: the problem is that materially relevant changes may fail to remain accessible to human users in forms that support understanding, calibrated reliance, and appropriate action under real role- and time-specific constraints. This makes update opacity a governance problem. Not all change is equally relevant, and disclosing every update would itself undermine use through overload. To address this problem, we combine two complementary governance approaches: the EU AI Act, which helps specify the system-level perimeter of normatively relevant change, and Machine Learning Operations, which provides operational tools for tracking and comparing change over time. On this basis, we propose a framework that models system change through trustworthiness profiles and trustworthiness levels, and uses threshold-based disclosure to surface materially relevant within-envelope change to different stakeholders over time. We illustrate the approach with a medical AI example and derive practical implications for lifecycle documentation, post-market monitoring, and update disclosure.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。