解释无法让人们对有偏见的预测警务系统建立合理信任。
"Even explanations will not help in trusting [this] fundamentally biased system": A Predictive Policing Case-Study
- 对比文本、视觉和混合解释形式对用户信任的影响。
- 专家用户主观信任提升,但决策质量未改善。
- 无论何种解释,均难建立恰当信任,适合政策设计参考。
在人工智能日益重要的今天,用户信任问题备受关注。高风险领域中,AI系统常导致用户信任不足或过度依赖。过往研究认为解释可帮助用户判断何时信任系统,但在高风险场景下,不同解释形式(文本、视觉、混合)与用户专业度(退伍警员与普通用户)对建立合理信任的影响仍不明确。本研究发现,混合解释虽提升专家用户的主观信任,却未改善决策质量;且任何解释形式均未能帮助用户建立恰当信任。结果强调需重新审视解释在存在争议的AI系统中的作用,并据此提出设计挑战与政策建议,以实现高风险场景下的合理信任。
原文摘要 · Abstract (English)
In today's society, where Artificial Intelligence (AI) has gained a vital role, concerns regarding user's trust have garnered significant attention. The use of AI systems in high-risk domains have often led users to either under-trust it, potentially causing inadequate reliance or over-trust it, resulting in over-compliance. Therefore, users must maintain an appropriate level of trust. Past research has indicated that explanations provided by AI systems can enhance user understanding of when to trust or not trust the system. However, the utility of presentation of different explanations forms still remains to be explored especially in high-risk domains. Therefore, this study explores the impact of different explanation types (text, visual, and hybrid) and user expertise (retired police officers and lay users) on establishing appropriate trust in AI-based predictive policing. While we observed that the hybrid form of explanations increased the subjective trust in AI for expert users, it did not led to better decision-making. Furthermore, no form of explanations helped build appropriate trust. The findings of our study emphasize the importance of re-evaluating the use of explanations to build [appropriate] trust in AI based systems especially when the system's use is questionable. Finally, we synthesize potential challenges and policy recommendations based on our results to design for appropriate trust in high-risk based AI-based systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。