人类在回路中监督推荐系统可能引发连锁失败，需警惕其潜在风险。
Cascade! Human in the loop shortcomings can increase the risk of failures in recommender systems
- 分析人类监督在推荐系统中的局限性及其引发连锁故障的机制
- 指出三种典型部署场景会加剧人类监督失效的概率
- 建议改进设计以应对人为监督带来的新风险，适合系统安全研究者
推荐系统是当今最广泛部署的系统之一。为提升数据收集、管理的规范性，业界普遍采用“人类在回路”（human-in-the-loop）模式，主要为增强可问责性。然而本文认为，这种监督模式本身也带来新型风险，且这些风险与系统部署的信息环境密切相关。现有研究表明，人类在监督其他AI系统时存在不足，提示其在推荐系统中也可能难以实现负责任的推荐。本文回顾了人类监督缺陷如何增加“连锁”或“复合”失败的可能性，并探讨三种常见部署场景下人类监督更易失职的动态机制。最后提出两条改进建议。
原文摘要 · Abstract (English)
Recommender systems are among the most commonly deployed systems today. Systems design approaches to AI-powered recommender systems have done well to urge recommender system developers to follow more intentional data collection, curation, and management procedures. So too has the "human-in-the-loop" paradigm been widely adopted, primarily to address the issue of accountability. However, in this paper, we take the position that human oversight in recommender system design also entails novel risks that have yet to be fully described. These risks are "codetermined" by the information context in which such systems are often deployed. Furthermore, new knowledge of the shortcomings of "human-in-the-loop" practices to deliver meaningful oversight of other AI systems suggest that they may also be inadequate for achieving socially responsible recommendations. We review how the limitations of human oversight may increase the chances of a specific kind of failure: a "cascade" or "compound" failure. We then briefly explore how the unique dynamics of three common deployment contexts can make humans in the loop more likely to fail in their oversight duties. We then conclude with two recommendations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。