用激进与克制两种AI推荐策略，提升视频匿名化任务效率与召回率。
Comparing Zealous and Restrained AI Recommendations in a Real-World Human-AI Collaboration Task
- 对比激进（高召回）与克制（高精度）AI在真实协作中的表现。
- 激进AI使任务完成时间更短，召回率更高，且提升整体效率。
- 使用克制AI训练的标注员后续独立工作表现变差，凸显策略影响。
设计AI辅助决策系统时，常面临精度与召回率之间的权衡。我们提出，合理利用这一权衡可充分发挥人机协作的互补优势，显著提升团队绩效。研究聚焦于一个实际的视频匿名化任务，该任务对召回率要求极高且提升成本高昂。分析了78名专业标注员在三种条件下：无AI协助、高精度‘克制型’AI协助、高召回‘激进型’AI协助，共计超过3,466小时的标注工作。结果显示，相较于无辅助或克制型AI，激进型AI能显著缩短任务完成时间并提高召回率。后续实验中，移除所有AI协助后发现，曾接受克制型AI训练的标注员表现下降，表明其存在负面训练效应。这些发现及其分析揭示了在高召回需求场景下，设计AI辅助策略的重要启示。
原文摘要 · Abstract (English)
When designing an AI-assisted decision-making system, there is often a tradeoff between precision and recall in the AI's recommendations. We argue that careful exploitation of this tradeoff can harness the complementary strengths in the human-AI collaboration to significantly improve team performance. We investigate a real-world video anonymization task for which recall is paramount and more costly to improve. We analyze the performance of 78 professional annotators working with a) no AI assistance, b) a high-precision "restrained" AI, and c) a high-recall "zealous" AI in over 3,466 person-hours of annotation work. In comparison, the zealous AI helps human teammates achieve significantly shorter task completion time and higher recall. In a follow-up study, we remove AI assistance for everyone and find negative training effects on annotators trained with the restrained AI. These findings and our analysis point to important implications for the design of AI assistance in recall-demanding scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。