AI越熟练的用户反而失败更多，但成功率更高。
A paradox of AI fluency
- 熟练用户主动迭代与AI协作，不盲目接受输出
- 熟练用户失败率更高但多为可见失败，易部分挽回
- 适合希望深度使用AI的用户和注重交互设计的产品方
用户对AI的熟练程度如何影响其实际获得的结果？这一问题对用户、AI产品设计者及社会都至关重要，但尚未被充分研究。基于来自WildChat-4.8M的27,000条带注释对话数据，我们发现熟练用户承担更复杂任务，采用与AI协同迭代的互动模式：不断调整目标并批判性评估输出；而新手则采取被动态度。这带来一个悖论：熟练用户失败更多——但其失败通常是可见的（源于主动参与），更可能实现部分恢复，并伴随在复杂任务上更高的成功率。相比之下，新手常遭遇不可见失败：看似成功实则偏离目标。这些结果重新定义了与AI交互成功的内涵：个体应采取主动参与而非被动接受的立场；产品设计者需意识到，他们不仅在设计模型行为，也在塑造用户行为——鼓励深度互动，而非追求零摩擦体验，才能带来整体更高成效。代码与数据已公开于https://github.com/bigspinai/bigspin-fluency-outcomes。
原文摘要 · Abstract (English)
How much does a user's skill with AI shape what AI actually delivers for them? This question is critical for users, AI product builders, and society at large, but it remains underexplored. Using a richly annotated sample of 27K transcripts from WildChat-4.8M, we show that fluent users take on more complex tasks than novices and adopt a fundamentally different interactional mode: they iterate collaboratively with the AI, refining goals and critically assessing outputs, whereas novices take a passive stance. These differences lead to a paradox of AI fluency: fluent users experience more failures than novices -- but their failures tend to be visible (a direct consequence of their engagement), they are more likely to lead to partial recovery, and they occur alongside greater success on complex tasks. Novices, by contrast, more often experience invisible failures: conversations that appear to end successfully but in fact miss the mark. Taken together, these results reframe what success with AI depends on. Individuals should adopt a stance of active engagement rather than passive acceptance. AI product builders should recognize that they are designing not just model behavior but user behavior; encouraging deep engagement, rather than friction-free experiences, will lead to more success overall. Our code and data are available at https://github.com/bigspinai/bigspin-fluency-outcomes
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。