o1-preview在多数高阶思维任务中超越人类，但逻辑与批判性思维仍有短板。
A Systematic Assessment of OpenAI o1-Preview for Higher Order Thinking in Education
- 用14项认知能力测试对比AI与人类表现，评估其高阶思维能力。
- 在系统、计算、数据等6个维度上表现超人类150%，但在逻辑推理上低25%。
- 适合教育界重新设计课程，聚焦人类不可替代的创造力与抽象思维。
随着人工智能持续发展，其能力已接近人类智能，具有重塑教育与职业发展的巨大潜力。本研究系统评估OpenAI o1-preview在14个高阶认知维度上的表现，包括批判性思维、系统思维、计算思维、设计思维、元认知、数据素养、创造性思维、抽象推理、量化推理、逻辑推理、类比推理和科学推理。采用经验证的工具如Ennis-Weir批判性思维论文测试和生物系统思维测试，对比其与人类的表现。结果表明，o1-preview在系统思维、计算思维、数据素养、创造性思维、科学推理和抽象推理等6个维度上优于人类150%；但在逻辑推理、批判性思维和量化推理方面落后约25%。在类比推理中，双方均达到满分。然而，在抽象推理任务中，人类心理学学生仍优于模型，凸显人类监督在高阶抽象任务中的必要性。研究提示教育需转向培养与AI互补的能力，如创造力、抽象思维与批判性思维，推动教育目标与教学方法向人工智能时代转型。
原文摘要 · Abstract (English)
As artificial intelligence (AI) continues to advance, it demonstrates capabilities comparable to human intelligence, with significant potential to transform education and workforce development. This study evaluates OpenAI o1-preview's ability to perform higher-order cognitive tasks across 14 dimensions, including critical thinking, systems thinking, computational thinking, design thinking, metacognition, data literacy, creative thinking, abstract reasoning, quantitative reasoning, logical reasoning, analogical reasoning, and scientific reasoning. We used validated instruments like the Ennis-Weir Critical Thinking Essay Test and the Biological Systems Thinking Test to compare the o1-preview's performance with human performance systematically. Our findings reveal that o1-preview outperforms humans in most categories, achieving 150% better results in systems thinking, computational thinking, data literacy, creative thinking, scientific reasoning, and abstract reasoning. However, compared to humans, it underperforms by around 25% in logical reasoning, critical thinking, and quantitative reasoning. In analogical reasoning, both o1-preview and humans achieved perfect scores. Despite these strengths, the o1-preview shows limitations in abstract reasoning, where human psychology students outperform it, highlighting the continued importance of human oversight in tasks requiring high-level abstraction. These results have significant educational implications, suggesting a shift toward developing human skills that complement AI, such as creativity, abstract reasoning, and critical thinking. This study emphasizes the transformative potential of AI in education and calls for a recalibration of educational goals, teaching methods, and curricula to align with an AI-driven world.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。