让教育成果评估自动追踪改进过程,避免因定义变化误判效果下滑
OBER+: Continuity-Aware Reporting and Traceable Continuous Improvement in Outcome-Based Education
- 基于规则的计算框架,将评估差距转化为可追溯的整改措施
- 实证发现核心课程成果定义频繁变动,误读可能导致25分虚假下滑
- 适用于教育机构评估系统升级,尤其关注成果连续性与证据透明
实施成果导向教育的机构定期计算学习成果达成度,但课程分析审查显示缺乏其如何支持决策的证据。本文提出OBER+,对已部署的机构成果平台进行扩展,实现从测量差距到评估纠正措施的自动化推导。该系统包含五个联动阶段:跨课程交付累积成果、识别短期与持续性差距、按监管方已有标准分级、记录决策并关联证据目录、跟踪变更并量化差距变化。新增规则确保在成果定义变更点前后不作直接比较。应用于两个真实课程的实时数据,发现核心课程中所有成果均在连续授课间实质性重定义,内容在不同编号间转移,若简单解读将误报25分下降。重新运行平台规则后,10项中有6项差异超过四舍五入误差,揭示出此前报告的缺陷。在15组成果陈述对比中,5组完全相同;其余10组中,编号相同的成果与前一编号成果最相似的有6组,结果依赖相似性排序,无需阈值或标注。贡献在于提供一套可被任何成果平台实现的计算设计,通过真实记录证明其可揭示关键问题。
原文摘要 · Abstract (English)
Institutions practising outcome-based education compute learning outcome attainment routinely, while reviews of curriculum analytics report an absence of evidence on how that computation informs decisions. This paper presents OBER+, an extension of a deployed institutional attainment platform that computes the step from a measured shortfall to an evaluated corrective action. Five connected stages accumulate attainment across deliveries of a course, signal a shortfall and a persistent shortfall, grade it on cutoffs the regulator already uses, record the decision against a catalogue of practices annotated with their evidence, log the change, and quantify the subsequent movement in the shortfall. A further rule compares successive statements of an outcome, so attainment is never read as a series across a point at which the outcome changed. Applying the rules to the live record of two real courses produced three results. Every outcome of a core course was substantively redefined between consecutive deliveries, with subject matter moving between outcome numbers, so a naive reading would have reported a twenty-five point collapse between quantities that do not refer to the same learning. Recomputing the platform's figures from its documented rule showed six of ten differing by more than rounding explains, in a pattern that identified a defect since reported to the institution. Across fifteen statement pairs from three transitions, five were identical character for character, and among the ten that were not, the outcome carrying a given number was nearest to a differently numbered earlier outcome in six, a result resting on an ordering of similarities and requiring no threshold and no labelling. The contribution is a computational design for outcome-based reporting, stated as rules any attainment platform can implement, with evidence of what they make visible in a live institutional record.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。