Pangu-ACE通过动态路由实现教育问答的高效生成,1B模型处理大部分简单问题。
Pangu-ACE: Adaptive Cascaded Experts for Educational Response Generation on EduBench
- 采用1B路由模型判断是否直接输出答案或升级到7B专家模型
- 在7013个样本上将准确率提升至0.538,格式正确率提升至0.866
- 适合需要精准与高效兼顾的教育类问答系统应用
教育助手应在任务复杂时增加计算量。本文提出Pangu-ACE系统,基于共享-8 EduBench基准,构建了从1B到7B的样本级自适应级联结构。系统使用1B导师路由模型生成初稿及路由信号,根据需求决定是否接受初稿或升级至7B专家模型进行处理。修复了早期离线评估中的重大错误:此前对开放式输出的评价过高,仅满足表面格式检查。经本地重评分后,完整中文测试集(7013样本)显示,cascade_final将确定性质量从0.457提升至0.538,格式有效性从0.707提升至0.866,同时19.7%的请求可由1B模型直接处理。路由策略高度依赖任务类型:输入预测(IP)有78.0%被1B模型接受,而问题生成(QG)和简答题(EC)几乎始终被升级。当前部署尚未体现延迟优势,效率提升主要来自路由选择性。论文还提供可复现的工作流程,并指出外部基线差距:虽然本地实现了GPT-5.4重新评判,但配置的接口和密钥无效,与GPT-5.4的最终对齐仍待基础设施修复。
原文摘要 · Abstract (English)
Educational assistants should spend more computation only when the task needs it. This paper rewrites our earlier draft around the system that was actually implemented and archived in the repository: a sample-level 1B to 7B cascade for the shared-8 EduBench benchmark. The final system, Pangu-ACE, uses a 1B tutor-router to produce a draft answer plus routing signals, then either accepts the draft or escalates the sample to a 7B specialist prompt. We also correct a major offline evaluation bug: earlier summaries over-credited some open-form outputs that only satisfied superficial format checks. After CPU-side rescoring from saved prediction JSONL, the full Chinese test archive (7013 samples) shows that cascade_final improves deterministic quality from 0.457 to 0.538 and format validity from 0.707 to 0.866 over the legacy rule_v2 system while accepting 19.7% of requests directly at 1B. Routing is strongly task dependent: IP is accepted by 1B 78.0% of the time, while QG and EC still escalate almost always. The current archived deployment does not yet show latency gains, so the defensible efficiency story is routing selectivity rather than wall-clock speedup. We also package a reproducible artifact-first paper workflow and clarify the remaining external-baseline gap: GPT-5.4 re-judging is implemented locally, but the configured provider endpoint and key are invalid, so final sampled-baseline alignment with GPT-5.4 remains pending infrastructure repair.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。