用多智能体系统自动检测教材中的事实错误、技术缺陷和语言问题。
Automated Textbook Auditing with Multi-Agent LLM Systems

- 分事实技术与语法两条轨道,用专业LLM检测错漏并搜索验证。
- 在两本罗马尼亚中学教材中发现56个技术错误和72个综合问题。
- 适合教育出版、教材审校人员快速定位需人工复核的内容。
保证教育材料质量不仅需要常规校对:教材必须同时审核事实准确性、领域专业知识正确性及语言质量——这是通用语法检查工具无法解决的任务。我们提出 extbf{AI Textbook Auditor},一个模块化的多智能体流水线,用于跨学科教育材料的自动化质量保障。系统接收教材PDF,通过两条分析路径生成结构化、可人工审查的报告: extbf{事实与技术轨道} 使用一组专业化LLM智能体检测事实错误、代码错误、定义不当和概念不一致,并在人文学科中结合网络搜索增强; extbf{语法轨道} 在原生PDF上运行,保留变音符号编码。一个 extbf{裁判智能体} 利用领域规则过滤误报,再将结果提交给人工审阅者。该系统支持视觉原生页面渲染和PyMuPDF文本提取两种输入模式,并可通过自定义提示词适配不同学科的错误分类体系。我们在两本罗马尼亚高中教材上进行了验证:一本计算机科学教材发现56项技术问题(分布在七个类别中),专家验证精确率为62.5%;另一本历史与社会科学教材发现72项问题,涵盖事实错误、意识形态偏见与语法问题。系统设计为筛查工具,旨在降低人工定位问题的负担,所有编辑决策仍需专家确认。
原文摘要 · Abstract (English)
Ensuring the quality of educational materials requires more than standard proofreading: textbooks must be audited for factual accuracy, domain-specific technical correctness, and linguistic quality simultaneously -- a task that general-purpose grammar checkers cannot address. We present \textbf{AI Textbook Auditor}, a modular multi-agent pipeline for automated quality assurance of educational materials across subject domains. The system accepts a textbook PDF and produces a structured, human-reviewable report via two analysis tracks: a \textbf{Factual and Technical Track} in which an ensemble of specialized LLM agents detects factual inaccuracies, code errors, incorrect definitions, and conceptual inconsistencies, augmented with web search for humanities domains; and a \textbf{Grammar Track} operating PDF-natively to preserve diacritical encoding. A \textbf{Judge Agent} filters false positives using domain-specific rules before presenting findings to a human reviewer. The pipeline supports two ingestion modes -- vision-native page rendering and PyMuPDF text extraction -- and is domain-adaptable via custom prompts encoding subject-specific error taxonomies. We demonstrate the system on two Romanian upper-secondary textbooks: a CS textbook (56 technical findings across seven categories, with an expert-validated precision of 62.5\%) and a history and social sciences textbook (72 findings spanning factual errors, ideological bias, and grammar). The system is designed as a triage tool that reduces the manual effort of locating candidate issues, with human expert validation required before any editorial action.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。