AVA用4000+世行报告打造可信政策生成工具，拒绝胡编乱造。
Learning from AVA: Early Lessons from a Curated and Trustworthy Generative AI for Policy and Development Research
- 基于4000多份世行报告构建多智能体系统，支持跨语言查询。
- 用户每周节省2.4至3.9小时，实证验证高效可用。
- 通过溯源引用和主动拒答建立可信机制，适合政策研究者使用。
通用大模型在发展与政策研究中存在信息失真风险，缺乏可验证输出的元认知谦逊。我们提出AVA（AI + Verified Analysis），一个基于超过4000份世界银行报告的专用生成式AI平台，具备多语言能力。AVA采用多智能体流程，使用户可查询并获得有证据支持的综合回答。其通过双重机制实现元认知谦逊：引用可追溯性（将主张回溯至原始来源）和理性回避（对无支持的问题拒绝回答并说明理由及引导）。我们在116个国家、来自不同组织与角色的2200余名用户中开展真实场景评估，结合日志分析、问卷调查和20次访谈。差异分析估计表明，持续使用可带来每周2.4至3.9小时的时间节省。定性反馈显示，参与者将AVA视作专业“证据引擎”；理性回避帮助厘清边界，信任感则通过机构来源背书和页面锚定引用得以建立。本文贡献了专用AI的设计指南，并提出面向生态感知的谦逊人工智能愿景。
原文摘要 · Abstract (English)
General-purpose LLMs pose misinformation risks for development and policy experts, lacking epistemic humility for verifiable outputs. We present AVA (AI + Verified Analysis), a GenAI platform built on a curated library of over 4,000 World Bank Reports with multilingual capabilities. AVA's multi-agent pipeline enables users to query and receive evidence-based syntheses. It operationalizes epistemic humility through two mechanisms: citation verifiability (tracing claims to sources) and reasoned abstention (declining unsupported queries with justification and redirection). We conducted an in-the-wild evaluation with over 2,200 individuals from heterogeneous organisations and roles in 116 countries, via log analysis, surveys, and 20 interviews. Difference-in-Differences estimates associate sustained engagement with 2.4-3.9 hours saved weekly. Qualitatively, participants used AVA as a specialized "evidence engine"; reasoned abstention clarified scope boundaries, and trust was calibrated through institutional provenance and page-anchored citations. We contribute design guidelines for specialized AI and articulate a vision for "ecosystem-aware" Humble AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。