用多个专业小助手让普通人也能轻松分析公开数据。
PublicAgent: Multi-Agent Design Principles From an LLM-Based Open Data Analysis Framework
- 拆分任务为发现、分析、报告等专用智能体，避免大模型注意力分散。
- 实测显示专业分工后分析成功率86%~92%，错误率显著下降。
- 适合想用自然语言分析公共数据但缺乏统计知识的科研或决策者。
开放数据仓库蕴含支持科学决策的巨大潜力，但非专家难以完成数据发现、模式映射和统计分析。尽管大语言模型在单项任务中表现良好，但端到端分析流程存在注意力衰减、推理模式干扰和错误传播等问题。本文提出PublicAgent，一个基于多智能体的框架，将任务分解为意图澄清、数据发现、分析与报告四个专业化智能体，保持各环节专注并实现阶段验证。在五个模型和50个查询上的评估提炼出五项设计原则：第一，专业化独立于模型规模，即使最强模型也达97.5%胜率，且收益与模型大小正交；第二，智能体分为通用型（发现、分析）与条件型（报告、意图），前者波动较小（标准差12.4%），后者受模型影响大（标准差20.5%）；第三，移除发现或分析模块导致灾难性失败（243–280例），而移除报告或意图仅降低质量；第四，架构优势在复杂任务中稳定存在，分析与发现胜率分别维持在86%–92%和84%–94%，体现工作流管理价值而非推理增强；第五，不同模型间智能体表现差异大（分析效率42%–96%），需采用模型感知的设计策略。这些原则为复杂分析流程中是否需要专业化提供了依据，并推动通过自然语言接口扩大公众对公共数据的可及性。
原文摘要 · Abstract (English)
Open data repositories hold potential for evidence-based decision-making, yet are inaccessible to non-experts lacking expertise in dataset discovery, schema mapping, and statistical analysis. Large language models show promise for individual tasks, but end-to-end analytical workflows expose fundamental limitations: attention dilutes across growing contexts, specialized reasoning patterns interfere, and errors propagate undetected. We present PublicAgent, a multi-agent framework that addresses these limitations through decomposition into specialized agents for intent clarification, dataset discovery, analysis, and reporting. This architecture maintains focused attention within agent contexts and enables validation at each stage. Evaluation across five models and 50 queries derives five design principles for multi-agent LLM systems. First, specialization provides value independent of model strength--even the strongest model shows 97.5% agent win rates, with benefits orthogonal to model scale. Second, agents divide into universal (discovery, analysis) and conditional (report, intent) categories. Universal agents show consistent effectiveness (std dev 12.4%) while conditional agents vary by model (std dev 20.5%). Third, agents mitigate distinct failure modes--removing discovery or analysis causes catastrophic failures (243-280 instances), while removing report or intent causes quality degradation. Fourth, architectural benefits persist across task complexity with stable win rates (86-92% analysis, 84-94% discovery), indicating workflow management value rather than reasoning enhancement. Fifth, wide variance in agent effectiveness across models (42-96% for analysis) requires model-aware architecture design. These principles guide when and why specialization is necessary for complex analytical workflows while enabling broader access to public data through natural language interfaces.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。