AgentAda能自动选技能,让大模型生成更专业的分析结果。
AgentAda: Skill-Adaptive Data Analytics for Tailored Insight Discovery
- 用RAG匹配技能库,自动选最合适的分析方法
- 人类评估中48.78%偏好其分析结果,优于无技能代理
- 适合需要深度数据洞察的研究者与分析师
我们提出AgentAda,首个可学习并使用新分析技能的大型语言模型驱动分析代理。与需用户手动选择分析方法的传统方式不同,AgentAda能从技能库中自动识别并调用所需技能,实现更精准的洞察提取。该技能库涵盖聚类、预测建模及BERT等NLP技术,支持复杂分析任务。其从数据到洞察的策略包括:(I) 问题生成器生成符合用户目标与角色的问题;(II) 基于混合RAG的技能匹配器从库中选出最优分析技能;(III) 代码生成器根据技能文档生成可执行代码以提取关键模式。我们还构建了KaggleBench——一个跨领域精选笔记本的基准测试集,用于评估性能。人类评估显示,AgentAda在48.78%情况下获得更高评价,优于未使用技能的代理(27.67%)。我们进一步提出一种新型的LLM-as-a-judge方法,证明其与人工评估高度一致,可用于大规模自动化洞察质量评估。
原文摘要 · Abstract (English)
We introduce AgentAda, the first LLM-powered analytics agent that can learn and use new analytics skills to extract more specialized insights. Unlike existing methods that require users to manually decide which data analytics method to apply, AgentAda automatically identifies the skill needed from a library of analytical skills to perform the analysis. This also allows AgentAda to use skills that existing LLMs cannot perform out of the box. The library covers a range of methods, including clustering, predictive modeling, and NLP techniques like BERT, which allow AgentAda to handle complex analytics tasks based on what the user needs. AgentAda's dataset-to-insight extraction strategy consists of three key steps: (I) a question generator to generate queries relevant to the user's goal and persona, (II) a hybrid Retrieval-Augmented Generation (RAG)-based skill matcher to choose the best data analytics skill from the skill library, and (III) a code generator that produces executable code based on the retrieved skill's documentation to extract key patterns. We also introduce KaggleBench, a benchmark of curated notebooks across diverse domains, to evaluate AgentAda's performance. We conducted a human evaluation demonstrating that AgentAda provides more insightful analytics than existing tools, with 48.78% of evaluators preferring its analyses, compared to 27.67% for the unskilled agent. We also propose a novel LLM-as-a-judge approach that we show is aligned with human evaluation as a way to automate insight quality evaluation at larger scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。