通过可控实验对比大模型智能体分解策略,找出税务判定中最佳分工方式。
Right-Sizing LLM-Agent Decomposition in VAT Determination: A Pilot Controlled Sweep
- 固定任务流程,仅改变智能体数量与分工粒度,测试不同配置效果。
- 中间配置准确率最高(0.830),但未超过预设标准,无法支持最优解假设。
- 实验揭示了任务分解的边界效应,适合关注系统设计的工程团队参考。
近期的大模型智能体系统在设计上存在分歧:是拆分任务给多个专精小智能体,还是用一个强大工具型智能体完成全部工作。本研究在跨境增值税反向征收这一有明确答案的限定场景下开展试点实验,每项任务均有真实标签,中间决策也可独立评分。保持活动面不变(子任务、工具、输入输出结构、验证机制、协调器、基础模型、合并策略),仅调整子任务分配给不同工作者的配置,从单一宽职能者到五个窄职能者,对比无协调器的单智能体(S0)和确定性规则引擎作为基准。实验共执行4,400次运行:包含40个案例、五次重复的主实验,匹配令牌预算与智能体数量的分离组,以及三组故障注入实验,均按预先注册的可证伪标准评估。两个中间配置在准确率上领先(0.830),而两端分别为0.720和0.770,但未达预设高标准,因此中间最优假设在试点规模下仍不成立。单智能体未能帕累托优于整体协同集。匹配令牌标准显示:预算对等的单智能体落后领先者6.5分,但置信区间包含零,优势可能源于提示预算差异。故障注入下,可用性故障在各粒度均可吸收;宽范围重启恢复基线提升0.160;而一个符合格式的幻觉记录破坏所有配置,尤其影响碎片化结构,导致顺序反转。贡献在于提供了一个受限、预注册的试点启发式方法,用于合理划分任务分解边界(在依赖层中点设置分界),并公开了真实标签、数据集、测试框架、原始追踪和分析流水线。
原文摘要 · Abstract (English)
Recent LLM-agent systems make conflicting design bets: decompose work across many narrow agents, or use one strong tool-using agent. This pilot studies that choice on bounded cross-border VAT determination with reverse charge, where every case has an oracle label and each intermediate decision is independently scoreable. We hold the activity surface fixed (subtasks, tools, I/O schemas, validation checks, orchestrator, base model, and merge policy) and vary only the assignment of subtasks to workers across four orchestrated configurations, from one wide worker to five narrow ones, against S0, a tuned no-orchestrator single agent, with a deterministic rule engine as oracle. The program spans 4,400 runs: a 40-case, five-repeat main sweep, matched-token arms separating prompt-budget from agent-count effects, and three failure-injection arms, all judged against pre-registered falsification criteria. The two intermediate configurations lead on accuracy (0.830, against endpoints at 0.720 and 0.770) but miss the pre-stated bar against the fine endpoint, so the intermediate-optimum hypothesis remains unsupported at pilot scale. The single agent does not Pareto-dominate the orchestrated set. The matched-token criterion fires: the budget-matched single agent lands 6.5 points below the leader, but the interval includes zero, so any advantage is consistent with a prompt-budget explanation. Under injection, availability faults are absorbed at every granularity, with wide-scope restart over-recovering its baseline by +0.160, while one schema-conforming hallucinated record degrades every configuration and inverts the ordering, hitting fragmented configurations hardest. The contribution is a bounded, preregistered pilot heuristic for right-sizing decomposition (place one partition boundary at the dependency-layer midpoint), released with oracle, dataset, harness, raw traces, and analysis pipeline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。