优化大模型代理的GPU执行效率，减少主机往返开销。
Ready Cohorts: Bounding GPU Opportunity and Avoiding Host Round Trips in LLM-Agent Control
- 提出就绪组边界理论，量化GPU并行机会
- 实测显示可恢复81.8%的计算损失
- 将决策留在设备端，提速最高2.39倍
LLM-agent服务反复执行小规模确定性状态转移：路由结果、更新状态、生成下一动作。本文研究何时控制路径能暴露足够并发工作供GPU执行，并探讨当GPU计算的路由决策保留在设备端时的变化。通过固定分区占比F、精确离线占比P*、本地上界U和在线实现占比A定义就绪组边界。在零服务时间、无限容量、等相对启动截止时间条件下，专用动态规划可精确计算P*。基于一个固定的851会话公共数据集，在10万目标活跃会话、K=256、50ms启动截止时间下，F=30.19%，P*=43.00%，U=45.85%。精确打包恢复了固定窗口边界下81.83%的损失机会。结果导向的路由键是条件代理，非可执行身份证明。另一机制研究将GPU计算的二元决策保留在设备端，避免返回4字节至主机再重发。在四个命名GPU位置中，所有36种配置均更快；同位置中位加速比为1.19x至2.39x。两种可接受机制下，全部14,557,440次批处理调用均与独立主机验证器一致。固定嵌套设备图（无主机决策）在五个位置的全部60种配置中更慢。两项研究共同确立了两个可测量的GPU代理控制门限：截止时间可行的组供应与观测放置。联合有限在线运行需测量实际实现占比A、CPU卸载程度和服务水平收益。
原文摘要 · Abstract (English)
LLM-agent services repeatedly execute small deterministic transitions between model and tool calls: route an outcome, update state, and emit the next effect. We ask when this control path exposes enough concurrent work for GPU execution, and what changes when a GPU-computed route decision remains on device. We formalize the ready-cohort boundary using fixed-partition share F, exact offline share P*, local upper bound U, and online achieved share A. Under zero service time, unlimited capacity, and equal relative launch deadlines, a specialized dynamic program computes P* exactly. In a stationary Poisson replay of one pinned 851-session public trace panel, the primary condition at 100,000 target active sessions, K=256, and a 50 ms launch deadline gives F=30.19%, P*=43.00%, and U=45.85%. Exact packing recovers 81.83% of the opportunity lost at fixed window boundaries. The outcome-derived route key is a conditioning proxy, not proof of executable identity. A separate mechanism study keeps a GPU-computed binary decision on device instead of returning four bytes to the host and redispatching. Across four named GPU placements, the device-resident path is faster in all 36 configurations; within-placement row-median ratios range from 1.19x to 2.39x. Across both admissible mechanisms, all 14,557,440 tested batched invocations match a separately implemented host oracle. A fixed nested device graph that removes no host decision is slower in all 60 configurations across five placements. Together, the studies establish two measurable gates for GPU agent control: deadline-feasible cohort supply and observation placement. A joined finite online runtime is required to measure A, CPU displacement, and service-level benefit.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。