提出新框架实现请求专属测试时训练的高效批量处理
RW-TTT: Batched Serving for Request-Owned Test-Time Training State
- 通过标记请求归属、版本和读写操作，分批处理兼容阶段
- 单卡八流并行达274.61词元/秒，速度比串行快9.31倍
- 保持长上下文任务表现，适合需个性化推理的场景
测试时训练（TTT）在生成过程中通过读取和更新请求专属状态（如快速权重、低秩增量或流式学习状态）来适应大语言模型。这破坏了传统批量服务中共享静态权重的假设：串行执行正确但慢，而简单批量会污染请求状态。本文将此问题建模为读写TTT服务，提出RW-TTT，为每个解码步骤标注所有者、版本和读写属性，仅对兼容阶段进行批量处理，并仅将更新提交给对应所有者。在单块GPU上运行八个快速权重InPlace-TTT流时，RW-TTT达到274.61词元/秒的聚合吞吐量，在相同内存预算下，分别比串行服务快9.31倍、比单流副本快3.44倍。其在RULER（长上下文基准）上的行为保持一致，且通过所有者/版本校验。
原文摘要 · Abstract (English)
Test-time training (TTT) adapts an LLM during generation by reading and updating request-owned state, such as fast weights, low-rank deltas, or streaming learner state. This breaks batched LLM serving, which assumes shared static weights: serial execution is correct but slow, while naive batching can corrupt request state. We formulate this problem as read-write TTT serving and present RW-TTT , which tags each decode step with its owner, version, and READ/WRITE effect, batches only compatible phases, and commits updates only to the owner. On one GPU with eight fast-weight InPlace-TTT streams, RW-TTT reaches 274.61 aggregate tok/s, 9.31x over sequential serving and 3.44x over per-stream replicas under the same memory budget. It preserves behavior on RULER, a long-context benchmark, and passes owner/version checks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。