通过可逆适配器实现高效多路输入推理，不降性能
RevMUX: Data Multiplexing with Reversible Adapters for Efficient LLM Batch Inference
- 用可逆结构设计多路复用器，共享前向计算
- 在4个数据集上保持分类准确率，提升推理效率
- 适合需要高并发推理的场景，如在线客服
大语言模型（LLM）在自然语言处理领域取得显著进展，但其高吞吐需求带来并发查询处理挑战。数据多路复用通过将多个输入合并为单一复合输入，实现共享前向传播以提升推理效率。然而，从复合输入中区分个体样本困难，传统方法通常需训练整个骨干网络，仍存在性能下降问题。本文提出RevMUX，一种参数高效的多路复用框架，引入可逆设计的多路复用器，其可通过解复用器执行反向操作，恢复原始样本用于分类。在四个数据集和三种类型LLM骨干上的大量实验表明，RevMUX在提升推理效率的同时，仍能保持满意的分类性能。
原文摘要 · Abstract (English)
Large language models (LLMs) have brought a great breakthrough to the natural language processing (NLP) community, while leading the challenge of handling concurrent customer queries due to their high throughput demands. Data multiplexing addresses this by merging multiple inputs into a single composite input, allowing more efficient inference through a shared forward pass. However, as distinguishing individuals from a composite input is challenging, conventional methods typically require training the entire backbone, yet still suffer from performance degradation. In this paper, we introduce RevMUX, a parameter-efficient data multiplexing framework that incorporates a reversible design in the multiplexer, which can be reused by the demultiplexer to perform reverse operations and restore individual samples for classification. Extensive experiments on four datasets and three types of LLM backbones demonstrate the effectiveness of RevMUX for enhancing LLM inference efficiency while retaining a satisfactory classification performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。