将三种自适应推理技术整合为可复现的框架，提升低延迟NLP任务效率。
Efficient Adaptive Transformer: An Empirical Study and Reproducible Framework
- 融合动态剪枝、稀疏注意力与早停机制，实现输入自适应计算。
- 在SST-2任务上略高于优化版DistilBERT的准确率，验证动态计算潜力。
- 提供开源完整工具链，支持自动化测试与对比分析。
高效自适应Transformer（EAT）框架将三种自适应效率技术——渐进式令牌剪枝、稀疏注意力和动态早停——统一为一个可复现的输入自适应推理架构。该框架提供开源基准测试流水线，自动完成GLUE任务（SST-2、QQP、MNLI）的数据处理、计时与消融实验。尽管实证研究发现，在浅层六层模型中结合这些机制可能增加延迟，但EAT在SST-2任务上仍略优于优化后的DistilBERT基线，展现了动态计算在低延迟NLP中的潜力。主要贡献在于提供一个完整的、端到端可复现的框架，包含脚本、CSV日志及分析工具，旨在成为社区研究自适应Transformer的共享工具。
原文摘要 · Abstract (English)
The Efficient Adaptive Transformer (EAT) framework unifies three adaptive efficiency techniques - progressive token pruning, sparse attention, and dynamic early exiting - into a single, reproducible architecture for input-adaptive inference. EAT provides an open-source benchmarking pipeline that automates data processing, timing, and ablation across GLUE tasks (SST-2, QQP, MNLI). Although this empirical study finds that combining these mechanisms can increase latency in shallow six-layer models, it demonstrates that EAT achieves slightly higher accuracy than the optimized DistilBERT baseline on SST-2, illustrating the potential of dynamic computation for latency-sensitive NLP. The main contribution is the open, end-to-end reproducible framework - complete with scripts, CSV logging, and analysis utilities - intended to serve as a community tool for further research on adaptive transformers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。