AILA通过自适应层间连接提升模型效率，三类任务表现媲美主流模型。
Adaptive Integrated Layered Attention (AILA)
- 引入自适应层间连接，动态复用多层特征以增强信息流动
- 在价格预测、图像识别与情感分析中均实现接近顶尖模型的精度
- 计算开销仅为基线模型的一小部分，适合资源受限场景
我们提出自适应集成分层注意力（AILA），一种结合密集跳跃连接与跨层自适应特征复用机制的神经网络架构。在三大挑战性任务上评估：多种商品及指数的价格预测（标普500、黄金、美元期货、咖啡、小麦）、基于CIFAR-10的图像识别，以及IMDB影评的情感分析。在所有任务中，AILA性能媲美强基准模型（如LSTM、Transformer、ResNet），同时训练与推理时间大幅降低。我们测试了两种版本：AILA-Architecture 1使用线性层作为层间连接机制；AILA-Architecture 2则采用注意力机制，有选择地关注前层输出。两者均在单任务学习设置下独立训练。结果表明，自适应层间连接能灵活复用相关特征，在不同深度实现稳健性能提升。该方法扩展了现有架构，有效改进长序列建模、高效图像识别，并在实际应用中达到最先进分类性能。
原文摘要 · Abstract (English)
We propose Adaptive Integrated Layered Attention (AILA), a neural network architecture that combines dense skip connections with different mechanisms for adaptive feature reuse across network layers. We evaluate AILA on three challenging tasks: price forecasting for various commodities and indices (S&P 500, Gold, US dollar Futures, Coffee, Wheat), image recognition using the CIFAR-10 dataset, and sentiment analysis on the IMDB movie review dataset. In all cases, AILA matches strong deep learning baselines (LSTMs, Transformers, and ResNets), achieving it at a fraction of the training and inference time. Notably, we implement and test two versions of the model - AILA-Architecture 1, which uses simple linear layers as the connection mechanism between layers, and AILA-Architecture 2, which implements an attention mechanism to selectively focus on outputs from previous layers. Both architectures are applied in a single-task learning setting, with each model trained separately for individual tasks. Results confirm that AILA's adaptive inter-layer connections yield robust gains by flexibly reusing pertinent features at multiple network depths. The AILA approach thus presents an extension to existing architectures, improving long-range sequence modeling, image recognition with optimised computational speed, and SOTA classification performance in practice.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。