提出可学习的正弦余弦激活函数，解决奇偶性判断难题。
TESLA: Taylor Expansion of Sinusoidal Learnable Activations
- 用可学习的正弦余弦组合设计新激活函数，控制多项式阶数。
- 在32位奇偶性任务上，仅需10万样本即实现高泛化能力。
- 适用于需要高阶频率响应的任务，如噪声鲁棒建模与图像识别。
奇偶性问题——判断二进制向量中1的个数是奇数还是偶数——因线性不可分且需全局交互，对标准神经网络仍具挑战。本文提出TESLA，一种由可学习正弦与余弦项组成的激活函数，能显式控制多项式阶数并选择性增强高阶成分。理论上，约束其系数可得利普希茨/雷达马赫复杂度界，并引导训练动态聚焦高频结构。实验上，在输入长度n=32的奇偶性任务中，使用10万训练样本（约0.002%的2^32输入空间）即可实现强泛化性能，且在高达30%标签噪声下仍保持高准确率。对比周期性与频域基线（SIREN、SNAKE、傅里叶特征嵌入）在奇偶性与Forrelation任务上的表现，验证了有效性。此外，特斯拉在ImageNet-100上也达到可比性能，表明激活级阶数控制具备迁移至通用视觉任务的能力。
原文摘要 · Abstract (English)
The parity problem--deciding whether the number of ones in a binary vector is odd or even--remains challenging for standard neural networks due to linear inseparability and the need for global interactions. We propose TESLA, an activation defined as a learnable combination of sine and cosine terms, enabling explicit control over polynomial degree and selective amplification of high-order components. Theoretically, we show that constraining TESLA's coefficients yields Lipschitz/Rademacher complexity bounds and shapes the training dynamics to emphasize higher-frequency structure. Empirically, on parity with input length n = 32, TESLA attains strong generalization with 100K training samples (approximately 0.002% of the 2^32 input space) and remains robust under heavy corruption, retaining high accuracy with up to 30% label noise. We also compare against periodic and frequency-based baselines (SIREN, SNAKE, and Fourier feature embeddings) on parity and Forrelation. Beyond synthetic structure, TESLA delivers comparable performance on ImageNet-100, indicating that activation-level degree control transfers to more general vision workloads. Code: https://github.com/KAU-QuantumAILab/TESLA
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。