用神经符号混合架构提升非对称噪声下的回归鲁棒性
Beyond Least Squares: Robust Regression Transformer (R2T)
- Transformer+压缩网络联合预测符号方程参数
- 在合成可穿戴数据上中位MSE达6e-6至3.5e-5
- 比普通最小二乘和鲁棒回归快10到300倍
传统鲁棒回归依赖最小二乘优化，对高斯噪声有效，但在存在非对称结构噪声时表现不佳。本文提出一种神经符号混合架构：使用Transformer编码器处理数值序列，压缩神经网络预测符号参数，固定符号方程重构原始序列。通过合成数据训练，目标是在添加非对称结构噪声后恢复原始序列，从而学习由神经网络参数估计引导的符号拟合。模型在合成可穿戴数据上的中位回归均方误差（MSE）为6×10⁻⁶至3.5×10⁻⁵，相比普通最小二乘法及Huber损失、SoftL1等鲁棒回归方法，性能提升10至300倍。
原文摘要 · Abstract (English)
Robust regression techniques rely on least-squares optimization, which works well for Gaussian noise but fails in the presence of asymmetric structured noise. We propose a hybrid neural-symbolic architecture where a transformer encoder processes numerical sequences, a compression NN predicts symbolic parameters, and a fixed symbolic equation reconstructs the original sequence. Using synthetic data, the training objective is to recover the original sequence after adding asymmetric structured noise, effectively learning a symbolic fit guided by neural parameter estimation. Our model achieves a median regression MSE of 6e-6 to 3.5e-5 on synthetic wearable data, which is a 10-300 times improvement when compared with ordinary least squares fit and robust regression techniques such as Huber loss or SoftL1.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。