PsiLogic让优化器自动感知训练混乱期，动态调整更新策略。
PsiLogic: Chaos-Aware Active Cancellation for Adam with a Fair Cross-Domain Benchmark
- 用双指数移动平均检测梯度不稳定性，动态调节修正项强度。
- 在4个任务中3个超越AdamW，NLP困惑度低至7.79（p=0.049）。
- 无需调学习率，适合追求稳定训练的深度学习研究者。
自适应优化器如Adam和AdamW在训练初期混沌阶段与收敛阶段使用相同更新规则。本文提出PsiLogic，通过双指数移动平均（EMA）控制的动态主动抵消项增强Adam，该机制基于归一化梯度幅值检测混沌状态：当梯度统计不稳定时增强阻尼，训练稳定后逐渐归零，实现隐式预热而无需手动调度。我们采用FairBench——一个可复现的基准协议，包含每优化器的学习率扫描、相同初始化种子及Welch t检验。在NVIDIA H100 80GB上运行（4个任务，3个种子，2000步，bf16 AMP），PsiLogic在4个任务中有3个取得最佳验证指标：NLP困惑度7.79±0.18，优于AdamW的8.17±0.08（p=0.049）；ViT Top-1准确率0.244±0.006，优于AdamW的0.223±0.002（p=0.015）；ResNet Top-1准确率0.222±0.001，优于Adam的0.172±0.004（p=0.001）。扩散模型上验证均方误差与Adam/AdamW无显著差异（p=0.49）；ResNet vs AdamW为数值平局（三种子，p=0.44）。峰值显存相近，但Transformer密集任务下有1.2–1.8倍时钟开销（实现限制）。代码、完整FairBench框架及原始数据均已开源。
原文摘要 · Abstract (English)
Adaptive optimizers such as Adam and AdamW apply the same update rule regardless of whether training is in a chaotic early phase or near convergence. We introduce PsiLogic, an optimizer that augments Adam with a dynamic Active Cancellation Term gated by a dual exponential moving average (EMA) of scale-normalized gradient norms. The resulting chaos detector strengthens damping when gradient statistics are unstable and fades to zero as training stabilizes, providing an implicit warmup without a hand-tuned schedule. We evaluate PsiLogic against Adam, AdamW, and Lion using FairBench -- a reproducible benchmark protocol with per-optimizer learning-rate sweeps, identical initialization per seed, and Welch t-tests. On an NVIDIA H100 80GB reference run (4 arenas, 3 seeds, 2000 steps, bf16 AMP), PsiLogic achieves the best validation metric in three of four arenas: NLP perplexity 7.79 +/- 0.18 vs. 8.17 +/- 0.08 (AdamW, p = 0.049), ViT top-1 accuracy 0.244 +/- 0.006 vs. 0.223 +/- 0.002 (AdamW, p = 0.015), and ResNet top-1 accuracy 0.222 +/- 0.001 vs. 0.172 +/- 0.004 (Adam, p = 0.001). On diffusion, validation MSE is statistically tied with Adam/AdamW (p = 0.49). ResNet accuracy vs. AdamW is a numerical tie without significance at three seeds (p = 0.44). Peak GPU memory is comparable across optimizers; PsiLogic incurs 1.2--1.8x wall-clock overhead on transformer-heavy arenas (implementation-bound). We release an open-source PyTorch implementation, the full FairBench harness, and all raw CSV outputs to support independent verification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。