发现循环网络权重空间中任务相关的对称性，可识别哪些改动不影响性能。
Task-Restricted Symmetries in Recurrent Weight Space
- 用舒尔分解分析权重空间，分离出谱块与非正规耦合。
- 部分非正规耦合移除后任务表现几乎不变，表明存在近似功能不变性。
- 适用于调试训练好的循环网络，帮助理解其计算机制。
循环网络在权重空间中存在显著的功能冗余：改变递归矩阵可能对任务分布上的输入输出轨迹影响极小，而类似规模的其他变化却会破坏原有行为。本文通过有序实舒尔坐标研究单层tanh RNN中的这种冗余。舒尔形式将谱块与定向非正规耦合分离，提供固定输入和读出映射下的结构化消融诊断基础。在固定长度复制任务中，某些非正规舒尔耦合可被移除且损失极小，而其他耦合对自主重放至关重要。在翻转、正弦生成和上下文依赖整合任务中，损失保持的消融模式随任务和训练解不同而异。结果揭示了候选的近似功能不变性，而非循环权重空间的普适对称性。舒尔坐标消融为判断何种结构扰动能保留训练好的递归解提供了实用诊断工具。
原文摘要 · Abstract (English)
Recurrent networks can contain substantial functional redundancy in weight space: changing a recurrent matrix may leave the input-output rollout nearly unchanged on a task distribution, while similar-scale changes can destroy the same behavior. We study this redundancy in one-layer tanh RNNs using ordered real Schur coordinates. The Schur form separates spectral blocks from directed nonnormal couplings, giving a diagnostic basis for structured ablations that keep the input and readout maps fixed. In a fixed-length copy task, selected nonnormal Schur couplings can be removed with little loss in some trained solutions, whereas other couplings are necessary for accurate autonomous replay. Across flip-flop, sine generation, and context-dependent integration, the loss-preserving ablation profile varies across tasks and trained solutions. These results identify candidate approximate functional invariances, not universal symmetries of recurrent weight space. Schur-coordinate ablations provide a practical diagnostic for which structured perturbations preserve a trained recurrent solution and which ones disrupt its computation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。