让RNN从时间序列拓展到多维度有序结构,挖掘隐藏依赖关系
Recurrent Neural Networks Beyond Time: Learning from Multiple Ordered Projections
- 通过多个有序投影训练独立专家模型,再融合其表示
- 在复杂合成数据上显著提升性能,简单数据上保持竞争力
- 适用于各类结构化学习任务,不依赖特定模型架构
循环神经网络(RNN)广泛用于序列建模,但通常局限于时间序列。本文提出有序结构依赖假设(OSDH),认为同一组观测的多种合法排序可揭示单一顺序无法捕捉的互补结构依赖。为此,提出独立结构专家原则(ISEP):先对不同投影训练独立序列模型,再由融合模型整合表示。作为实例,提出结构演化RNN(SE-RNNs),使用标准RNN作为投影特异性结构专家,保持原有递归计算不变。三个具有显著不同结构复杂度的合成数据集上的概念验证实验表明,当存在隐含结构依赖时,该架构能稳定受益于多有序投影,而在简单数据上仍具竞争力。由于OSDH与底层序列处理模型无关,该框架可自然扩展至其他架构,在多样化结构学习问题中提供通用计算视角。
原文摘要 · Abstract (English)
Recurrent neural networks (RNNs) are widely used for sequence learning, yet their application is commonly associated with temporal data, although recurrent computation fundamentally operates on ordered sequences rather than on time itself. Building on this observation, we introduce the Ordered Structural Dependency Hypothesis (OSDH), which proposes that multiple admissible orderings of the same observations may reveal complementary structural dependencies inaccessible through a single sequential organization. To operationalize this hypothesis, we propose the Independent Structural Expert Principle (ISEP), whereby projection-specific sequence models are trained independently before their learned representations are integrated through a dedicated fusion model. As a concrete realization, we present Structural Evolution RNNs (SE-RNNs), which employ conventional RNNs as projection-specific structural experts while preserving the underlying recurrent computation unchanged. Proof-of-concept experiments on three synthetic datasets with substantially different levels of structural complexity demonstrate that the proposed architecture consistently benefits from multiple ordered projections when hidden structural dependencies are present, while remaining competitive on simpler datasets. Since OSDH is independent of the underlying sequence-processing model, the proposed framework naturally extends beyond recurrent networks and may be instantiated using alternative architectures. The results suggest a general computational perspective for exploiting complementary ordered representations across diverse structured learning problems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。