用几何方法加速神经符号学习,实现实时在线推理。
ActPC-Geom: Towards Scalable Online Neural-Symbolic Learning via Accelerating Active Predictive Coding with Information Geometry & Diverse Cognitive Mechanisms
- 以沃尔德斯坦度量替代KL散度,提升网络鲁棒性
- 通过低秩近似与超向量嵌入,实现高效计算
- 支持符号与亚符号推理融合,适合实时系统
本文提出ActPC-Geom,通过引入信息几何中的沃尔德斯坦度量,加速主动预测编码(ActPC)在神经网络中的计算。将传统中用于预测误差评估的KL散度替换为沃尔德斯坦度量,可增强网络鲁棒性。为实现计算可行性,提出三项策略:(1) 使用神经近似器处理逆测度依赖的拉普拉斯算子;(2) 采用近似核主成分分析(kPCA)进行低秩近似;(3) 基于kPCA输出构建组合超向量嵌入,并优化其代数结构以适配模糊形式概念分析(FCA)格。该架构支持实时在线学习,融合连续(如Transformer或霍普菲尔德网络)与离散符号化模型,包括OpenCog Hyperon及用于算法化学进化的ActPC-Chem框架。共享的概率、概念格与超向量模型实现符号-亚符号集成。关键优势包括:(1) 在Transformer类架构中通过超向量嵌入实现组合推理,适用于常识推理任务;(2) 霍普菲尔德网络动态支持关联式长期记忆与吸引子驱动的认知功能。进一步提出将少量样本学习与在线权重更新结合,实现反思性思维与无缝推理。探索伽罗瓦连接以优化混合式ActPC/ActPC-Chem处理流程。最后,设计专用高性能计算架构,满足实时专注注意与反思性推理需求。
原文摘要 · Abstract (English)
This paper introduces ActPC-Geom, an approach to accelerate Active Predictive Coding (ActPC) in neural networks by integrating information geometry, specifically using Wasserstein-metric-based methods for measure-dependent gradient flows. We propose replacing KL-divergence in ActPC's predictive error assessment with the Wasserstein metric, suggesting this may enhance network robustness. To make this computationally feasible, we present strategies including: (1) neural approximators for inverse measure-dependent Laplacians, (2) approximate kernel PCA embeddings for low-rank approximations feeding into these approximators, and (3) compositional hypervector embeddings derived from kPCA outputs, with algebra optimized for fuzzy FCA lattices learned through neural architectures analyzing network states. This results in an ActPC architecture capable of real-time online learning and integrating continuous (e.g., transformer-like or Hopfield-net-like) and discrete symbolic ActPC networks, including frameworks like OpenCog Hyperon or ActPC-Chem for algorithmic chemistry evolution. Shared probabilistic, concept-lattice, and hypervector models enable symbolic-subsymbolic integration. Key features include (1) compositional reasoning via hypervector embeddings in transformer-like architectures for tasks like commonsense reasoning, and (2) Hopfield-net dynamics enabling associative long-term memory and attractor-driven cognitive features. We outline how ActPC-Geom combines few-shot learning with online weight updates, enabling deliberative thinking and seamless symbolic-subsymbolic reasoning. Ideas from Galois connections are explored for efficient hybrid ActPC/ActPC-Chem processing. Finally, we propose a specialized HPC design optimized for real-time focused attention and deliberative reasoning tailored to ActPC-Geom's demands.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。