用张量压缩提升神经符号学习的可扩展性，让模型在千级输入下仍保持高精度。
CTSketch: Compositional Tensor Sketching for Scalable Neurosymbolic Learning
- 将符号程序拆解为子程序，并用张量草图压缩表示。
- 在千级输入任务中实现高精度，仅依赖最终输出监督。
- 理论保证近似误差上限，适合大规模符号推理场景。
许多计算任务可通过神经网络与离散符号程序的组合来建模。神经符号学习的目标是利用复合系统的端到端输入-输出标签训练神经网络。本文提出CTSkech，一种新型可扩展的神经符号学习算法。该方法通过将符号程序分解为子程序，并用草图张量总结每个子程序，从而利用简单的张量运算近似程序输出分布。我们提供了最大近似误差的理论分析。在神经符号学习领域的基准测试中评估，包括部分专为评估可扩展性设计的任务。结果表明，CTSketch将神经符号学习推至前所未有的规模，即使在一千个输入的复杂任务中，神经预测器仍能获得高精度，且仅使用最终输出进行监督。
原文摘要 · Abstract (English)
Many computational tasks benefit from being formulated as the composition of neural networks followed by a discrete symbolic program. The goal of neurosymbolic learning is to train the neural networks using end-to-end input-output labels of the composite. We introduce CTSketch, a novel, scalable neurosymbolic learning algorithm. CTSketch uses two techniques to improve the scalability of neurosymbolic inference: decompose the symbolic program into sub-programs and summarize each sub-program with a sketched tensor. This strategy allows us to approximate the output distribution of the program with simple tensor operations over the input distributions and the sketches. We provide theoretical insight into the maximum approximation error. Furthermore, we evaluate CTSketch on benchmarks from the neurosymbolic learning literature, including some designed for evaluating scalability. Our results show that CTSketch pushes neurosymbolic learning to new scales that were previously unattainable, with neural predictors obtaining high accuracy on tasks with one thousand inputs, despite supervision only on the final output.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。