用可学习的频谱嵌入替代固定傅里叶基,提升图像分类效率
Directional Non-Commutative Monoidal Embeddings for MNIST
- 按维度使用可学习的非交换算子构建嵌入,满足交换律
- 嵌入维度从32降到2时,性能优于固定DFT基
- 适合需要紧凑高效表示的视觉任务
我们对近期提出的方向性非交换幺半群嵌入框架进行了实证验证。该框架通过每维使用不同的非交换算子构建可学习的组合嵌入,满足交换律,推广了经典一维变换。核心目标是验证其在真实数据上的建模能力,因此在经典的MNIST图像分类任务上进行测试。关键假设是:该嵌入能通过学习特定任务的频率成分,推广离散傅里叶变换(DFT)。实验对比了可学习嵌入与固定DFT基在不同嵌入维度下的表现。结果表明,当维度从32降至8再降至2时,可学习嵌入与固定DFT基之间的性能差距逐渐扩大。这一现象说明:可学习嵌入能捕获任务最判别性的频谱成分。整体实验确认,方向性非交换幺半群嵌入在图像数据表示中极为有效,能以紧凑表示维持高任务性能。代码已开源。
原文摘要 · Abstract (English)
We present an empirical validation of the directional non-commutative monoidal embedding framework recently introduced in prior work~\cite{Godavarti2025monoidal}. This framework defines learnable compositional embeddings using distinct non-commutative operators per dimension (axis) that satisfy an interchange law, generalizing classical one-dimensional transforms. Our primary goal is to verify that this framework can effectively model real data by applying it to a controlled, well-understood task: image classification on the MNIST dataset~\cite{lecun1998gradient}. A central hypothesis for why the proposed monoidal embedding works well is that it generalizes the Discrete Fourier Transform (DFT)~\cite{oppenheim1999discrete} by learning task-specific frequency components instead of using fixed basis frequencies. We test this hypothesis by comparing learned monoidal embeddings against fixed DFT-based embeddings on MNIST. The results show that as the embedding dimensionality decreases (e.g., from 32 to 8 to 2), the performance gap between the learned monoidal embeddings and fixed DFT-based embeddings on MNIST grows increasingly large. This comparison is used as an analytic tool to explain why the framework performs well: the learnable embeddings can capture the most discriminative spectral components for the task. Overall, our experiments confirm that directional non-commutative monoidal embeddings are highly effective for representing image data, offering a compact learned representation that retains high task performance. The code used in this work is available at https://github.com/mahesh-godavarti/directional_composition_mnist.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。