In conventional machine learning applications, each data attribute is assumed to be orthogonal to others. Namely, every pair of dimension is orthogonal to each other and thus there is no distinction of in-between relations of dimensions. However, this is certainly not the case in real world signals which naturally originate from a spatio-temporal configuration. As a result, the conventional vectorization process disrupts all of the spatio-temporal information about the order/place of data whether it be 1D, 2D, 3D, or 4D. In this paper, the problem of orthogonality is first investigated through conventional k-means of images, where images are to be processed as vectors. As a solution, shift-invariant k-means is proposed in a novel framework with the help of sparse representations. A generalization of shift-invariant k-means, convolutional dictionary learning is then utilized as an unsupervised feature extraction method for classification. Experiments suggest that Gabor feature extraction as a simulation of shallow convolutional neural networks provides a little better performance compared to convolutional dictionary learning. Other alternatives of convolutional-logic are also discussed for spatio-temporal information preservation, including a spatio-temporal hypercomplex encoding scheme.

在传统的机器学习应用程序中，假设每个数据属性都与其他数据属性正交。也就是说，每一对维度都是相互正交的，因此没有维度之间的关系之间的区别。然而，在自然源自时空配置的现实世界信号中，情况肯定不是这样。结果，传统的矢量化过程破坏了有关数据顺序/位置的所有时空信息，无论是 1D、2D、3D 还是 4D。在本文中，正交性问题首先通过传统的图像k-means来研究，其中图像将被处理为向量。作为一种解决方案，在稀疏表示的帮助下，在一个新颖的框架中提出了移位不变的 k-means。移位不变k-means的推广，然后将卷积字典学习用作分类的无监督特征提取方法。实验表明，与卷积字典学习相比，Gabor 特征提取作为浅层卷积神经网络的模拟提供了更好的性能。还讨论了用于时空信息保存的卷积逻辑的其他替代方案，包括时空超复杂编码方案。