Implementing machine learning models on resource-constrained platforms such as hardware devices requires sparse models that can generalize well. This article analyzes the effect of parameter (or weight) quantization on the performance, number of support vectors, model size in bits, $L_2$ norm and training time on various Support Vector Machines (SVM) and Minimal Complexity Machhines (MCM)-based kernel methods. We show that, Empirical Feature Space (EFS) and hinge loss-based MCM algorithms result in comparable accuracy, (8–190)x smaller model size in bits and (10k–16k)x smaller $L_2$ norm at full precision compared with LIBSVM. The Least Squares (LS) variants of MCM based methods results in $\approx$ 2% improvement in accuracy, upto 16x reduction in model size and upto 3x reduction in $L_2$ norm at full precision compared with its state-of-the-art counterpart Sparse Fixed Size variant of LS-SVM (SFS-LS-SVM). We quantize the weights of the compared variants post-training and demonstrate that our methods can retain their accuracies even with 7 bits as opposed to 10 and 14 bits used by LIBSVM and SFS-LS-SVM, respectively. Our experiments illustrate that quantization further improves upon the model sizes used by our methods by upto 300x and 30x compared with the LIBSVM and SFS-LS-SVM. This has significant implications for implementation in Internet of Things (IoT) devices, which benefit from model sparsity and good generalization.

中文翻译：

---

权重量化下的最小复杂度机器

在资源受限的平台（例如硬件设备）上实现机器学习模型需要能够很好地泛化的稀疏模型。本文分析了参数（或权重）量化对性能、支持向量数量、模型大小（以比特为单位）的影响，$L_2$ 各种支持向量机 (SVM) 和基于最小复杂度机器 (MCM) 的内核方法的规范和训练时间。我们表明，经验特征空间 (EFS) 和基于铰链损失的 MCM 算法的精度相当，模型尺寸小 (8-190) 倍，模型尺寸小 (10k-16k)$L_2$ 与 LIBSVM 相比，全精度归一化。基于 MCM 的方法的最小二乘 (LS) 变体导致$\大约$ 准确度提高 2%，模型尺寸减少多达 16 倍，模型尺寸减少多达 3 倍 $L_2$ 与最先进的 LS-SVM 的稀疏固定大小变体 (SFS-LS-SVM) 相比，全精度规范。我们在训练后量化了比较变体的权重，并证明我们的方法即使使用 7 位也可以保持其准确性，而 LIBSVM 和 SFS-LS-SVM 分别使用 10 和 14 位。我们的实验表明，与 LIBSVM 和 SFS-LS-SVM 相比，量化进一步将我们方法使用的模型大小提高了 300 倍和 30 倍。这对物联网 (IoT) 设备的实现具有重要意义，这些设备受益于模型稀疏性和良好的泛化性。