Gaussian mixture models (GMMs) are a family of generative models used extensively in many machine learning applications. The modeling power of GMMs is directly linked to the number of components. Memory, computational load and lack of enough data hinders using GMMs with large number of components. To tackle this problem, GMMs with a tying scheme that we call flexibly tied GMM was proposed in the literature of the speech recognition community. In the literature, a coordinate-descent EM algorithm was proposed for estimating the parameters of flexibly tied GMMs.

In this paper, we aim at reintroducing flexibly tied GMMs to the pattern recognition community. We rigorously investigate various optimization methods and see none of the out-of-the-box optimization methods can solve the parameter estimation problem due to the complexity of the cost function. To this end, we develop a fast Newton EM algorithm that combined with the coordinate descent EM algorithm, it significantly outperforms pure coordinate descent EM and all other optimization algorithms. Furthermore, we propose a computation factorization technique to increase the speed and decrease memory requirement of both Newton and coordinate descent EM algorithms in the case of large number of components. Experimental results on many datasets verifies the efficacy of the proposed algorithm. It also verifies that flexibly tied GMM outperforms both basic GMM and other types of tied GMMs on the datasets in terms of the log-likelihood. We also evaluate the performance of flexibly tied GMM on a clustering problem, and show that it can outperform basic GMM and kmeans algorithm.

高斯混合模型（GMM）是广泛用于许多机器学习应用程序的生成模型系列。GMM的建模能力直接与组件的数量有关。内存，计算量大和缺少足够的数据阻碍了使用具有大量组件的GMM。为了解决这个问题，语音识别社区的文献中提出了一种具有捆绑方案的GMM，我们称之为柔性捆绑GMM。在文献中，提出了一种协调下降的EM算法来估计柔性绑扎的GMM的参数。

在本文中，我们旨在将灵活绑定的GMM重新引入模式识别社区。我们对各种优化方法进行了严格的研究，发现由于成本函数的复杂性，现成的优化方法都无法解决参数估计问题。为此，我们开发了一种结合坐标下降EM算法的快速Newton EM算法，它明显优于纯坐标下降EM和所有其他优化算法。此外，我们提出了一种计算分解技术，以在部件数量众多的情况下提高牛顿和坐标下降EM算法的速度并减少内存需求。在许多数据集上的实验结果证明了该算法的有效性。它还验证了就日志似然而言，灵活绑定的GMM优于数据集上的基本GMM和其他类型的绑定GMM。我们还评估了在集群问题上灵活绑定的GMM的性能，并表明它可以胜过基本的GMM和kmeans算法。