The mixture-of-experts (MoE) paradigm attempts to learn complex models by combining several “experts” via probabilistic mixture models. Each expert in the MoE model handles a small area of the data space in which a gating function controls the data-to-expert assignment. The MoE framework has been used extensively in designing non-linear models in machine learning and statistics to model the heterogeneity in data for the purpose of regression, classification and clustering. The existing MoE of multi-target regression (MoE-MTR) models for continuous data is based on multivariate normal distributions. However, in many practical situations, for a set of data, a group or groups of observations may exhibit asymmetric and heavy-tailed behaviour, and inference based on symmetric distributions in such situations can unduly affect the fit of the regression model. We introduce here a novel robust multivariate non-normal MoE model by the use of mean mixture of normal distributions. The proposed model can handle the issues of MoE-MTR models regarding possibly skewed, heavy-tailed and noisy data. Maximum likelihood estimates of model parameters are developed based on an expectation-maximization (EM)-type algorithm. Parsimony is also obtained by imposing suitable constraints on the expert dispersion matrices. The usefulness of the proposed methodology is illustrated using simulated and real data sets.

中文翻译：

---

基于多元正态分布平均混合的简约混合专家

专家混合（MoE）范式试图通过概率混合模型组合几个“专家”来学习复杂模型。MoE 模型中的每个专家处理数据空间的一小块区域，其中门控函数控制数据到专家的分配。MoE 框架已广泛用于设计机器学习和统计中的非线性模型，以对数据中的异质性进行建模，以达到回归、分类和聚类的目的。现有的用于连续数据的多目标回归 (MoE-MTR) 模型的 MoE 是基于多元正态分布的。然而，在许多实际情况下，对于一组数据，一组或多组观察结果可能表现出不对称和重尾行为，在这种情况下，基于对称分布的推断可能会过度影响回归模型的拟合。我们在这里通过使用正态分布的平均混合来介绍一种新的稳健的多元非正态 MoE 模型。所提出的模型可以处理 MoE-MTR 模型关于可能偏斜、重尾和噪声数据的问题。模型参数的最大似然估计是基于期望最大化 (EM) 型算法开发的。简约也可以通过对专家分散矩阵施加适当的约束来获得。使用模拟和真实数据集说明了所提出方法的有用性。所提出的模型可以处理 MoE-MTR 模型关于可能偏斜、重尾和噪声数据的问题。模型参数的最大似然估计是基于期望最大化 (EM) 型算法开发的。简约也可以通过对专家分散矩阵施加适当的约束来获得。使用模拟和真实数据集说明了所提出方法的有用性。所提出的模型可以处理 MoE-MTR 模型关于可能偏斜、重尾和噪声数据的问题。模型参数的最大似然估计是基于期望最大化 (EM) 型算法开发的。简约也可以通过对专家分散矩阵施加适当的约束来获得。使用模拟和真实数据集说明了所提出方法的有用性。