In the paper we consider a new approach to regularize the maximum likelihood estimator of a discrete probability distribution and its application in variable selection. The method relies on choosing a parameter of its convex combination with a low-dimensional target distribution by minimising the squared error (SE) instead of the mean SE (MSE). The choice of an optimal parameter for every sample results in not larger MSE than MSE for James–Stein shrinkage estimator of discrete probability distribution. The introduced parameter is estimated by cross-validation and is shown to perform promisingly for synthetic dependence models. The method is applied to introduce regularized versions of information based variable selection criteria which are investigated in numerical experiments and turn out to work better than commonly used plug-in estimators under several scenarios.

在本文中，我们考虑了一种新方法来规范离散概率分布的最大似然估计量及其在变量选择中的应用。该方法依赖于通过最小化平方误差 (SE) 而不是平均 SE (MSE) 来选择具有低维目标分布的凸组合的参数。为每个样本选择最佳参数导致的 MSE 不大于离散概率分布的 James-Stein 收缩估计器的 MSE。引入的参数是通过交叉验证估计的，并且显示出在合成依赖模型中表现良好。