One of the major challenges in multivariate analysis is the estimation of population covariance matrix from the sample covariance matrix (SCM). Most recent covariance matrix estimators use either shrinkage transformations or asymptotic results from Random Matrix Theory (RMT). Both of these techniques try to achieve a similar goal which is to remove noisy correlations and add structure to SCM to overcome the bias-variance trade-off. Both methods have their respective pros and cons. In this paper, we propose an improved estimator which exploits the advantages of these techniques by taking optimally weighted convex combination of covariance matrices estimated by shrinkage transformation and a filter based on RMT. It is a generalized estimator which can adapt to changing sampling noise conditions by performing hyperparameter optimization. Using data from six of the world's biggest stock exchanges, we show that the proposed estimator outperforms the existing estimators in minimizing the out-of-sample risk of the portfolio and hence predicts population statistics more precisely. The proposed estimator can be useful in a wide range of machine learning and signal processing applications.

中文翻译：

---

改进的协方差矩阵估计在投资组合优化中的应用

多元分析的主要挑战之一是从样本协方差矩阵 (SCM) 估计总体协方差矩阵。最近的协方差矩阵估计器使用随机矩阵理论 (RMT) 的收缩变换或渐近结果。这两种技术都试图实现类似的目标，即去除噪声相关性并为 SCM 添加结构以克服偏差-方差权衡。这两种方法各有优缺点。在本文中，我们提出了一种改进的估计器，它通过采用收缩变换估计的协方差矩阵的最优加权凸组合和基于 RMT 的滤波器来利用这些技术的优点。它是一种广义估计器，可以通过执行超参数优化来适应不断变化的采样噪声条件。使用来自世界上最大的六家证券交易所的数据，我们表明建议的估计量在最小化投资组合的样本外风险方面优于现有的估计量，因此更准确地预测人口统计数据。建议的估计器可用于广泛的机器学习和信号处理应用。