当前位置: X-MOL 学术Mach. Learn. › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
A scalable sparse Cholesky based approach for learning high-dimensional covariance matrices in ordered data
Machine Learning ( IF 7.5 ) Pub Date : 2019-06-04 , DOI: 10.1007/s10994-019-05810-5
Kshitij Khare , Sang-Yun Oh , Syed Rahman , Bala Rajaratnam

Covariance estimation for high-dimensional datasets is a fundamental problem in machine learning, and has numerous applications. In these high-dimensional settings the number of features or variables p is typically larger than the sample size n. A popular way of tackling this challenge is to induce sparsity in the covariance matrix, its inverse or a relevant transformation. In many applications, the data come with a natural ordering. In such settings, methods inducing sparsity in the Cholesky parameter of the inverse covariance matrix can be quite useful. Such methods are also better positioned to yield a positive definite estimate of the covariance matrix, a critical requirement for several downstream applications. Despite some important advances in this area, a principled approach to general sparse-Cholesky based covariance estimation with both statistical and algorithmic convergence safeguards has been elusive. In particular, the two popular likelihood based methods proposed in the literature either do not lead to a well-defined estimator in high-dimensional settings, or consider only a restrictive class of models. In this paper, we propose a principled and general method for sparse-Cholesky based covariance estimation that aims to overcome some of the shortcomings of current methods, but retains their respective strengths. We obtain a jointly convex formulation for our objective function, and show that it leads to rigorous convergence guarantees and well-defined estimators, even when $$p > n$$p>n. Very importantly, the approach always leads to a positive definite and symmetric estimator of the covariance matrix. We establish both high-dimensional estimation and selection consistency, and also demonstrate excellent finite sample performance on simulated/real data.

中文翻译:

一种在有序数据中学习高维协方差矩阵的基于可扩展稀疏 Cholesky 的方法

高维数据集的协方差估计是机器学习中的一个基本问题,并且有很多应用。在这些高维设置中,特征或变量的数量 p 通常大于样本大小 n。解决这一挑战的一种流行方法是在协方差矩阵、其逆或相关变换中引入稀疏性。在许多应用程序中,数据具有自然排序。在这种情况下,在逆协方差矩阵的 Cholesky 参数中引入稀疏性的方法可能非常有用。这些方法也更适合生成协方差矩阵的正定估计,这是几个下游应用的关键要求。尽管在该领域取得了一些重要进展,具有统计和算法收敛保护措施的基于一般稀疏 Cholesky 的协方差估计的原则方法一直难以捉摸。特别是,文献中提出的两种流行的基于似然的方法要么不会在高维设置中产生明确定义的估计量,要么只考虑一类限制性的模型。在本文中,我们提出了一种基于稀疏 Cholesky 的协方差估计的原则性和通用方法,旨在克服当前方法的一些缺点,但保留各自的优势。我们为我们的目标函数获得了一个联合凸公式,并表明它会导致严格的收敛保证和明确定义的估计量,即使 $$p > n$$p>n。非常重要的是,该方法总是导致协方差矩阵的正定对称估计量。我们建立了高维估计和选择一致性,并在模拟/真实数据上展示了出色的有限样本性能。
更新日期:2019-06-04
down
wechat
bug