Abstract

Density estimation plays a fundamental role in many areas of statistics and machine learning. Parametric, nonparametric, and semiparametric density estimation methods have been proposed in the literature. Semiparametric density models are flexible in incorporating domain knowledge and uncertainty regarding the shape of the density function. Existing literature on semiparametric density models is scattered and lacks a systematic framework. In this article, we consider a unified framework based on reproducing kernel Hilbert space for modeling, estimation, computation, and theory. We propose general semiparametric density models for both a single sample and multiple samples which include many existing semiparametric density models as special cases. We develop penalized likelihood based estimation methods and computational methods under different situations. We establish joint consistency and derive convergence rates of the proposed estimators for both finite dimensional Euclidean parameters and an infinite-dimensional functional parameter. We validate our estimation methods empirically through simulations and an application. Supplementary materials for this article are available online.

摘要

密度估计在统计和机器学习的许多领域都发挥着重要作用。文献中已经提出了参数、非参数和半参数密度估计方法。半参数密度模型可以灵活地结合领域知识和关于密度函数形状的不确定性。现有的关于半参数密度模型的文献比较分散，缺乏系统的框架。在本文中，我们考虑了一个基于再现核希尔伯特空间的统一框架，用于建模、估计、计算和理论。我们为单个样本和多个样本提出了通用半参数密度模型，其中包括许多现有的半参数密度模型作为特例。我们在不同情况下开发了基于惩罚似然的估计方法和计算方法。我们为有限维欧几里得参数和无限维函数参数建立联合一致性并导出所提出的估计量的收敛速度。我们通过模拟和应用程序验证了我们的估计方法。本文的补充材料可在线获取。