We revisit the classic semi‐parametric problem of inference on a low‐dimensional parameter θ in the presence of high‐dimensional nuisance parameters η. We depart from the classical setting by allowing for η to be so high‐dimensional that the traditional assumptions (e.g. Donsker properties) that limit complexity of the parameter space for this object break down. To estimate η, we consider the use of statistical or machine learning (ML) methods, which are particularly well suited to estimation in modern, very high‐dimensional cases. ML methods perform well by employing regularization to reduce variance and trading off regularization bias with overfitting in practice. However, both regularization bias and overfitting in estimating η cause a heavy bias in estimators of θ that are obtained by naively plugging ML estimators of η into estimating equations for θ. This bias results in the naive estimator failing to be consistent, where N is the sample size. We show that the impact of regularization bias and overfitting on estimation of the parameter of interest θ can be removed by using two simple, yet critical, ingredients: (1) using Neyman‐orthogonal moments/scores that have reduced sensitivity with respect to nuisance parameters to estimate θ; (2) making use of cross‐fitting, which provides an efficient form of data‐splitting. We call the resulting set of methods double or debiased ML (DML). We verify that DML delivers point estimators that concentrate in an ‐neighbourhood of the true parameter values and are approximately unbiased and normally distributed, which allows construction of valid confidence statements. The generic statistical theory of DML is elementary and simultaneously relies on only weak theoretical requirements, which will admit the use of a broad array of modern ML methods for estimating the nuisance parameters, such as random forests, lasso, ridge, deep neural nets, boosted trees, and various hybrids and ensembles of these methods. We illustrate the general theory by applying it to provide theoretical properties of the following: DML applied to learn the main regression parameter in a partially linear regression model; DML applied to learn the coefficient on an endogenous variable in a partially linear instrumental variables model; DML applied to learn the average treatment effect and the average treatment effect on the treated under unconfoundedness; DML applied to learn the local average treatment effect in an instrumental variables setting. In addition to these theoretical applications, we also illustrate the use of DML in three empirical examples.

中文翻译：

---

用于处理和结构参数的双/无偏机器学习

我们在存在高维扰动参数η的情况下，重新审视对低维参数θ进行推论的经典半参数问题。我们通过允许η如此高的维数来偏离经典设置，以至于打破了限制该对象参数空间复杂性的传统假设（例如Donsker属性）。为了估计η，我们考虑使用统计方法或机器学习（ML）方法，这些方法特别适合于在现代超高维情况下进行估计。ML方法通过采用正则化来减少方差并在实践中过度拟合来权衡正则化偏差，从而表现良好。然而，正则化偏差和估计η的过度拟合都会导致θ估计量的严重偏差，这是通过将η的ML估计量天真地插入θ的估计方程而获得的。这种偏差会导致天真的估算器不一致，其中N是样本大小。我们证明，可以通过使用两个简单但关键的成分来消除正则化偏差和过度拟合对目标参数θ的估计的影响：（1）使用对有害参数敏感度降低的内曼正交矩/分数估计θ；（2）利用交叉拟合，这提供了一种有效的数据拆分形式。我们称方法的结果集为双重或无偏ML（DML）。我们验证DML提供的点估计量集中在真实参数值附近，并且近似无偏且呈正态分布，从而可以构造有效的置信度语句。DML的通用统计理论是基础知识，同时仅依赖于较弱的理论要求，这将允许使用各种各样的现代ML方法来估计令人讨厌的参数，例如随机森林，套索，山脊，深层神经网络，增强树木，以及这些方法的各种混合体和合奏。我们通过应用通用理论提供以下方面的理论特性来说明通用理论：DML用于学习部分线性回归模型中的主要回归参数；DML用于在部分线性工具变量模型中学习内生变量的系数；DML用于学习无混杂情况下被治疗者的平均治疗效果和平均治疗效果；DML用于在工具变量设置中了解局部平均治疗效果。除了这些理论应用之外，我们还在三个经验示例中说明了DML的用法。