We study deep neural networks and their use in semiparametric inference. We establish novel rates of convergence for deep feedforward neural nets. Our new rates are sufficiently fast (in some cases minimax optimal) to allow us to establish valid second-step inference after first-step estimation with deep learning, a result also new to the literature. Our estimation rates and semiparametric inference results handle the current standard architecture: fully connected feedforward neural networks (multi-layer perceptrons), with the now-common rectified linear unit activation function and a depth explicitly diverging with the sample size. We discuss other architectures as well, including fixed-width, very deep networks. We establish nonasymptotic bounds for these deep nets for a general class of nonparametric regression-type loss functions, which includes as special cases least squares, logistic regression, and other generalized linear models. We then apply our theory to develop semiparametric inference, focusing on causal parameters for concreteness, such as treatment effects, expected welfare, and decomposition effects. Inference in many other semiparametric contexts can be readily obtained. We demonstrate the effectiveness of deep learning with a Monte Carlo analysis and an empirical application to direct mail marketing.

中文翻译：

---

用于估计和推理的深度神经网络

我们研究深度神经网络及其在半参数推理中的应用。我们为深度前馈神经网络建立了新的收敛速度。我们的新速率足够快（在某些情况下为极小极大最优），使我们能够在深度学习的第一步估计之后建立有效的第二步推理，这也是文献中的新结果。我们的估计率和半参数推理结果处理当前的标准架构：完全连接的前馈神经网络（多层感知器），具有现在常见的修正线性单元激活函数，深度与样本大小明显不同。我们还讨论了其他架构，包括固定宽度、非常深的网络。我们为这些深度网络建立了非参数回归类型损失函数的非渐近边界，其中包括作为特殊情况的最小二乘法、逻辑回归和其他广义线性模型。然后，我们应用我们的理论来开发半参数推理，重点关注具体性的因果参数，例如处理效果、预期福利和分解效果。可以很容易地获得许多其他半参数上下文中的推理。我们通过蒙特卡罗分析和直接邮件营销的实证应用证明了深度学习的有效性。