This article contributes to the current statistical theory of deep neural networks (DNNs). It was shown that DNNs are able to circumvent the so--called curse of dimensionality in case that suitable restrictions on the structure of the regression function hold. In most of those results the tuning parameter is the sparsity of the network, which describes the number of non-zero weights in the network. This constraint seemed to be the key factor for the good rate of convergence results. Recently, the assumption was disproved. In particular, it was shown that simple fully connected DNNs can achieve the same rate of convergence. Those fully connected DNNs are based on the unbounded ReLU activation function. In this article we extend the results to smooth activation functions, i.e., to the sigmoid activation function. It is shown that estimators based on fully connected DNNs with sigmoid activation function also achieve the minimax rates of convergence (up to $\ln n$-factors). In our result the number of hidden layers is fixed, the number of neurons per layer tends to infinity for sample size tending to infinity and a bound for the weights in the network is given.

中文翻译：

---

具有平滑激活函数的全连接深度神经网络回归估计的收敛率分析

本文对当前深度神经网络 (DNN) 的统计理论做出了贡献。结果表明，在对回归函数结构的适当限制成立的情况下，DNN 能够规避所谓的维数灾难。在大多数结果中，调整参数是网络的稀疏性，它描述了网络中非零权重的数量。这种约束似乎是获得良好收敛速度的关键因素。最近，这个假设被推翻了。特别是，它表明简单的全连接 DNN 可以实现相同的收敛速度。那些完全连接的 DNN 基于无界 ReLU 激活函数。在本文中，我们将结果扩展到平滑激活函数，即 sigmoid 激活函数。结果表明，基于具有 sigmoid 激活函数的全连接 DNN 的估计器也实现了极小极大收敛率（高达 $\ln n$-factors）。在我们的结果中，隐藏层的数量是固定的，每层的神经元数量趋于无穷大，样本量趋于无穷大，并且给出了网络中权重的界限。