In this paper, we explore some basic questions on the complexity of training neural networks with ReLU activation function. We show that it is NP-hard to train a two-hidden layer feedforward ReLU neural network. If dimension of the input data and the network topology is fixed, then we show that there exists a polynomial time algorithm for the same training problem. We also show that if sufficient over-parameterization is provided in the first hidden layer of ReLU neural network, then there is a polynomial time algorithm which finds weights such that output of the over-parameterized ReLU neural network matches with the output of the given data.

中文翻译：

---

训练ReLU神经网络的复杂性

在本文中，我们探讨了有关使用ReLU激活函数训练神经网络的复杂性的一些基本问题。我们证明训练一个两层前馈ReLU神经网络是NP难的。如果输入数据的维数和网络拓扑是固定的，则表明存在针对同一训练问题的多项式时间算法。我们还表明，如果在ReLU神经网络的第一个隐藏层中提供了足够的超参数化，则可以使用多项式时间算法来找到权重，以使超参数化ReLU神经网络的输出与给定数据的输出相匹配。