Deep neural networks have evolved into a powerful tool applicable for a wide range of problems. However, a clear understanding of their internal mechanism has not been developed satisfactorily yet. Factors such as the architecture, number of hidden layers and neurons, and activation function are largely determined in a guess-and-test manner that is reminiscent of alchemy more than of chemistry. In this paper, we attempt to address these concerns systematically using carefully chosen model systems to gain insights for classification problems. We show how wider networks result in several simple patterns identified on the input space, while deeper networks result in more complex patterns. We show also the transformation of input space by each layer and identify the origin of techniques such as transfer learning, weight normalization and early stopping. This paper is an initial step towards a systematic approach to uncover key hidden properties that can be exploited to improve the performance and understanding of deep neural networks.

深度神经网络已发展成为适用于各种问题的强大工具。但是，对它们的内部机制的清楚的理解还没有令人满意地发展。诸如体系结构，隐藏层和神经元数量以及激活功能之类的因素在很大程度上是通过一种猜测和测试的方式来确定的，这使人联想起炼金术而不是化学。在本文中，我们尝试使用精心选择的模型系统来系统地解决这些问题，以获取有关分类问题的见解。我们展示了更广泛的网络如何导致在输入空间上识别出几个简单的模式，而更深的网络如何导致更复杂的模式。我们还展示了每一层输入空间的变换，并确定了诸如转移学习，体重正常化并尽早停止。本文是朝着揭示关键隐藏属性的系统方法迈出的第一步，可以利用这些隐藏属性来提高性能和对深度神经网络的理解。