The lack of interpretability and trust is a much-criticized feature of deep neural networks. In fully connected nets, the signaling between inner layers is scrambled because backpropagation training does not require perceptrons to be arranged in any particular order. The result is a black box; this problem is particularly severe in scientific computing and digital signal processing (DSP), where neural nets perform abstract mathematical transformations that do not reduce to features or concepts. We present here a group-theoretical procedure that attempts to bring inner-layer signaling into a human-readable form, the assumption being that this form exists and has identifiable and quantifiable features—for example, smoothness or locality. We applied the proposed method to DEERNet (a DSP network used in electron spin resonance) and managed to descramble it. We found considerable internal sophistication: the network spontaneously invents a bandpass filter, a notch filter, a frequency axis rescaling transformation, frequency-division multiplexing, group embedding, spectral filtering regularization, and a map from harmonic functions into Chebyshev polynomials—in 10 min of unattended training from a random initial guess.

缺乏可解释性和信任度是深度神经网络广受批评的特征。在完全连接的网络中，内层之间的信号被打乱，因为反向传播训练不需要将感知器以任何特定顺序排列。结果是一个黑匣子；这个问题在科学计算和数字信号处理（DSP）中尤为严重，其中神经网络执行的抽象数学变换不会简化为特征或概念。我们在这里提出了一种组理论过程，该过程试图将内层信号传递为人类可读的形式，假设该形式存在并且具有可识别和可量化的特征（例如，平滑度或局部性）。我们将提出的方法应用于DEERNet（用于电子自旋共振的DSP网络）并设法对其进行解密。我们发现内部机制非常复杂：网络自发地发明了一个带通滤波器，一个陷波滤波器，一个频率轴重定标变换，频分多路复用，组嵌入，频谱滤波正则化以及从谐波函数到Chebyshev多项式的映射-在10分钟内来自随机初始猜测的无人值守训练。