Symbolic regression is a powerful technique to discover analytic equations that describe data, which can lead to explainable models and the ability to predict unseen data. In contrast, neural networks have achieved amazing levels of accuracy on image recognition and natural language processing tasks, but they are often seen as black-box models that are difficult to interpret and typically extrapolate poorly. In this article, we use a neural network-based architecture for symbolic regression called the equation learner (EQL) network and integrate it with other deep learning architectures such that the whole system can be trained end-to-end through backpropagation. To demonstrate the power of such systems, we study their performance on several substantially different tasks. First, we show that the neural network can perform symbolic regression and learn the form of several functions. Next, we present an MNIST arithmetic task where a convolutional network extracts the digits. Finally, we demonstrate the prediction of dynamical systems where an unknown parameter is extracted through an encoder. We find that the EQL-based architecture can extrapolate quite well outside of the training data set compared with a standard neural network-based architecture, paving the way for deep learning to be applied in scientific exploration and discovery.

中文翻译：

---

基于神经网络的符号回归在深度学习中的集成用于科学发现

符号回归是一种发现描述数据的分析方程的强大技术，它可以导致可解释的模型和预测看不见的数据的能力。相比之下，神经网络在图像识别和自然语言处理任务上取得了惊人的准确性，但它们通常被视为难以解释且通常推断不佳的黑盒模型。在本文中，我们使用基于神经网络的符号回归架构，称为方程学习器 (EQL) 网络，并将其与其他深度学习架构集成，从而可以通过反向传播对整个系统进行端到端训练。为了展示此类系统的强大功能，我们研究了它们在几个截然不同的任务上的表现。第一的，我们展示了神经网络可以执行符号回归并学习几个函数的形式。接下来，我们提出一个 MNIST 算术任务，其中卷积网络提取数字。最后，我们演示了通过编码器提取未知参数的动态系统的预测。我们发现，与标准的基于神经网络的架构相比，基于 EQL 的架构可以很好地推断出训练数据集之外，为深度学习应用于科学探索和发现铺平了道路。