The human brain has the ability to carry out new tasks with limited experience. It utilizes prior learning experiences to adapt the solution strategy to new domains. On the other hand, deep neural networks (DNNs) generally need large amounts of data and computational resources for training. However, this requirement is not met in many settings. To address these challenges, we propose the TUTOR (training neural networks using decision rules as model priors) DNN synthesis framework. TUTOR targets tabular datasets. It synthesizes accurate DNN models with limited available data and reduced memory/computational requirements. It consists of three sequential steps. The first step involves generation, verification, and labeling of synthetic data. The synthetic data generation module targets both categorical and continuous features. TUTOR generates synthetic data from the same probability distribution as the real data. It then verifies the integrity of the generated synthetic data using a semantic integrity classifier module. It labels synthetic data based on a set of rules extracted from the real dataset. Next, TUTOR uses two training schemes that combine synthetic and training data to learn the DNN model parameters. These two schemes focus on two different ways in which synthetic data can be used to derive a prior on the model parameters and, hence, provide a better DNN initialization for training with real data. In the third step, TUTOR employs a grow-and-prune synthesis paradigm to learn both the weights and the architecture of the DNN to reduce model size while ensuring its accuracy. We evaluate the performance of TUTOR on nine datasets of various sizes. We show that in comparison to fully connected DNNs, TUTOR, on an average, reduces the need for data by $5.9\times $ (geometric mean), improves accuracy by 3.4%, and reduces the number of parameters (floating-point operations) by $4.7\times $ ( $4.3\times $ ) (geometric mean). Thus, TUTOR enables less data-hungry, more accurate, and more compact DNN synthesis.

中文翻译：

---

导师：使用决策规则作为模型先验训练神经网络

人脑有能力在有限的经验下执行新的任务。它利用先前的学习经验使解决方案策略适应新领域。另一方面，深度神经网络 (DNN) 通常需要大量数据和计算资源来进行训练。然而，这个要求在很多情况下都没有得到满足。为了应对这些挑战，我们提出了 TUTOR（使用决策规则作为模型先验训练神经网络）DNN 综合框架。TUTOR 以表格数据集为目标。它使用有限的可用数据和减少的内存/计算要求来合成准确的 DNN 模型。它由三个连续的步骤组成。第一步涉及合成数据的生成、验证和标记。合成数据生成模块同时针对分类特征和连续特征。TUTOR 从与真实数据相同的概率分布中生成合成数据。然后，它使用语义完整性分类器模块验证生成的合成数据的完整性。它根据从真实数据集中提取的一组规则来标记合成数据。接下来，TUTOR 使用两种结合合成数据和训练数据的训练方案来学习 DNN 模型参数。这两种方案侧重于两种不同的方式，其中可以使用合成数据来推导模型参数的先验，从而为使用真实数据进行训练提供更好的 DNN 初始化。在第三步中，TUTOR 采用增长和修剪综合范式来学习 DNN 的权重和架构，以减少模型大小，同时确保其准确性。我们评估了 TUTOR 在九个不同大小的数据集上的性能。 $5.9\次 $（几何平均数），精度提高 3.4%，参数数量（浮点运算）减少 $4.7\次 $( $4.3\次 $ ） （几何平均数）。因此，TUTOR 可以减少数据消耗、更准确、更紧凑的 DNN 合成。