Abstract Several practical difficulties arise when trying to apply deep learning to image-based industrial inspection tasks: training datasets are difficult to obtain, each image must be inspected in milliseconds, and defects must be detected with 99% or greater accuracy. In this paper we show how, for image-based industrial inspection tasks, transfer learning can be leveraged to address these challenges. Whereas transfer learning is known to work well only when the source and target domain images are similar, we show that using ImageNet—whose images differ significantly from our target industrial domain—as the source domain, and performing transfer learning, works remarkably well. For one benchmark problem involving 5,520 training images, the resulting transfer-learned network achieves 99.90% accuracy, compared to only a 70.87% accuracy achieved by the same network trained from scratch. Further analysis reveals that the transfer-learned network produces a considerably more sparse and disentangled representation compared to the trained-from-scratch network. The sparsity can be exploited to compress the transfer-learned network up to 1/128 the original number of convolution filters with only a 0.48% drop in accuracy, compared to a drop of nearly 5% when compressing a trained-from-scratch network. Our findings are validated by extensive systematic experiments and empirical analysis.

中文翻译：

---

通过机器视觉检测的迁移学习实现高效的神经网络压缩

摘要 在尝试将深度学习应用于基于图像的工业检测任务时，会出现几个实际困难：训练数据集难以获得，必须在毫秒内检测每个图像，并且必须以 99% 或更高的准确度检测缺陷。在本文中，我们展示了如何在基于图像的工业检测任务中利用迁移学习来应对这些挑战。众所周知，只有当源域图像和目标域图像相似时，迁移学习才能很好地工作，但我们表明，使用 ImageNet（其图像与我们的目标工业域显着不同）作为源域并执行迁移学习，效果非常好。对于一个涉及 5,520 张训练图像的基准问题，由此产生的迁移学习网络实现了 99.90% 的准确率，而只有 70%。从头开始训练的同一网络实现了 87% 的准确率。进一步的分析表明，与从头开始训练的网络相比，转移学习网络产生了更加稀疏和解开的表示。可以利用稀疏性将转移学习网络压缩到原始卷积滤波器数量的 1/128，而准确率仅下降 0.48%，而压缩从头开始训练的网络下降近 5%。我们的发现得到了广泛的系统实验和实证分析的验证。可以利用稀疏性将转移学习网络压缩到原始卷积滤波器数量的 1/128，而准确率仅下降 0.48%，而压缩从头开始训练的网络下降近 5%。我们的发现得到了广泛的系统实验和实证分析的验证。可以利用稀疏性将转移学习网络压缩到原始卷积滤波器数量的 1/128，而准确率仅下降 0.48%，而压缩从头开始训练的网络下降近 5%。我们的发现得到了广泛的系统实验和实证分析的验证。