In state-of-the-art deep learning, centralized deep learning forces end devices to pool their data in the cloud in order to train a global model on the joint data, while distributed deep learning requires a parameter server to mediate the training process among multiple end devices. However, none of these architectures scale gracefully to large-scale privacy and time-sensitive IoT applications. Therefore, we are motivated to propose a FOg-based pRivacy prEServing dEep lEarNing framework named FORESEEN, so as to achieve scalable, accurate yet private analytics. In FORESEEN, the intermediate fog nodes and the cloud collaboratively perform noisy training of deep neural networks (DNNs), while each end device and its connected fog node collaboratively perform fast, private yet accurate inference. To enhance robustness and ensure privacy, we put forward a collaborative noisy training algorithm and develop a novel representation perturber to perturb the extracted features by combining random projection, random noise addition and data nullification. To meet the required constraints of accuracy, memory and energy in IoT end devices, we build deep models with mixed-precision. Through these sophisticated designs, FORESEEN is able to not only preserve privacy but also maintain comparable inference performance. Extensive experimental results under different datasets, different inference schemes and different noise addition strategies validate the effectiveness of FORESEEN. Moreover, FORESEEN is capable of reducing the communication cost and providing inherent support for robustness and scalability.

中文翻译：

---

FORESEEN：面向智能物联网的差异化私有深度推理

在最先进的深度学习中，集中式深度学习迫使终端设备将它们的数据集中在云端，以便在联合数据上训练全局模型，而分布式深度学习需要一个参数服务器来调解训练过程多个终端设备。然而，这些架构都不能优雅地扩展到大规模隐私和时间敏感的物联网应用程序。因此，我们有动力提出一个名为 FORESEEN 的基于 FOg 的隐私保护深度学习框架，以实现可扩展、准确且私密的分析。在 FORESEEN 中，中间雾节点和云协同执行深度神经网络 (DNN) 的噪声训练，而每个终端设备及其连接的雾节点协同执行快速、私密但准确的推理。为了增强健壮性并确保隐私，我们提出了一种协同噪声训练算法，并开发了一种新颖的表示干扰器，通过结合随机投影、随机噪声添加和数据无效来干扰提取的特征。为了满足物联网终端设备对精度、内存和能量的要求，我们构建了混合精度的深度模型。通过这些复杂的设计，FORESEEN 不仅能够保护隐私，而且还能保持可比的推理性能。不同数据集、不同推理方案和不同噪声添加策略下的大量实验结果验证了 FORESEEN 的有效性。此外，FORESEEN 能够降低通信成本并为健壮性和可扩展性提供内在支持。