Deep learning models are increasingly used in mobile applications as critical components. Unlike the program bytecode whose vulnerabilities and threats have been widely-discussed, whether and how the deep learning models deployed in the applications can be compromised are not well-understood since neural networks are usually viewed as a black box. In this paper, we introduce a highly practical backdoor attack achieved with a set of reverse-engineering techniques over compiled deep learning models. The core of the attack is a neural conditional branch constructed with a trigger detector and several operators and injected into the victim model as a malicious payload. The attack is effective as the conditional logic can be flexibly customized by the attacker, and scalable as it does not require any prior knowledge from the original model. We evaluated the attack effectiveness using 5 state-of-the-art deep learning models and real-world samples collected from 30 users. The results demonstrated that the injected backdoor can be triggered with a success rate of 93.5%, while only brought less than 2ms latency overhead and no more than 1.4% accuracy decrease. We further conducted an empirical study on real-world mobile deep learning apps collected from Google Play. We found 54 apps that were vulnerable to our attack, including popular and security-critical ones. The results call for the awareness of deep learning application developers and auditors to enhance the protection of deployed models.

中文翻译：

---

DeepPayload：通过神经有效负载注入对深度学习模型进行黑盒后门攻击

深度学习模型越来越多地在移动应用程序中用作关键组件。与漏洞和威胁已被广泛讨论的程序字节码不同，由于神经网络通常被视为黑匣子，因此无法很好地理解是否会破坏应用程序中部署的深度学习模型以及如何破坏它们。在本文中，我们介绍了一种高度实用的后门攻击，该攻击是通过在编译后的深度学习模型上使用一组反向工程技术实现的。攻击的核心是一个神经条件分支，该神经条件分支由触发检测器和几个操作员构成，并作为恶意有效负载注入受害者模型。攻击是有效的，因为条件逻辑可以由攻击者灵活地自定义，并且可以扩展，因为它不需要原始模型的任何先验知识。我们使用5种最先进的深度学习模型和从30个用户那里收集的真实样本评估了攻击的有效性。结果表明，注入后门可以成功触发93.5％，而仅带来不到2ms的等待时间开销，并且准确度下降不超过1.4％。我们还对从Google Play收集的现实世界中的移动深度学习应用程序进行了实证研究。我们找到了54个容易受到攻击的应用程序，包括流行的和对安全性要求很高的应用程序。结果呼吁深度学习应用程序开发人员和审核人员提高对部署模型的保护。结果表明，注入后门可以成功触发93.5％，而仅带来不到2ms的等待时间开销，并且准确度下降不超过1.4％。我们还对从Google Play收集的现实世界中的移动深度学习应用程序进行了实证研究。我们找到了54个容易受到攻击的应用程序，包括流行的和对安全性要求很高的应用程序。结果呼吁深度学习应用程序开发人员和审核人员提高对部署模型的保护。结果表明，注入后门可以成功触发93.5％，而仅带来不到2ms的等待时间开销，并且准确度下降不超过1.4％。我们还对从Google Play收集的现实世界中的移动深度学习应用程序进行了实证研究。我们找到了54个容易受到攻击的应用程序，包括流行的和对安全性要求很高的应用程序。结果呼吁深度学习应用程序开发人员和审核人员提高对部署模型的保护。