Security of Mobile Multimedia Data:The Adversarial Examples for Spatio-temporal Data☆
Graphical abstract
Introduction
Mobile multimedia devices combine computer and video technology. These devices are becoming more and more portable for people. They include smartphones, laptops, wireless sensors, etc [1], [2], [3], [4], [5]. Besides, they can also process multimedia data, such as audio, video, and images. With the popularity of these mobile devices, a large amount of multimedia data is generated. These multimedia data has great commercial value. Therefore, how to utilize these multimedia data is essential in the wireless communication area. Compared to previous desktop devices, mobile multimedia devices bring new challenges and opportunities.
In recent years, deep learning shows great potential in data analysis. It has made significant progress in Natural Language Process (NLP) and Computer Vision (CV) fields [6], [7], [8], [9], [10]. To better mine the value of multimedia data, deep learning is also applied in mobile multimedia computing systems. Some studies show that deep learning has achieved good performance in Multimedia data analysis. However, owing to upload data and process data at the cloud end, there is a certain delay in deep learning models. Fortunately, edge computing makes up for the shortcoming. Recently, some studies show that deep learning models are vulnerable to the crafted samples. The crafted samples are called adversarial examples. In image classification task, the small perturbations are barely recognized for human but can make deep learning models predict the wrong labels. Therefore, AI-based Security and Privacy in mobile multimedia computing systems need to draw significant attention.
In the paper, we conduct experiments to explore the security of mobile multimedia computing systems in deep learning modes. In multimedia data, we use the Spatio-temporal data (GPS trajectory data) to conduct experiments. The Spatio-temporal data contains longitude, latitude, and timestamp information. Specially, based on Spatio-temporal data, we generate adversarial examples to attack trajectory mode classification models. The attacked task is trajectory mode classification [11], [12], [13], [14], [15], [16], [17], [18]. In the task, the trajectory mode classification model takes the four-channel features of each trajectory segment as input and outputs the corresponding trajectory modes, including biking, walking, driving, etc. The model is deep learning networks.
To the best of our knowledge, the research of adversarial examples mainly focuses on Computer Vision (CV) and Natural Language Process (NLP). Since both Spatio-temporal data and image data are continuous, we try to adopt the algorithm of Computer Vision (Fast Gradient Sign Method (FGSM)) to generate adversarial examples. In detail, firstly, we extract four-channel features (speed, acceleration, jerk, bearing rate) from each trajectory segment. Then we use a Convolutional AutoEncoder (CAE) to convert four-channel features into image format data. Moreover, in trajectory mode classification task, we use the image format data to train the three deep learning models, including VGG16, VGG 19, shallow convolutional neural networks. Finally, based on image format data and trained models, we adopt FGSM algorithm to generate adversarial examples. We use adversarial examples to attack three deep learning models. Experimental results show that adversarial examples have strong attack ability to three deep models.
In summary, the main contribution of this paper is as follow:
- •
We develop a CAE architecture to convert GPS trajectory data into image format data. We firstly extract four-channel features (speed, acceleration, jerk, bearing rate) from each trajectory segment. Then we train the CAE model to convert four-channel features into image format data.
- •
We utilize image format data to train three trajectory mode classification models. In original trajectory data, each trajectory segment corresponds to a trajectory mode, including biking, walking, driving, etc. We use image format data of each trajectory segment to train three deep learning models (VGG16, VGG19, shallow convolutional neural networks).
- •
We adopt FGSM algorithm to attack the three trained models. In the test set, based on the gradient of the trained model, we use FGSM algorithm to generate adversarial examples. Moreover, we use the decoder to reconstruct the four-channel features from adversarial examples and original image format data. By visualizing the results, we find that there is no difference between adversarial examples and original data.
- •
We conduct extensive experiments on GPS trajectory datasets. The adversarial examples obviously reduce the accuracy of three deep models. Furthermore, the adversarial examples have good transferability.
The rest of this paper is organized as follows. In Section 2, we discuss related work. We introduce our ASCAE model architecture in Section 3. In Section 4, we discuss experimental results and Section 5 summarizes the paper.
Section snippets
Related work
In this section, we firstly discuss the related work on trajectory mode classification task. Then we summarize the literature work in adversarial examples. Finally we provide the motivations of our work.
Trajectory Mode Classification Task. The algorithms used for trajectory mode classification are mainly divided into two categories: traditional machine learning algorithms and deep learning algorithms. In traditional trajectory mode classification algorithms, firstly the features need to be
Model Architecture
In the section, we introduce the proposed model ASCAE. The model architecture is presented in Fig. 1. Firstly, we present several preliminaries. Secondly, we introduce the data preprocessing part. Thirdly, we show the attacked models. Finally, we describe the attack process and the attack algorithm.
Experiment
In the section, we firstly introduce our experiment settings. Then, we briefly present the evaluation metric, task descriptions and baseline models. Finally, we show the attack ability and transferability of adversarial examples by experiments.
Conclusion
In the paper, we conduct experiments to explore the security of mobile multimedia computing systems in deep learning modes. In multimedia data, we use the Spatio-temporal data (GPS trajectory data) to conduct experiments. Based on Spatio-temporal data, we generate adversarial examples to attack deep learning models. The deep learning models attacked are three trajectory mode classification models, including shallow convolutional neural networks, VGG16 networks, and VGG19 networks. Since both
CRediT authorship contribution statement
Yuanyuan Chen: Conceptualization, Methodology, Software, Formal analysis, Writing - original draft. Jing Qiu: Writing - original draft, Data curation. Xiaojiang Du: Funding acquisition, Writing - review & editing. Lihua Yin: Conceptualization, Formal analysis. Zhihong Tian: Data curation, Formal analysis, Writing - review & editing.
Declaration of Competing Interest
No conflict of interest.
Yuanyuan Chen is working toward the master’s degree at Guangzhou University. Her current research interests include spatio-temporal data and data mining. E-mail: [email protected].
References (39)
- et al.
Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition
Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security
(2016) - et al.
Deep learning for mobile multimedia
ACM Transactions on Multimedia Computing Communications and Applications
(2017) Location-dependent multimedia computing
IEEE Concurrency
(1999)- et al.
Adaptive bandwidth reservation mechanism using mobility probability in mobile multimedia computing environment
Proceedings 25th Annual IEEE Conference on Local Computer Networks. LCN 2000
(2000) - et al.
Real time lateral movement detection based on evidence reasoning network for edge computing environment
IEEE Transactions on Industrial Informatics
(2019) - et al.
From iot to 5g i-iot: The next generation iot-based intelligent algorithms and 5g technologies
IEEE Communications Magazine
(2018) - et al.
Inception-v4
Inception-ResNet and the Impact of Residual Connections on Learning
(2016) Attention is all you need
Advances in Neural Information Processing Systems
(2017)- et al.
Multiple moving targets surveillance based on a cooperative network for multi-UAV
IEEE Communications Magazine
(2018) - et al.
Vcash: A novel reputation framework for identifying denial of traffic service in internet of connected vehicles
IEEE Internet of Things Journal.
(2020)
Defending dos attacks on broadcast authentication in wireless sensor networks
Proc. of IEEE International Conference on Communications (ICC 2008)
Trajectory data mining: an overview
ACM Transactions on Intelligent Systems and Technology (TIST)
Transportation mode-based segmentation and classification of movement trajectories
International Journal of Geographical Information Science
Understanding transportation modes based on GPS data for web applications
ACM Transactions on the Web (TWEB)
Learning transportation mode from raw gps data for geographic applications on the web
Proceedings of the 17th international conference on World Wide Web.
Security in mobile edge caching with reinforcement learning
IEEE Wireless Communications
Cloud-based malware detection game for mobile devices with offloading
IEEE Transactions on Mobile Computing
Urban traffic prediction from spatio-temporal data using deep meta learning
KDD
Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting
IJCAI
Cited by (3)
Threshold-guided multi-objective Generative Adversarial Networks for constructing artificial yet representative driving cycles
2024, Engineering Applications of Artificial IntelligenceThe Shortest Verification Path of the MHT Scheme for Verifying Distributed Data
2022, Applied Sciences (Switzerland)Seek-and-Hide: Adversarial Steganography via Deep Reinforcement Learning
2022, IEEE Transactions on Pattern Analysis and Machine Intelligence
Yuanyuan Chen is working toward the master’s degree at Guangzhou University. Her current research interests include spatio-temporal data and data mining. E-mail: [email protected].
Jing Qiu received the Ph.D. degree in computer applications technology from Beijing Institute of Technology. She is currently a Professor of the Cyberspace Institute of Advanced Technology, Guangzhou University. She was a Visiting Scholar with the University of Southern California, LA, USA, under the supervision of Professor Craig A. Knoblock. Her current research interest is Information Extraction, Network Representation, and Big Data Analysis. E-mail: [email protected].
Xiaojiang Du (S’99M’03SM’09F’20) received his B.S. and M.S. degree in Electrical Engineering (Automation Department) from Tsinghua University, Beijing, China in 1996 and 1998, respectively. He received his M.S. and Ph.D. degree in Electrical Engineering from the University of Maryland, College Park in 2002 and 2003, respectively. Dr. Du is a tenured Full Professor and the Director of the Security And Networking (SAN) Lab in the Department of Computer and Information Sciences at Temple University, Philadelphia, USA. His research interests are security, wireless networks, and systems. He has authored over 400 journal and conference papers in these areas, as well as a book published by Springer. Dr. Du has been awarded more than 6 million US Dollars research grants from the US National Science Foundation (NSF), Army Research Office, Air Force Research Lab, NASA, the State of Pennsylvania, and Amazon. He won the best paper award at IEEE ICC 2020 and IEEE GLOBECOM 2014, and the best poster runner-up award at the ACM MobiHoc 2014. He serves on the editorial boards of two international journals. Dr. Du served as the lead Chair of the Communication and Information Security Symposium of the IEEE International Communication Conference (ICC) 2015, and a Co-Chair of Mobile and Wireless Networks Track of IEEE Wireless Communications and Networking Conference (WCNC) 2015. He is (was) a Technical Program Committee (TPC) member of several premier ACM/IEEE conferences such as INFOCOM (2007 - 2020), IM, NOMS, ICC, GLOBECOM, WCNC, BroadNet, and IPCCC. Dr. Du is an IEEE Fellow and a Life Member of ACM.
Lihua Yin born in 1973. Ph.D., professor, PHD supervisor, Guangzhou University. Member of China Computer Federation. Her current research interest is computer network and network security. E-mail: [email protected].
Zhihong Tian is currently a Professor, and Dean, with the Cyberspace Institute of Advanced Technology, Guangzhou University, Guangdong Province, China. Guangdong Province Universities and Colleges Pearl River Scholar (Distinguished Professor). He is also a part-time Professor at Carlton University, Ottawa, Canada. Previously, he served in different academic and administrative positions at the Harbin Institute of Technology, and China Academy of Engineering Physics. His research interests include computer networks and security. His research has been supported in part by the National Natural Science Foundation of China, National Key research and Development Plan of China, National High-tech R D Program of China (863 Program), National Basic Research Program of China (973 Program), and Postdoctoral Science Foundation of China. He served as a member, Chair, and General Chair of a number of international conferences. He is the director of China Cyberspace Security Association, Lingnan outstanding talents of Guangzhou, and Excellent experts in Guangzhou. He is a Senior Member of the China Computer Federation, and a Member of IEEE.
- ☆
This research is supported by the Guangdong Province Key Research and Development Plan (2019B010136003), the National Key research and Development Plan (2018YFB1004003), the National Natural Science Foundation of China (U1636215, 61871140, 61872100, U1803263), and Guangdong Province Universities and Colleges Pearl River Scholar Funded Scheme (2019).