Elsevier

Computer Networks

Volume 181, 9 November 2020, 107432
Computer Networks

Security of Mobile Multimedia Data:The Adversarial Examples for Spatio-temporal Data

https://doi.org/10.1016/j.comnet.2020.107432Get rights and content

Highlights

  • Research highlights item 1: We develop a Convolutional AutoEncoder (CAE) architecture to convert GPS trajectory data into image format data. We firstly extract four-channel features (speed, acceleration, jerk, bearing rate) from each trajectory segment. Then we train the CAE model to convert four-channel features into image format data.

  • Research highlights item 2: We utilize the image format data to train three trajectory mode classification models. In original trajectory data, each trajectory segment corresponds to a trajectory mode, including biking, walking, driving, etc. We use image format data of each trajectory segment to train three deep learning models (VGG16, VGG19, shallow convolutional neural networks).

  • Research highlights item 3: We adopt FGSM algorithm to attack the three trained models. In the test set, based on the gradient of the trained model, we use FGSM algorithm to generate adversarial examples. Moreover, we use the decoder to reconstruct the four-channel features from adversarial examples and original image format data. By visualizing the results, we find that there is no difference between adversarial examples and original data.

Abstract

With the rapid development of the internet, the number of mobile multimedia devices is increasing among people. As the popularity of these mobile devices, a large amount of multimedia data is produced. How to utilize these multimedia data is essential in the wireless communication area. In recent years, deep learning has achieved good performance in Multimedia data analysis. However, recent research shows that deep learning models are vulnerable to the crafted samples. The crafted samples are called adversarial examples. In the paper, we explore the security of deep learning in mobile multimedia computing systems. We use the Spatio-temporal data (GPS trajectory data) to conduct experiments. The deep learning networks attacked are trajectory mode classification models. Since both Spatio-temporal data and image data are continuous, we adopt the algorithms of Computer Vision (Fast Gradient Sign Method (FGSM)) to generate adversarial examples. We propose an Adversarial Sample generation model based on a Convolutional AutoEncoder (ASCAE). Firstly, we design a convolutional AutoEncoder (CAE) to convert trajectory data into image format data. Then, we utilize image format data to train trajectory mode classification models. Based on image format data and the trained models, we use FGSM algorithm to generate adversarial examples. Finally, we utilize adversarial examples to attack the trained models. Experiment results demonstrate that the adversarial examples successfully attack the deep learning models. Besides, the adversarial examples have good transferability. Therefore, in mobile multimedia computing systems, adversarial examples are a serious data security problem for deep learning models.

Introduction

Mobile multimedia devices combine computer and video technology. These devices are becoming more and more portable for people. They include smartphones, laptops, wireless sensors, etc [1], [2], [3], [4], [5]. Besides, they can also process multimedia data, such as audio, video, and images. With the popularity of these mobile devices, a large amount of multimedia data is generated. These multimedia data has great commercial value. Therefore, how to utilize these multimedia data is essential in the wireless communication area. Compared to previous desktop devices, mobile multimedia devices bring new challenges and opportunities.

In recent years, deep learning shows great potential in data analysis. It has made significant progress in Natural Language Process (NLP) and Computer Vision (CV) fields [6], [7], [8], [9], [10]. To better mine the value of multimedia data, deep learning is also applied in mobile multimedia computing systems. Some studies show that deep learning has achieved good performance in Multimedia data analysis. However, owing to upload data and process data at the cloud end, there is a certain delay in deep learning models. Fortunately, edge computing makes up for the shortcoming. Recently, some studies show that deep learning models are vulnerable to the crafted samples. The crafted samples are called adversarial examples. In image classification task, the small perturbations are barely recognized for human but can make deep learning models predict the wrong labels. Therefore, AI-based Security and Privacy in mobile multimedia computing systems need to draw significant attention.

In the paper, we conduct experiments to explore the security of mobile multimedia computing systems in deep learning modes. In multimedia data, we use the Spatio-temporal data (GPS trajectory data) to conduct experiments. The Spatio-temporal data contains longitude, latitude, and timestamp information. Specially, based on Spatio-temporal data, we generate adversarial examples to attack trajectory mode classification models. The attacked task is trajectory mode classification [11], [12], [13], [14], [15], [16], [17], [18]. In the task, the trajectory mode classification model takes the four-channel features of each trajectory segment as input and outputs the corresponding trajectory modes, including biking, walking, driving, etc. The model is deep learning networks.

To the best of our knowledge, the research of adversarial examples mainly focuses on Computer Vision (CV) and Natural Language Process (NLP). Since both Spatio-temporal data and image data are continuous, we try to adopt the algorithm of Computer Vision (Fast Gradient Sign Method (FGSM)) to generate adversarial examples. In detail, firstly, we extract four-channel features (speed, acceleration, jerk, bearing rate) from each trajectory segment. Then we use a Convolutional AutoEncoder (CAE) to convert four-channel features into image format data. Moreover, in trajectory mode classification task, we use the image format data to train the three deep learning models, including VGG16, VGG 19, shallow convolutional neural networks. Finally, based on image format data and trained models, we adopt FGSM algorithm to generate adversarial examples. We use adversarial examples to attack three deep learning models. Experimental results show that adversarial examples have strong attack ability to three deep models.

In summary, the main contribution of this paper is as follow:

  • We develop a CAE architecture to convert GPS trajectory data into image format data. We firstly extract four-channel features (speed, acceleration, jerk, bearing rate) from each trajectory segment. Then we train the CAE model to convert four-channel features into image format data.

  • We utilize image format data to train three trajectory mode classification models. In original trajectory data, each trajectory segment corresponds to a trajectory mode, including biking, walking, driving, etc. We use image format data of each trajectory segment to train three deep learning models (VGG16, VGG19, shallow convolutional neural networks).

  • We adopt FGSM algorithm to attack the three trained models. In the test set, based on the gradient of the trained model, we use FGSM algorithm to generate adversarial examples. Moreover, we use the decoder to reconstruct the four-channel features from adversarial examples and original image format data. By visualizing the results, we find that there is no difference between adversarial examples and original data.

  • We conduct extensive experiments on GPS trajectory datasets. The adversarial examples obviously reduce the accuracy of three deep models. Furthermore, the adversarial examples have good transferability.

The rest of this paper is organized as follows. In Section 2, we discuss related work. We introduce our ASCAE model architecture in Section 3. In Section 4, we discuss experimental results and Section 5 summarizes the paper.

Section snippets

Related work

In this section, we firstly discuss the related work on trajectory mode classification task. Then we summarize the literature work in adversarial examples. Finally we provide the motivations of our work.

Trajectory Mode Classification Task. The algorithms used for trajectory mode classification are mainly divided into two categories: traditional machine learning algorithms and deep learning algorithms. In traditional trajectory mode classification algorithms, firstly the features need to be

Model Architecture

In the section, we introduce the proposed model ASCAE. The model architecture is presented in Fig. 1. Firstly, we present several preliminaries. Secondly, we introduce the data preprocessing part. Thirdly, we show the attacked models. Finally, we describe the attack process and the attack algorithm.

Experiment

In the section, we firstly introduce our experiment settings. Then, we briefly present the evaluation metric, task descriptions and baseline models. Finally, we show the attack ability and transferability of adversarial examples by experiments.

Conclusion

In the paper, we conduct experiments to explore the security of mobile multimedia computing systems in deep learning modes. In multimedia data, we use the Spatio-temporal data (GPS trajectory data) to conduct experiments. Based on Spatio-temporal data, we generate adversarial examples to attack deep learning models. The deep learning models attacked are three trajectory mode classification models, including shallow convolutional neural networks, VGG16 networks, and VGG19 networks. Since both

CRediT authorship contribution statement

Yuanyuan Chen: Conceptualization, Methodology, Software, Formal analysis, Writing - original draft. Jing Qiu: Writing - original draft, Data curation. Xiaojiang Du: Funding acquisition, Writing - review & editing. Lihua Yin: Conceptualization, Formal analysis. Zhihong Tian: Data curation, Formal analysis, Writing - review & editing.

Declaration of Competing Interest

No conflict of interest.

Yuanyuan Chen is working toward the master’s degree at Guangzhou University. Her current research interests include spatio-temporal data and data mining. E-mail: [email protected].

References (39)

  • M. Sharif et al.

    Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition

    Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security

    (2016)
  • O. Kaoru et al.

    Deep learning for mobile multimedia

    ACM Transactions on Multimedia Computing Communications and Applications

    (2017)
  • A. Krikelis

    Location-dependent multimedia computing

    IEEE Concurrency

    (1999)
  • H. Chang et al.

    Adaptive bandwidth reservation mechanism using mobility probability in mobile multimedia computing environment

    Proceedings 25th Annual IEEE Conference on Local Computer Networks. LCN 2000

    (2000)
  • Z. Tian et al.

    Real time lateral movement detection based on evidence reasoning network for edge computing environment

    IEEE Transactions on Industrial Informatics

    (2019)
  • D. Wang et al.

    From iot to 5g i-iot: The next generation iot-based intelligent algorithms and 5g technologies

    IEEE Communications Magazine

    (2018)
  • C. Szegedy et al.

    Inception-v4

    Inception-ResNet and the Impact of Residual Connections on Learning

    (2016)
  • A. Vaswani

    Attention is all you need

    Advances in Neural Information Processing Systems

    (2017)
  • J. Gu et al.

    Multiple moving targets surveillance based on a cooperative network for multi-UAV

    IEEE Communications Magazine

    (2018)
  • Z. Tian et al.

    Vcash: A novel reputation framework for identifying denial of traffic service in internet of connected vehicles

    IEEE Internet of Things Journal.

    (2020)
  • X. Du et al.

    Defending dos attacks on broadcast authentication in wireless sensor networks

    Proc. of IEEE International Conference on Communications (ICC 2008)

    (2008)
  • Y. Zheng

    Trajectory data mining: an overview

    ACM Transactions on Intelligent Systems and Technology (TIST)

    (2015)
  • F. Biljecki et al.

    Transportation mode-based segmentation and classification of movement trajectories

    International Journal of Geographical Information Science

    (2013)
  • Y. Zheng et al.

    Understanding transportation modes based on GPS data for web applications

    ACM Transactions on the Web (TWEB)

    (2010)
  • Y. Zheng et al.

    Learning transportation mode from raw gps data for geographic applications on the web

    Proceedings of the 17th international conference on World Wide Web.

    (2008)
  • L. Xiao et al.

    Security in mobile edge caching with reinforcement learning

    IEEE Wireless Communications

    (2018)
  • L. Xiao et al.

    Cloud-based malware detection game for mobile devices with offloading

    IEEE Transactions on Mobile Computing

    (2017)
  • Z. Pan et al.

    Urban traffic prediction from spatio-temporal data using deep meta learning

    KDD

    (2019)
  • B. Yu et al.

    Spatio-temporal graph convolutional networks: A deep learning framework for traffic forecasting

    IJCAI

    (2018)
  • Cited by (3)

    • Seek-and-Hide: Adversarial Steganography via Deep Reinforcement Learning

      2022, IEEE Transactions on Pattern Analysis and Machine Intelligence

    Yuanyuan Chen is working toward the master’s degree at Guangzhou University. Her current research interests include spatio-temporal data and data mining. E-mail: [email protected].

    Jing Qiu received the Ph.D. degree in computer applications technology from Beijing Institute of Technology. She is currently a Professor of the Cyberspace Institute of Advanced Technology, Guangzhou University. She was a Visiting Scholar with the University of Southern California, LA, USA, under the supervision of Professor Craig A. Knoblock. Her current research interest is Information Extraction, Network Representation, and Big Data Analysis. E-mail: [email protected].

    Xiaojiang Du (S’99M’03SM’09F’20) received his B.S. and M.S. degree in Electrical Engineering (Automation Department) from Tsinghua University, Beijing, China in 1996 and 1998, respectively. He received his M.S. and Ph.D. degree in Electrical Engineering from the University of Maryland, College Park in 2002 and 2003, respectively. Dr. Du is a tenured Full Professor and the Director of the Security And Networking (SAN) Lab in the Department of Computer and Information Sciences at Temple University, Philadelphia, USA. His research interests are security, wireless networks, and systems. He has authored over 400 journal and conference papers in these areas, as well as a book published by Springer. Dr. Du has been awarded more than 6 million US Dollars research grants from the US National Science Foundation (NSF), Army Research Office, Air Force Research Lab, NASA, the State of Pennsylvania, and Amazon. He won the best paper award at IEEE ICC 2020 and IEEE GLOBECOM 2014, and the best poster runner-up award at the ACM MobiHoc 2014. He serves on the editorial boards of two international journals. Dr. Du served as the lead Chair of the Communication and Information Security Symposium of the IEEE International Communication Conference (ICC) 2015, and a Co-Chair of Mobile and Wireless Networks Track of IEEE Wireless Communications and Networking Conference (WCNC) 2015. He is (was) a Technical Program Committee (TPC) member of several premier ACM/IEEE conferences such as INFOCOM (2007 - 2020), IM, NOMS, ICC, GLOBECOM, WCNC, BroadNet, and IPCCC. Dr. Du is an IEEE Fellow and a Life Member of ACM.

    Lihua Yin born in 1973. Ph.D., professor, PHD supervisor, Guangzhou University. Member of China Computer Federation. Her current research interest is computer network and network security. E-mail: [email protected].

    Zhihong Tian is currently a Professor, and Dean, with the Cyberspace Institute of Advanced Technology, Guangzhou University, Guangdong Province, China. Guangdong Province Universities and Colleges Pearl River Scholar (Distinguished Professor). He is also a part-time Professor at Carlton University, Ottawa, Canada. Previously, he served in different academic and administrative positions at the Harbin Institute of Technology, and China Academy of Engineering Physics. His research interests include computer networks and security. His research has been supported in part by the National Natural Science Foundation of China, National Key research and Development Plan of China, National High-tech R D Program of China (863 Program), National Basic Research Program of China (973 Program), and Postdoctoral Science Foundation of China. He served as a member, Chair, and General Chair of a number of international conferences. He is the director of China Cyberspace Security Association, Lingnan outstanding talents of Guangzhou, and Excellent experts in Guangzhou. He is a Senior Member of the China Computer Federation, and a Member of IEEE.

    This research is supported by the Guangdong Province Key Research and Development Plan (2019B010136003), the National Key research and Development Plan (2018YFB1004003), the National Natural Science Foundation of China (U1636215, 61871140, 61872100, U1803263), and Guangdong Province Universities and Colleges Pearl River Scholar Funded Scheme (2019).

    View full text