Skip to main content
SpringerLink
Log in

Contactless Continuous Activity Recognition based on Meta-Action Temporal Correlation in Mobile Environments

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

Continuous activity recognition (CAR) plays an important role in human daily indoor activity monitoring and can be widely used in smart home, human-computer interaction and user authentication. Due to the privacy issue and limited coverage of video signals, RF-based CAR has attracted more and more attention in recent years. This paper focuses on three key problems in RF-based CAR: denoising, segmentation and recognition. We present the design and implementation of a contactless and sensorless continuous activity recognition system, namely WiCheck. Our basic idea is to utilize the temporal correlation between two adjacent actions in continuous activity to eliminate the cumulative error in continuous activity segmentation. Firstly, the multi-layer optimized noise elimination method is used to decrease the environment interference. Secondly, a method based on dual-swing window is proposed to reduce the cumulative error of continuous activity segmentation. Finally, WiCheck is implemented in different indoor environments, and 6 continuous activity sequences are designed to evaluate and analyze the influencing factors. The continuous activity recognition accuracy of WiCheck to two actions and three actions can approach 90% and 75%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Ali A, Aggarwal J (2001) Segmentation and recognition of continuous human activity. In: Proceedings IEEE Workshop on Detection and Recognition of Events in Video, pp 28–35. https://doi.org/10.1109/event.2001.938863

  2. Davies L, Gather U (1993) The identification of multiple outliers. J Am Stat Assoc 88(423):782–792. https://doi.org/10.1080/01621459.1993.10476339

    Article  MathSciNet  Google Scholar 

  3. Deng S, Xiang Z, Zhao P, Taheri J, Gao H, Yin J, Zomaya AY (2020) Dynamical resource allocation in edge for trustable iot systems: a reinforcement learning method. IEEE Transactions on Industrial Informatics:1–1. https://doi.org/10.1109/TII.2020.2974875

  4. Ding H, Shangguan L, Yang Z, Han J, Zhou Z, Yang P, Xi W, Zhao J (2015) Femo:a platform for free-weight exercise monitoring with rfids. In: Proceedings of ACM Conference on Embedded Networked Sensor Systems, pp 141–154. https://doi.org/10.1145/2809695.2809708

  5. Ehatisham-ul-Haq M, Azam MA, Naeem U, Amin Y, Loo J (2018) Continuous authentication of smartphone users based on activity pattern recognition using passive mobile sensing. J Netw Comput Appl 109:24–35. https://doi.org/10.1016/j.jnca.2018.02.020

  6. Gao H, Duan Y, Shao L, Sun X (2019) Transformation-based processing of typed resources for multimedia sources in the iot environment. Wireless Networks:1–17. https://doi.org/10.1007/s11276-019-02200-6

  7. Gao H, Xu Y, Yin Y, Zhang W, Li R, Wang X (2019) Context-aware qos prediction with neural collaborative filtering for internet-of-things services. IEEE Internet of Things Journal:1–1. https://doi.org/10.1109/JIOT.2019.2956827

  8. Gjengset J, Xiong J, Mcphillips G, Jamieson K (2014) Phaser: enabling phased array signal processing on commodity wifi access points. In: Proceedings of International Conference on Mobile Computing and NETWORKING, pp 153–164. https://doi.org/10.1145/2639108.2639139

  9. Honghao Gao Wanqiu Huang Y.D. (2020) The cloud-edge based dynamic reconfiguration to service workflow for mobile ecommerce environments: A qos prediction perspective. ACM Transactions on Internet Technology. https://doi.org/10.1145/3391198

  10. Honghao Gao Can Liu YLXY (2020) V2vr: Reliable hybrid-network-oriented v2v data transmission and routing considering rsus and connectivity probability. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/TITS.2020.2983835

  11. Honghao Gao Li Kuang YYBGKD (2020) Mining consuming behaviors with temporal evolution for personalized recommendation in mobile marketing apps ACM/springer Mobile Networks and Applications (MONET). https://doi.org/10.1007/s11036-020-01535-1

  12. Hsu CY, Ahuja A, Yue S, Hristov R, Kabelac Z, Katabi D (2017) Zero-effort in-home sleep and insomnia monitoring using radio signals. Proc ACM Interact Mob Wearab Ubiquit Technol 1(3):59. https://doi.org/10.1145/3025453.3025937

    Article  Google Scholar 

  13. Hsu CY, Liu Y, Kabelac Z, Hristov R, Katabi D, Liu C (2017) Extracting gait velocity and stride length from surrounding radio signals. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp 2116–2126. https://doi.org/10.1145/3025453.3025937

  14. Kuang L, Gong T, Ouyang S, Gao H, Deng S (2020) Offloading decision methods for multiple users with structured tasks in edge computing for smart cities. Futur Gener Comput Syst 105:717–729. https://doi.org/10.1016/j.future.2019.12.039

    Article  Google Scholar 

  15. Lea C, Flynn MD, Vidal R, Reiter A, Hager GD (2017) Temporal convolutional networks for action segmentation and detection. In: proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 156–165. https://doi.org/10.1109/cvpr.2017.113

  16. Lei P, Todorovic S (2018) Temporal deformable residual networks for action segmentation in videos. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 6742–6751. https://doi.org/10.1109/CVPR.2018.00705

  17. Lin W, Xing S, Nan J, Wenyuan L, Binbin L (2019) Concurrent recognition of cross-scale activities via sensorless sensing. IEEE Sens J 19(2):658–669. https://doi.org/10.1109/JSEN.2018.2874506

    Article  Google Scholar 

  18. Liu J, Wang Y, Chen Y, Yang J, Chen X, Cheng J (2015) Tracking vital signs during sleep leveraging off-the-shelf wifi. In: Proceedings of the 16th ACM International Symposium on Mobile Ad Hoc Networking and Computing, pp 267–276. https://doi.org/10.1145/2746285.2746303

  19. Ni B, Yang X, Gao S (2016) Progressively parsing interactional objects for fine grained action detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1020–1028. https://doi.org/10.1109/cvpr.2016.116

  20. Qian K, Wu C, Yang Z, Liu Y, Zhou Z (2014) Pads: Passive detection of moving targets with dynamic speed using phy layer information. In: 2014 20Th IEEE international conference on parallel and distributed systems (ICPADS), pp 1–8

  21. Qian K, Wu C, Yang Z, Yang C, Liu Y (2016) Decimeter level passive tracking with wifi. In: Proceedings of the 3rd Workshop on Hot Topics in Wireless, pp 44–48. https://doi.org/10.1145/2980115.2980131

  22. Qian K, Wu C, Zhou Z, Zheng Y, Yang Z, Liu Y (2017) Inferring motion direction using commodity wi-fi for interactive exergames. In: Proceedings of the CHI Conference on Human Factors in Computing Systems, pp 1961–1972. https://doi.org/10.1145/3025453.3025678

  23. Stiefmeier T, Roggen D, Troster G (2007) Fusion of string-matched templates for continuous activity recognition. In: Proceedings of the 11th IEEE International Symposium on Wearable Computers, pp 41–44. https://doi.org/10.1109/iswc.2007.4373775

  24. Tan S, Yang J (2016) Wifinger: leveraging commodity wifi for fine-grained finger gesture recognition. In: Proceedings of the 17th ACM international symposium on mobile ad hoc networking and computing, pp 201–210. https://doi.org/10.1145/2942358.2942393

  25. Tan Z, Xu L, Zhong W, Guo X, Wang G (2018) Online activity recognition and daily habit modeling for solitary elderly through indoor position-based stigmergy. Eng Appl Artif Intell 76:214–225. https://doi.org/10.1016/j.engappai.2018.08.009

    Article  Google Scholar 

  26. Wang H, Zhang D, Wang Y, Ma J, Wang Y, Li S (2017) Rt-fall: a real-time and contactless fall detection system with commodity wifi devices. IEEE Trans Mob Comput 16(2):511–526. https://doi.org/10.1109/tmc.2016.2557795

    Article  Google Scholar 

  27. Wang W, Liu AX, Shahzad M (2016) Gait recognition using wifi signals. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing, pp 363– 373

  28. Wang W, Liu AX, Shahzad M, Ling K, Lu S (2017) Device-free human activity recognition using commercial wifi devices. IEEE J Sel Areas Commun. 35(5):1118–1131. https://doi.org/10.1109/jsac.2017.2679658

    Article  Google Scholar 

  29. Wenyuan L, Siyang W, Lin W, Binbin L, Xing S, Nan J (2018) From lens to prism: Device-free modeling and recognition of multi-part activities. IEEE Access 6:36,271–36,282. https://doi.org/10.1109/access.2018.2850534

    Article  Google Scholar 

  30. Wu X, Chu Z, Yang P, Xiang C, Zheng X, Huang W (2019) Tw-see: Human activity recognition through the wall with commodity wi-fi devices. IEEE Trans Veh Technol 68(1):306–319

    Article  Google Scholar 

  31. Xu M, Gao M, Chen YT, Davis LS, Crandall DJ (2019) Temporal recurrent networks for online action detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp 5532–5541. https://doi.org/10.1109/ICCV.2019.00563

  32. Yan Z, Subbaraju V, Chakraborty D, Misra A, Aberer K (2012) Energy-efficient continuous activity recognition on mobile phones: an activity-adaptive approach. In: 2012 16Th international symposium on wearable computers, pp 17–24. https://doi.org/10.1109/tvt.2018.2878754

  33. Yu J, Li J, Yu Z, Huang Q (2019) Multimodal transformer with multi-view visual representation for image captioning. IEEE Transactions on Circuits and Systems for Video Technology, pp 1–1

  34. Yu J, Tan M, Zhang H, Tao D, Rui Y (2019) Hierarchical deep click feature prediction for fine-grained image recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence:1–1. https://doi.org/10.1109/TPAMI.2019.2932058

  35. Yu J, Zhu C, Zhang J, Huang Q, Tao D (2020) Spatial pyramid-enhanced netvlad with weighted triplet loss for place recognition. IEEE Trans Neural Netw 31(2):661–674. https://doi.org/10.1109/TNNLS.2019.2908982

    Article  Google Scholar 

  36. Yu N, Wang W, Liu AX, Kong L (2018) Qgesture: Quantifying gesture distance and direction with wifi signals. Proc ACM Interact Mob Wearable Ubiquit Technol 2(1):51. https://doi.org/10.1145/3191783

    Article  Google Scholar 

  37. Zhang H, Zhou W, Parker LE (2014) Fuzzy segmentation and recognition of continuous human activities. In: 2014 IEEE International conference on robotics and automation (ICRA), pp 6305–6312. https://doi.org/10.1109/icra.2014.6907789

  38. Zheng X, Wang J, Shangguan L, Zhou Z, Liu Y (2017) Design and implementation of a csi-based ubiquitous smoking detection system. IEEE/ACM Transactions on Networking. https://doi.org/10.1109/tnet.2017.2752367

  39. Zhu G, Zhang L, Shen P, Song J (2016) An online continuous human action recognition algorithm based on the kinect sensor. Sensors 16(2):161. https://doi.org/10.3390/s16020161

    Article  Google Scholar 

  40. Zhuo Y, Zhu H, Xue H, Chang S (2017) Perceiving accurate csi phases with commodity wifi devices. In: IEEE Conference on computer communications, pp 1–9. https://doi.org/10.1109/infocom.2017.8056964

Download references

Acknowledgments

Supported by NSFC under Grant 61772453, 61672448, the Natural Science Foundation of Hebei Province under Grant F2018203444, F2016203176, Overseas Students Science and Technology Activities Project Merit Funding of Hebei Province under Grant CL201625, and the Youth Fundation in Basic Research of Yanshan University under Grant 16LGA009.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Yanshan University, 438 Hebei Avenue, Qinhuangdao, People’s Republic of China

    Lin Wang, Hecheng Su & Nan Jing

  2. Beijing Institute of Computer Technology and Application, 52 Yongding Road, Beijing, People’s Republic of China

    Xing Su

Authors
  1. Lin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xing Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hecheng Su
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nan Jing
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lin Wang.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, L., Su, X., Su, H. et al. Contactless Continuous Activity Recognition based on Meta-Action Temporal Correlation in Mobile Environments. Mobile Netw Appl 25, 2174–2190 (2020). https://doi.org/10.1007/s11036-020-01658-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-020-01658-5

Keywords

Search

Navigation