Skip to main content
SpringerLink
Log in

RFnet: Automatic Gesture Recognition and Human Identification Using Time Series RFID Signals

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

Utilizing wireless signals for gesture recognition and human identification is an emerging type of technology for touchless user interface, which allows the computer to automatically identify the user and interpret his/her gestures as commands. Such techniques extract features to profile the fluctuation of time series wireless signals to infer human gestures/identities. Among which, device-free approach becomes more attractive because it does not need human to carry or wear sensing devices. Existing device-free solutions, though yielding good performance, require heavy crafting on data preprocessing and feature extraction. In this paper, we propose RFnet, a multi-branch 1D-CNN based framework, that explores the possibility of directly utilizing time series RFID signal to recognize static/dynamic gestures as well as the identity of users, which can benefit a large number of applications such as smart homes where security is also a prior concern. We conduct extensive experiments in three different environments. The results demonstrate the superior effectiveness of the proposed RFnet framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. American sign language (2019) https://www.nidcd.nih.gov/health/american-sign-language

  2. Impinj Application Note - Low Level User Data Support

  3. Leap motion (2017) https://www.vicon.com

  4. X-Box Kinect (2017) https://www.xbox.com

  5. Adib F, Kabelac Z, Katabi D, Miller RC (2014) 3D tracking via body radio reflections. In: Proceedings of USENIX NSDI. https://doi.org/10.5555/2616448.2616478

  6. Deng S, Xiang Z, Zhao P, Taheri J, Gao H, Yin J (2020) Dynamical resource allocation in edge for trustable iot systems: a reinforcement learning method. IEEE Transactions on Industrial Informatics. https://doi.org/10.1109/TII.2020.2974875

  7. Ding H, Han J, Shangguan L, Xi W, Jiang Z, Yang Z, Zhou Z, Yang P, Zhao J (2017) A platform for free-weight exercise monitoring with RFIDs. IEEE Trans Mob Comput 16(12):3279–3293. https://doi.org/10.1109/TMC.2017.2691705

    Article  Google Scholar 

  8. Ding H, Qian C, Han J, Wang G, Xi W, Zhao K, Zhao J (2017) RFIpad: enabling cost-efficient and device-free in-air handwriting using passive tags. In: Proceedings of IEEE ICDCS

  9. Ding H, Qian C, Han J, Xiao J, Zhang X, Wang G, Xi W, Zhao J (2019) Close-proximity detection for hand approaching using backscatter communication. IEEE Trans Mob Comput 18(10):2285–2297. https://doi.org/10.1109/TMC.2018.2872558

    Article  Google Scholar 

  10. Gao H, Duan Y, Shao L, Sun X (2019) Transformation-based processing of typed resources for multimedia sources in the IoT environment. Wirel Netw. https://doi.org/10.1007/s11276-019-02200-6

  11. Gao H, Huang W, Duan Y (2020) The cloud-edge based dynamic reconfiguration to service workflow for mobile ecommerce environments: a QoS prediction perspective. ACM Trans Internet Technol. https://doi.org/10.1145/3391198

  12. Gao H, Li J, Yin Y, Guo B, Dou K (2020) Mining consuming behaviors with temporal evolution for personalized recommendation in mobile marketing apps. ACM/Springer Mobile Networks and Applications (MONET). https://doi.org/10.1007/s11036-020-01535-1

  13. Gao H, Liu C, Li Y, Yang X (2020) V2VR: reliable hybrid-network-oriented V2V data transmission and routing considering RSUs and connectivity probability. IEEE Trans Intell Transp Syst. https://doi.org/10.1109/TITS.2020.2983835

  14. Gao H, Xu Y, Yin Y, Zhang W, Li R, Wang X (2019) Context-aware QoS prediction with neural collaborative filtering for internet-of-things services. IEEE Internet of Things Journal. https://doi.org/10.1109/JIOT.2019.2956827

  15. Guo X, Liu J, Chen Y (2017) Fitcoach: virtual fitness coach empowered by wearable mobile devices. In: Proceedings of IEEE INFOCOM

  16. Han J, Ding H, Qian C, Xi W, Wang Z, Jiang Z, Shangguan L, Zhao J (2016) CBID: a customer behavior identification system using passive tags. IEEE/ACM Trans Networking 24 (5):2885–2898. https://doi.org/10.1109/TNET.2015.2501103

    Article  Google Scholar 

  17. Hou J, Li XY, Zhu P, Wang Z, Wang Y, Qian J, Yang P (2019) Signspeaker: a real-time, high-precision smartwatch-based sign language translator. In: Proceedings of ACM MobiCom

  18. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of IEEE CVPR

  19. Kuang L, Yan X, Tan X, Li S, Yang X (2019) Predicting taxi demand based on 3D convolutional neural network and multi-task learning. Remote Sens 11(11):1265. https://doi.org/10.3390/rs11111265

    Article  Google Scholar 

  20. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of IEEE ICCV

  21. Shangguan L, Zhou Z, Jamieson K (2017) Enabling gesture-based interactions with objects. In: Proceedings of ACM MobiSys

  22. Song J, Sörös G, Pece F, Fanello SR, Izadi S, Keskin C, Hilliges O (2014) In-air gestures around unmodified mobile devices. In: Proceedings of ACM UIST

  23. Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M (2014) Striving for simplicity: the all convolutional net. arXiv:1412.6806

  24. Taylor J, Bordeaux L, Cashman T, et al. (2016) Efficient and precise interactive hand tracking through joint, continuous optimization of pose and correspondences. ACM Trans Graph 35(4):143. https://doi.org/10.1145/2897824.2925965

    Article  Google Scholar 

  25. Wang G, Cai H, Qian C, Han J, Li X, Ding H, Zhao J (2018) Towards replay-resilient RFID authentication. In: Proceedings of ACM Mobicom

  26. Wang J, Vasisht D, Katabi D (2014) RF-IDRaw: virtual touch screen in the air using RF signals. In: Proceedings of ACM SIGCOMM

  27. Wen H, Ramos Rojas J, Dey AK (2016) Serendipity: finger gesture recognition using an off-the-shelf smartwatch. In: Proceedings of ACM CHI

  28. Yang L, Lin Q, Li X, Liu T, Liu Y (2015) See through walls with COTS RFID system!. In: Proceedings of ACM MobiCom

  29. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: European conference on computer vision. Springer

  30. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proceedings of Springer CVPR

  31. Zhang C, Tabor J, Zhang J, Zhang X (2015) Extending mobile interaction through near-field visible light sensing. In: Proceedings of ACM Mobicom

  32. Zhao C, Li Z, Liu T, Ding H, Han J, Xi W, Gui R (2019) RF-Mehndi: a fingertip profiled RF identifier. In: Proceedings of IEEE INFOCOM

  33. Zhao C, Li Z, Liu T, Ding H, Han J, Xi W, Gui R (2019) RF-Mehndi: a fingertip profiled RF identifier. In: Proceedings of IEEE INFOCOM. IEEE

  34. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of IEEE CVPR

Download references

Acknowledgements

This work was supported by National Key R&D Program of China 2020YFB1707700, NSFC Grant No. 61832008, 61802299, 61772413, 61802291, Project funded by China Postdoctoral Science Foundation No. 2018M643663.

Author information

Authors and Affiliations

  1. Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an, China

    Han Ding, Lei Guo, Cui Zhao, Fei Wang, Ge Wang, Wei Xi & Jizhong Zhao

  2. Xidian University, Xi’an, China

    Zhiping Jiang

Authors
  1. Han Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cui Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ge Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zhiping Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Wei Xi
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jizhong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Han Ding.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, H., Guo, L., Zhao, C. et al. RFnet: Automatic Gesture Recognition and Human Identification Using Time Series RFID Signals. Mobile Netw Appl 25, 2240–2253 (2020). https://doi.org/10.1007/s11036-020-01659-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-020-01659-4

Keywords

Search

Navigation