Skip to main content
SpringerLink
Log in

Secure and efficient big data deduplication in fog computing

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

With the rapid development of the Internet of Things, the massive amount of big data generated by the Internet of Things terminals and the real-time processing requirements have brought enormous challenges. A two-tier computing model consisting solely of two entities, cloud and user, will not be sufficient to support processing large numbers of concurrent data requests. Therefore, fog computing was proposed. How to realize the secure and efficient deduplication of ciphertext in fog computing has become a new research topic. In this paper, we firstly present a new decentralized deduplication structure and then show how to apply it to construct a secure and efficient big data deduplication scheme in fog computing. The cloud server, in the proposed paper, can quickly determine which fog server needs to be traversed to search duplicate data, and instead of traversing all fog servers. This significantly improves the efficiency of big data deduplication in fog computing. Furthermore, the proposed scheme allows fog server to verify whether the user possesses the ownership of the data. Performance analysis and experimental results show the proposed scheme has less overheads than existing schemes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Boneh D, Gentry C, Lynn B et al (2003) Aggregate and verifiably encrypted signatures from bilinear maps. In: International conference on the theory and applications of cryptographic techniques. Springer, Berlin, Heidelberg, pp 416–432

  • Cui H, Deng RH, Li Y et al (2017) Attribute-based storage supporting secure deduplication of encrypted data in cloud. IEEE Trans Big Data. https://doi.org/10.1109/TBDATA.2017.2656120

  • Di Pietro R, Sorniotti A (2016) Proof of ownership for deduplication systems: a secure, scalable, and efficient solution. Comput Commun 82:71–82

    Article  Google Scholar 

  • Douceur JR, Adya A, Bolosky WJ et al (2002) Reclaiming space from duplicate files in a serverless distributed file system. In: Proceedings 22nd international conference on distributed computing systems. IEEE, pp 617–624

  • Fu Y, Xiao N, Jiang H et al (2017) Application-aware big data deduplication in cloud environment. IEEE Trans Cloud Comput. https://doi.org/10.1109/TCC.2017.2710043

  • Gou Z, Yamaguchi S, Gupta BB (2017) Analysis of various security issues and challenges in cloud computing environment: a survey. In: Identity theft: breakthroughs in research and practice. IGI Global, pp 221–247

  • Gupta BB, Yamaguchi S, Agrawal DP (2018) Advances in security and privacy of multimedia big data in mobile and cloud computing. Multim Tools Appl 77(7):9203–9208

    Article  Google Scholar 

  • Halevi S, Harnik D, Pinkas B et al (2011) Proofs of ownership in remote storage systems. In: Proceedings of the 18th ACM conference on computer and communications security. ACM, pp 491–500

  • Jiang T, Chen X, Wu Q et al (2017) Secure and efficient cloud data deduplication with randomized tag. IEEE Trans Inf Forensics Secur 12(3):532–543

    Article  Google Scholar 

  • Koo D, Hur J (2018) Privacy-preserving deduplication of encrypted data with dynamic ownership management in fog computing. Future Gener Comput Syst 78:739–752

    Article  Google Scholar 

  • Koo D, Shin Y, Yun J et al (2016) A hybrid deduplication for secure and efficient data outsourcing in fog Computing. In: 2016 IEEE international conference on cloud computing technology and science (CloudCom). IEEE, pp 285–293

  • Kwon H, Hahn C, Kim D et al (2017) Secure deduplication for multimedia data with user revocation in cloud storage. Multim Tools Appl 76(4):5889–5903

    Article  Google Scholar 

  • Mishra S, Singh S, Ali ST (2018) MPoWS: merged proof of ownership and storage for block level deduplication in cloud storage. In: 2018 9th International conference on computing, communication and networking technologies (ICCCNT). IEEE, pp 1–7

  • Ni J, Zhang K, Yu Y et al (2018) Providing task allocation and secure deduplication for mobile crowdsensing via fog computing[J]. IEEE Trans Dependable Secure Comput. https://doi.org/10.1109/TDSC.2018.2791432

  • Pooranian Z, Chen KC, Yu CM et al (2018) RARE: defeating side channels based on data-deduplication in cloud storage. In: IEEE INFOCOM 2018-IEEE conference on computer communications workshops (INFOCOM WKSHPS). IEEE, pp 444–449

  • Shin Y, Koo D, Hur J et al (2017a) Secure proof of storage with deduplication for cloud storage systems. Multim Tools Appl 76(19):19363–19378

    Article  Google Scholar 

  • Shin Y, Koo D, Yun J et al (2017b) Decentralized server-aided encryption for secure deduplication in cloud storage. IEEE Trans Serv Comput. https://doi.org/10.1109/TSC.2017.2748594

  • Stanek J, Kencl L (2018) Enhanced secure thresholded data deduplication scheme for cloud storage. IEEE Trans Dependable Secure Comput 15(4):694–707

    Article  Google Scholar 

  • Stergiou C, Psannis KE, Kim BG et al (2018a) Secure integration of IoT and cloud computing. Future Gener Comput Syst 78:964–975

    Article  Google Scholar 

  • Stergiou C, Psannis KE, Xifilidis T et al (2018) Security and privacy of big data for social networking services in cloud. In:IEEE INFOCOM 2018-IEEE conference on computer communications workshops (INFOCOM WKSHPS). IEEE, pp 438–443

  • Xiong J, Zhang Y, Lin L et al (2017) ms-PoSW: a multi-server aided proof of shared ownership scheme for secure deduplication in cloud. Concurr Comput: Pract Exp. https://doi.org/10.1002/cpe.4252

  • Yang X, Lu R, Choo KKR et al (2017) Achieving efficient and privacy-preserving cross-domain big data deduplication in cloud. IEEE Trans Big Data. https://doi.org/10.1109/TBDATA.2017.2721444

  • Yaseen Q, Aldwairi M, Jararweh Y et al (2018) Collusion attacks mitigation in internet of things: a fog based model. Multim Tools Appl 77(14):18249–18268

    Article  Google Scholar 

  • Yu CM, Gochhayat SP, Conti M et al (2018) Privacy aware data deduplication for side channel in cloud storage. IEEE Trans Cloud Comput. https://doi.org/10.1109/TCC.2018.2794542

  • Zhang Y, Xu C, Li H et al (2018) Healthdep: an efficient and secure deduplication scheme for cloud-assisted ehealth systems. IEEE Trans Ind Inf 14(9):4101–4112

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 61070164 and Grant 61272415, in part by the Natural Science Foundation of Guangdong Provience, China, under Grant S2012010008767, in part by the Science and Technology Planning Project of Guangdong Provience, China, under Grant 2013B010401015, and in part by the Zhuhai Top Discipline-Information Security.

Author information

Authors and Affiliations

  1. The Department of Computer Science, Jinan University, Guangzhou, 510632, China

    Jiajun Yan, Xiaoming Wang, Qingqing Gan, Suyu Li & Daxin Huang

Authors
  1. Jiajun Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoming Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qingqing Gan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Suyu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daxin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaoming Wang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Communicated by B. B. Gupta.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, J., Wang, X., Gan, Q. et al. Secure and efficient big data deduplication in fog computing. Soft Comput 24, 5671–5682 (2020). https://doi.org/10.1007/s00500-019-04215-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-019-04215-9

Keywords

Search

Navigation