当前位置: X-MOL 学术J. Big Data › 论文详情
Our official English website, www.x-mol.net, welcomes your feedback! (Note: you will need to create a separate account there.)
IoT Big Data provenance scheme using blockchain on Hadoop ecosystem
Journal of Big Data ( IF 8.1 ) Pub Date : 2021-08-30 , DOI: 10.1186/s40537-021-00505-y
Houshyar Honar Pajooh 1 , Mohammed A. Rashid 1 , Fakhrul Alam 1, 2 , Serge Demidenko 1, 2
Affiliation  

The diversity and sheer increase in the number of connected Internet of Things (IoT) devices have brought significant concerns associated with storing and protecting a large volume of IoT data. Storage volume requirements and computational costs are continuously rising in the conventional cloud-centric IoT structures. Besides, dependencies of the centralized server solution impose significant trust issues and make it vulnerable to security risks. In this paper, a layer-based distributed data storage design and implementation of a blockchain-enabled large-scale IoT system are proposed. It has been developed to mitigate the above-mentioned challenges by using the Hyperledger Fabric (HLF) platform for distributed ledger solutions. The need for a centralized server and a third-party auditor was eliminated by leveraging HLF peers performing transaction verifications and records audits in a big data system with the help of blockchain technology. The HLF blockchain facilitates storing the lightweight verification tags on the blockchain ledger. In contrast, the actual metadata are stored in the off-chain big data system to reduce the communication overheads and enhance data integrity. Additionally, a prototype has been implemented on embedded hardware showing the feasibility of deploying the proposed solution in IoT edge computing and big data ecosystems. Finally, experiments have been conducted to evaluate the performance of the proposed scheme in terms of its throughput, latency, communication, and computation costs. The obtained results have indicated the feasibility of the proposed solution to retrieve and store the provenance of large-scale IoT data within the Big Data ecosystem using the HLF blockchain. The experimental results show the throughput of about 600 transactions, 500 ms average response time, about 2–3% of the CPU consumption at the peer process and approximately 10–20% at the client node. The minimum latency remained below 1 s however, there is an increase in the maximum latency when the sending rate reached around 200 transactions per second (TPS).



中文翻译:

在 Hadoop 生态系统上使用区块链的物联网大数据来源方案

连接物联网 (IoT) 设备数量的多样性和急剧增加带来了与存储和保护大量物联网数据相关的重大问题。在传统的以云为中心的物联网结构中,存储量要求和计算成本不断上升。此外,集中式服务器解决方案的依赖性带来了严重的信任问题,使其容易受到安全风险的影响。在本文中,提出了基于区块链的大规模物联网系统的基于层的分布式数据存储设计和实现。它的开发旨在通过将 Hyperledger Fabric (HLF) 平台用于分布式账本解决方案来缓解上述挑战。通过利用 HLF 节点在区块链技术的帮助下在大数据系统中执行交易验证和记录审计,消除了对中央服务器和第三方审计师的需求。HLF 区块链有助于在区块链分类账上存储轻量级验证标签。相比之下,实际元数据存储在链下大数据系统中,以减少通信开销并增强数据完整性。此外,已经在嵌入式硬件上实现了一个原型,展示了在物联网边缘计算和大数据生态系统中部署所提议解决方案的可行性。最后,进行了实验以评估所提出方案的吞吐量、延迟、通信和计算成本方面的性能。获得的结果表明所提出的解决方案在使用 HLF 区块链的大数据生态系统中检索和存储大规模物联网数据的来源的可行性。实验结果表明,大约 600 个事务的吞吐量,500 毫秒的平均响应时间,对等进程的 CPU 消耗大约为 2-3%,客户端节点大约为 10-20%。最小延迟保持在 1 秒以下,但是当发送速率达到每秒 200 个事务 (TPS) 左右时,最大延迟会增加。对等进程大约占 CPU 消耗的 2-3%,在客户端节点大约占 10-20%。最小延迟保持在 1 秒以下,但是当发送速率达到每秒 200 个事务 (TPS) 左右时,最大延迟会增加。对等进程大约占 CPU 消耗的 2-3%,在客户端节点大约占 10-20%。最小延迟保持在 1 秒以下,但是当发送速率达到每秒 200 个事务 (TPS) 左右时,最大延迟会增加。

更新日期:2021-08-31
down
wechat
bug