Extensive usage of Internet based applications in day to day life has led to generation of huge amounts of data every minute. Apart from humans, data is generated by machines like sensors, satellite, CCTV etc. This huge collection of heterogeneous data is often referred as Big Data which can be processed to draw useful insights. Apache Hadoop has emerged has widely used open source software framework for Big Data Processing and it is a cluster of cooperative computers enabling distributed parallel processing. Hadoop Distributed File System is used to store data blocks replicated and spanned across different nodes. HDFS uses an AES based cryptographic techniques at block level which is transparent and end to end in nature. However cryptography provides security from unauthorized access to the data blocks, but a legitimate user can still harm the data. One such example was execution of malicious map reduce jar files by legitimate user which can harm the data in the HDFS. We developed a mechanism where every map reduce jar will be tested by our sandbox security to ensure the jar is not malicious and suspicious jar files are not allowed to process the data in the HDFS. This feature is not present in the existing Apache Hadoop framework and our work is made available in github for consideration and inclusion in the future versions of Apache Hadoop.

基于Internet的应用程序在日常生活中的广泛使用已导致每分钟生成大量数据。除人类外，数据还由传感器，卫星，CCTV等机器生成。这种庞大的异构数据集合通常被称为大数据，可以对其进行处理以获取有用的见解。Apache Hadoop的出现已经广泛用于大数据处理的开源软件框架，它是协作计算机的集群，支持分布式并行处理。Hadoop分布式文件系统用于存储跨不同节点复制和跨越的数据块。HDFS在块级别使用基于AES的加密技术，该技术是透明的并且本质上是端对端的。但是，加密技术可防止未经授权访问数据块，从而提供安全性，但是合法用户仍然可以损害数据。一个这样的例子是合法用户执行恶意的map reduce jar文件，这可能会损害HDFS中的数据。我们开发了一种机制，将通过沙盒安全性对每个map reduce罐进行测试，以确保该罐不是恶意的，并且不允许可疑罐文件处理HDFS中的数据。现有的Apache Hadoop框架中没有此功能，我们的工作可在github中获得，以供考虑并包含在Apache Hadoop的未来版本中。我们开发了一种机制，将通过沙盒安全性对每个map reduce罐进行测试，以确保该罐不是恶意的，并且不允许可疑罐文件处理HDFS中的数据。现有的Apache Hadoop框架中没有此功能，我们的工作可在github中获得，以供考虑并包含在Apache Hadoop的未来版本中。我们开发了一种机制，将通过沙盒安全性对每个map reduce罐进行测试，以确保该罐不是恶意的，并且不允许可疑罐文件处理HDFS中的数据。该功能在现有的Apache Hadoop框架中不存在，我们的工作在github中可用，以供考虑并包含在Apache Hadoop的未来版本中。