Big Data constructed based on the advancement of distributed computing and virtualization is considered as the current emerging trends in Data Analytics. It is used for supporting potential utilization of computing resources focusing on, on-demand services and resource scalability. In particular, resource scheduling is considered as the process of resource distribution through an effective decision making process with the objective of facilitating required tasks over time. The incorporation of heterogeneous computing resources by the Big Data consumers also permits the option of reducing energy usage and enhanced resource efficiency. Further, optimal scheduling of resources is considered as an NP hard problem due to the dynamic characteristics of the resources and fluctuating users’ demand. In this paper, a Hybrid Gradient Descent Spider Monkey Optimization (HGDSMO) algorithm is proposed to efficient resource scheduling by handling the issues and challenges in the Hadoop heterogenous environment. The proposed HGDSMO algorithm uses the Gradient Descentand foraging and social behavior of the spider monkey optimization algorithm involved in the objective of effective resource allocation. It is designed as the efficient task scheduling approach that balances the load of the cloud by allocating them to appropriate VMs depending on their requirements. It is also proposed as a dynamic resource management scheme for efficiently allocating the cloud resources for effective execution of clients’ tasks. The simulation results of the proposed HGDSMO algorithm confirmed to be potent in throughput, load balancing and makespan compared to the baseline hybrid meta-heuristic resource allocation algorithms used for investigation.

中文翻译：

---

异构环境中大数据处理的高效资源调度的混合梯度下降蜘蛛猴优化（HGDSMO）算法

基于分布式计算和虚拟化的进步而构建的大数据被认为是数据分析的当前新兴趋势。它用于支持对按需服务和资源可伸缩性的计算资源的潜在利用。特别地，资源调度被认为是通过有效的决策过程进行资源分配的过程，目的是随着时间的推移促进所需的任务。大数据使用者将异构计算资源纳入其中也可以选择减少能源使用并提高资源效率。此外，由于资源的动态特性和用户需求的波动，资源的最佳调度被认为是NP难题。在本文中，提出了一种混合梯度下降蜘蛛猴优化（HGDSMO）算法，通过处理Hadoop异构环境中的问题和挑战来有效地进行资源调度。提出的HGDSMO算法利用蜘蛛猴优化算法的梯度下降和觅食以及社会行为参与有效资源分配的目的。它被设计为一种高效的任务调度方法，通过根据其需求将其分配给适当的VM来平衡云的负载。还提出了一种动态资源管理方案，用于有效分配云资源以有效执行客户端任务。所提出的HGDSMO算法的仿真结果被证实在吞吐量方面很有效，