Skip to main content

Advertisement

SpringerLink
Log in

A reinforcement learning algorithm for rescheduling preempted tasks in fog nodes

  • Published:
Journal of Scheduling Aims and scope Submit manuscript

Abstract

The fog server in a fog computing paradigm extends cloud services to latency-sensitive tasks by employing fog nodes (FNs) near user devices. The resource-constrained FNs face the challenge of meeting stringent deadlines of latency-sensitive tasks. The completion deadline of such tasks becomes critical on preemption. Task preemption is unavoidable in uncertain events, such as FN hostility, overloading, and mobility of the host FN or the user device. Rescheduling the task that is likely to face preemption is a better solution than terminating it. This paper proposes a rescheduling algorithm for the fog server to reschedule preempted tasks to FNs that can serve them to completion within their expected time. The rescheduling algorithm aims to attain a rescheduling list that guarantees the task deadline requirements. The brain-inspired rescheduling decision-making (BIRD) algorithm proposed in this paper uses the actor-critic reinforcement learning method for rescheduling preempted tasks to FNs. It mimics the decision-making model of the human brain to control voluntary motor activity. It guarantees the deadline requirement of the preempted task by ensuring the optimal performance of the FN through load balancing while rescheduling the preempted tasks to FNs. Experimental evaluation shows that the BIRD algorithm offers better FN selection than other scheduling policies such as first come first served (FCFS), greedy task allocation, task allocation based on least laxity, shortest job first (SJF), and earliest deadline first (EDF).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • GWA-T-12 Bitbrainsdataset. http://gwa.ewi.tudelft.nl/datasets/gwa-t-12-bitbrains. Accessed: 2020-04-21.

  • Abdel-Basset, M., Mohamed, R., Elhoseny, M., Bashir, A.K., Jolfaei, A., & Kumar, N. (2020). Energy-aware marine predators algorithm for task scheduling in iot-based fog computing applications. IEEE Transactions on Industrial Informatics.

  • Adhikari, M., Mukherjee, M., & Srirama, S.N. (2019). Dpto: A deadline and priority-aware task offloading in fog computing framework leveraging multi-level feedback queueing. IEEE Internet of Things Journal.

  • Berns, G.S., & Sejnowski, T.J. (1996). How the basal ganglia make decisions. In: Neurobiology of decision-making, pp. 101–113. Springer.

  • Bhatia, M., Sood, S. K., & Kaur, S. (2019). Quantum-based predictive fog scheduler for iot applications. Computers in Industry, 111, 51–67.

    Article  Google Scholar 

  • Bitam, S., Zeadally, S., & Mellouk, A. (2018). Fog computing job scheduling optimization based on bees swarm. Enterprise Information Systems, 12(4), 373–397.

    Article  Google Scholar 

  • Calabresi, P., Picconi, B., Tozzi, A., Ghiglieri, V., & Di Filippo, M. (2014). Direct and indirect pathways of basal ganglia: A critical reappraisal. Nature Neuroscience, 17(8), 1022–1030.

    Article  Google Scholar 

  • Chekired, D. A., Khoukhi, L., & Mouftah, H. T. (2018). Industrial iot data scheduling based on hierarchical fog computing: A key for enabling smart factory. IEEE Transactions on Industrial Informatics, 14(10), 4590–4602.

    Article  Google Scholar 

  • Crites, R.H., & Barto, A.G. (1995) An actor/critic algorithm that is equivalent to q-learning. In: Advances in Neural Information Processing Systems, pp. 401–408.

  • Ghanavati, S., Abawajy, J., & Izadi, D. (2020). Automata-based dynamic fault tolerant task scheduling approach in fog computing. IEEE Transactions on Emerging Topics in Computing.

  • Hafaiedh, I. B., & Slimane, M. B. (2020). A parameterized formal model for the analysis of preemption-threshold scheduling in real-time systems. IEEE Access, 8, 58180–58193.

    Article  Google Scholar 

  • He, Z., Zhang, Y., Tak, B., & Peng, L. (2019). Green fog planning for optimal internet-of-thing task scheduling. IEEE Access.

  • Hosseinioun, P., Kheirabadi, M., Tabbakh, S.R.K., & Ghaemi, R. (2020). A new energy-aware tasks scheduling approach in fog computing using hybrid meta-heuristic algorithm. Journal of Parallel and Distributed Computing.

  • Houk, J.C., & Adams, J.L. (1995). 13 a model of how the basal ganglia generate and use neural signals that. Models of information processing in the basal ganglia, p. 249.

  • Hussein, M. K., & Mousa, M. H. (2020). Efficient task offloading for iot-based applications in fog computing using ant colony optimization. IEEE Access, 8, 37191–37201.

    Article  Google Scholar 

  • Ikemoto, S., Glazier, B. S., Murphy, J. M., & McBride, W. J. (1997). Role of dopamine d1 and d2 receptors in the nucleus accumbens in mediating reward. Journal of Neuroscience, 17(21), 8580–8587.

    Article  Google Scholar 

  • Jie, Y., Tang, X., Choo, K. K. R., Su, S., Li, M., & Guo, C. (2018). Online task scheduling for edge computing based on repeated stackelberg game. Journal of Parallel and Distributed Computing, 122, 159–172.

    Article  Google Scholar 

  • Joel, D., Niv, Y., & Ruppin, E. (2002). Actor-critic models of the basal ganglia: New anatomical and computational perspectives. Neural Networks, 15(4–6), 535–547.

    Article  Google Scholar 

  • Konda, V. R., & Borkar, V. S. (1999). Actor-critic-type learning algorithms for markov decision processes. SIAM Journal on control and Optimization, 38(1), 94–123.

    Article  Google Scholar 

  • Lera, I., Guerrero, C., & Juiz, C. (2018). Availability-aware service placement policy in fog computing based on graph partitions. IEEE Internet of Things Journal, 6(2), 3641–3651.

    Article  Google Scholar 

  • Li, H., Ota, K., & Dong, M. (2019). Deep reinforcement scheduling for mobile crowdsensing in fog computing. ACM Transactions on Internet Technology (TOIT), 19(2), 1–18.

    Article  Google Scholar 

  • Luo, J., Yin, L., Hu, J., Wang, C., Liu, X., Fan, X., & Luo, H. (2019). Container-based fog computing architecture and energy-balancing scheduling algorithm for energy iot. Future Generation Computer Systems, 97, 50–60.

    Article  Google Scholar 

  • Ma, P. Y. R., et al. (1982). A task allocation model for distributed computing systems. IEEE Transactions on Computers, 100(1), 41–47.

    Google Scholar 

  • Mahmud, R., Ramamohanarao, K., & Buyya, R. (2020). Application management in fog computing environments: A taxonomy, review and future directions. ACM Computing Surveys (CSUR), 53(4), 1–43.

    Article  Google Scholar 

  • Mahmud, R., Srirama, S. N., Ramamohanarao, K., & Buyya, R. (2020). Profit-aware application placement for integrated fog-cloud computing environments. Journal of Parallel and Distributed Computing, 135, 177–190.

    Article  Google Scholar 

  • Nair, B., & Somasundaram, M. S. B. (2019). Overload prediction and avoidance for maintaining optimal working condition in a fog node. Computers and Electrical Engineering, 77, 147–162.

    Article  Google Scholar 

  • Oueis, J., Strinati, E.C., & Barbarossa, S. (2015). The fog balancing: Load distribution for small cell cloud computing. In: 2015 IEEE 81st vehicular technology conference (VTC spring), pp. 1–6. IEEE.

  • Pham, X. Q., Man, N. D., Tri, N. D. T., Thai, N. Q., & Huh, E. N. (2017). A cost-and performance-effective approach for task scheduling based on collaboration between cloud and fog computing. International Journal of Distributed Sensor Networks, 13(11), 1550147717742073.

    Article  Google Scholar 

  • Rahbari, D., & Nickray, M. (2020). Task offloading in mobile fog computing by classification and regression tree. Peer-to-Peer Networking and Applications, 13(1), 104–122.

    Article  Google Scholar 

  • Rivest, F., Bengio, Y., & Kalaska, J. (2004). Brain inspired reinforcement learning. Advances in Neural Information Processing Systems, 17, 1129–1136.

    Google Scholar 

  • Sun, Y., Lin, F., & Xu, H. (2018). Multi-objective optimization of resource scheduling in fog computing using an improved nsga-ii. Wireless Personal Communications, 102(2), 1369–1385.

    Article  Google Scholar 

  • Suri, R. E., & Schultz, W. (1999). A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task. Neuroscience, 91(3), 871–890.

    Article  Google Scholar 

  • Sutton, R.S., & Barto, A.G. (2018). Reinforcement learning: An introduction. MIT press.

  • Trantham-Davidson, H., Neely, L. C., Lavin, A., & Seamans, J. K. (2004). Mechanisms underlying differential d1 versus d2 dopamine receptor regulation of inhibition in prefrontal cortex. Journal of Neuroscience, 24(47), 10652–10659.

    Article  Google Scholar 

  • Tychalas, D., & Karatza, H. (2020). A scheduling algorithm for a fog computing system with bag-of-tasks jobs: Simulation and performance evaluation. Simulation Modelling Practice and Theory, 98, 101982.

    Article  Google Scholar 

  • Wan, J., Chen, B., Wang, S., Xia, M., Li, D., & Liu, C. (2018). Fog computing for energy-aware load balancing and scheduling in smart factory. IEEE Transactions on Industrial Informatics, 14(10), 4548–4556.

    Article  Google Scholar 

  • Wang, J. X., Kurth-Nelson, Z., Kumaran, D., Tirumala, D., Soyer, H., Leibo, J. Z., et al. (2018). Prefrontal cortex as a meta-reinforcement learning system. Nature Neuroscience, 21(6), 860–868.

    Article  Google Scholar 

  • Wang, K., Tan, Y., Shao, Z., Ci, S., & Yang, Y. (2019). Learning-based task offloading for delay-sensitive applications in dynamic fog networks. IEEE Transactions on Vehicular Technology, 68(11), 11399–11403.

    Article  Google Scholar 

  • Wang, S., Zhao, T., & Pang, S. (2020). Task scheduling algorithm based on improved firework algorithm in fog computing. IEEE Access, 8, 32385–32394.

    Article  Google Scholar 

  • Xu, J., Hao, Z., Zhang, R., & Sun, X. (2019). A method based on the combination of laxity and ant colony system for cloud-fog task scheduling. IEEE Access, 7, 116218–116226.

    Article  Google Scholar 

  • Yin, L., Luo, J., & Luo, H. (2018). Tasks scheduling and resource allocation in fog computing based on containers for smart manufacturing. IEEE Transactions on Industrial Informatics, 14(10), 4712–4721.

    Article  Google Scholar 

  • Yu, M., Liu, A., Xiong, N.N., & Wang, T. (2020). An intelligent game based offloading scheme for maximizing benefits of iot-edge-cloud ecosystems. IEEE Internet of Things Journal.

  • Zeng, D., Gu, L., Guo, S., Cheng, Z., & Yu, S. (2016). Joint optimization of task scheduling and image placement in fog computing supported software-defined embedded system. IEEE Transactions on Computers, 65(12), 3702–3712.

    Article  Google Scholar 

  • Zhu, X., Luo, Y., Liu, A., Bhuiyan, M. Z. A., & Zhang, S. (2020). Multiagent deep reinforcement learning for vehicular computation offloading in iot. IEEE Internet of Things Journal, 8(12), 9763–9773.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology, Tiruchirappalli, Tamil Nadu, India

    Biji Nair & S. Mary Saira Bhanu

Authors
  1. Biji Nair
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Mary Saira Bhanu
    View author publications

    You can also search for this author in PubMed Google Scholar

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nair, B., Bhanu, S.M.S. A reinforcement learning algorithm for rescheduling preempted tasks in fog nodes. J Sched 25, 547–565 (2022). https://doi.org/10.1007/s10951-022-00725-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10951-022-00725-x

Keywords

Search

Navigation