Skip to main content
SpringerLink
Log in

Queue-priority optimized algorithm: a novel task scheduling for runtime systems of application integration platforms

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

The need for integration of applications and services in business processes from enterprises has increased with the advancement of cloud and mobile applications. Enterprises started dealing with high volumes of data from the cloud and from mobile applications, besides their own. This is the reason why integration tools must adapt themselves to handle with high volumes of data, and to exploit the scalability of cloud computational resources without increasing enterprise operations costs. Integration platforms are tools that integrate enterprises’ applications through integration processes, which are nothing but workflows composed of a set of atomic tasks connected through communication channels. Many integration platforms schedule tasks to be executed by computational resources through the First-in-first-out heuristic. This article proposes a Queue-priority algorithm that uses a novel heuristic and tackles high volumes of data in the task scheduling of integration processes. This heuristic is optimized by the Particle Swarm Optimization computational method. The results of our experiments were confirmed by statistical tests, and validated the proposal as a feasible alternative to improve integration platforms in the execution of integration processes under a high volume of data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://github.com/gca-research-group/simulation-queuepriory-pso.

References

  1. Abdulhamid S, Shafie AL, Idris I (2014) Tasks scheduling technique using league championship algorithm for Makespan minimization in IaaS cloud. J Eng Appl Sci 9(1):2528–2533

    Google Scholar 

  2. Alexander C, Ishikawa S, Silvertein M (1977) A pattern language: towns, buildings, construction. Oxford University Press, Oxford

    Google Scholar 

  3. Alkhanak EN, Lee SP, Rezaei R, Parizi RM (2016) Cost optimization approaches for scientific workflow scheduling in cloud and grid computing: a review, classifications, and open issues. J Syst Softw 113:1–26

    Article  Google Scholar 

  4. AlRashidi MR, El-Hawary ME (2009) A survey of particle swarm optimization applications in electric power systems. IEEE Trans Evolut Comput 13(4):913–918

    Article  Google Scholar 

  5. Anwar N, Deng H (2018) Elastic scheduling of scientific workflows under deadline constraints in cloud computing environments. Future Int 10(5):1–23

    Google Scholar 

  6. Aron R, Chana I, Abraham A (2015) A hyper-heuristic approach for resource provisioning-based scheduling in grid environment. J Supercomput 71(4):1427–1450

    Article  Google Scholar 

  7. Basili VR, Rombach D, Kitchenham KSB, Selby D, Pfahl RW (2007) Empirical software engineering issues. Springer, Berlin

    Google Scholar 

  8. Basu S, Karuppiah M, Selvakumar K, Li KC, Islam SKH, Hassan MM, Bhuiyan MZA (2018) An intelligentcognitive model of task scheduling for IoT applications in cloud computing environment. Future Gener Comput Syst 88:254–261

    Article  Google Scholar 

  9. Blythe J, Jain S, Deelman E, Gil Y, Vahi K, Mandal A, Kennedy K (2005) Task scheduling strategies for workflow-based applications in grids. Int Symp Cluster Comput Grid 2:759–767

    Google Scholar 

  10. Boehm M, Habich D, Preissler S, Lehner W, Wloka U (2011) Cost-based vectorization of instance-based integration processes. Inf Syst 36(1):3–29

    Article  Google Scholar 

  11. Bratton D, Kennedy J (2007) Defining a standard for particle swarm optimization. In: Swarm intelligence Symposium, pp 120–127

  12. Buyya R, Yeo CS, Venugopal S, Broberg J, Brandic I (2009) Cloud computing and emerging it platforms: vision, hype, and reality for delivering computing as the 5th utility. Future Gener Comput Syst 25(6):599–616

    Article  Google Scholar 

  13. Chhabra A, Oshin (2018) Hybrid psacga algorithm for job scheduling to minimize makespan in heterogeneous grids. In: Industry Interactive Innovations in Science, Engineering and Technology, pp 107–120

  14. Cruz CD (2006) Programa genes: estatística experimental e matrizes. Editora Universidade Federal de Viçosa

  15. Cruzes DS, Ben Othman L (2017) Threats to validity in empirical software security research. In: Empirical Research for Sof. Security, pp 295–320

  16. Eberhart R, Kennedy J (1995) Particle swarm optimization. In: Proceedings of the IEEE International Joint Conference on Neural Networks, pp 1942–1948

  17. Fan K, Zhai Y, Li X, Wang M (2018) Review and classification of hybrid shop scheduling. Prod Eng 12(5):597–609

    Article  Google Scholar 

  18. Feldt R, Magazinius A (2010) Validity threats in empirical software engineering research-an initial survey. In: International Conference on Software Engineering and Knowledge Engineering, pp 374–379

  19. Frantz RZ (2012) Enterprise application integration: an easy-to-maintain model-driven engineering approach. PhD thesis, University of Seville

  20. Frantz RZ, Corchuelo R, Molina-Jiménez C (2012) A proposal to detect errors in enterprise application integration solutions. J Syst Softw 85(3):480–497

    Article  Google Scholar 

  21. Frantz RZ, Corchuelo R, Roos-Frantz F (2016) On the design of a maintainable software development kit to implement integration solutions. J Syst Softw 111:89–104

    Article  Google Scholar 

  22. Freire DL, Frantz RZ, Roos-Frantz F, Sawicki S (2019) Survey on the run-time systems of enterprise application integration platforms focusing on performance. Softw Pract Exp 49(3):341–360

    Article  Google Scholar 

  23. Ghafouri R, Movaghar A, Mohsenzadeh M (2019) A budget constrained scheduling algorithm for executing workflow application in infrastructure as a service clouds. Peer-to-Peer Netw Appl 12(1):241–268

    Article  Google Scholar 

  24. Ghosh TK, Das S (2018) Job scheduling in computational grid using a hybrid algorithm based on particle swarm optimization and extremal optimization. J Inf Technol Res 11(4):72–86

    Article  Google Scholar 

  25. Guo F, Yu L, Tian S, Yu J (2015) A workflow task scheduling algorithm based on the resources’ fuzzy clustering in cloud computing environment. Int J Commun Syst 28(6):1053–1067

    Article  Google Scholar 

  26. Harman M, Lakhotia K, Singer J, White DR, Yoo S (2013) Cloud engineering is search based software engineering too. J Syst Softw 86(9):2225–2241

    Article  Google Scholar 

  27. Hilman MH, Rodriguez MA, Buyya R (2018) Multiple workflows scheduling in multi-tenant distributed systems: a taxonomy and future directions. ACM Comput Surv 1(1):1–33

    Article  Google Scholar 

  28. Hohpe G (2005) Your coffee shop doesn’t use two-phase commit [asynchronous messaging architecture]. IEEE Softw 22(2):64–66

    Article  Google Scholar 

  29. Hohpe G, Woolf B (2004) Enterprise integration patterns: designing, building, and deploying messaging solutions. Addison-Wesley Professional, Boston

    Google Scholar 

  30. Jedlitschka A, Pfahl D (2005) Reporting guidelines for controlled experiments in software engineering. In: International Symposium on Empirical Software Engineering, pp 95–104

  31. Jian C, Tao M, Wang Y (2014) A particle swarm optimisation algorithm for cloud-oriented workflow scheduling based on reliability. Int J Comput Appl Technol 50(3–4):220–225

    Article  Google Scholar 

  32. Konsek H (2013) Instant Apache ServiceMix How-to. Packt Publishing

  33. Kuhn R, Hanafee B, Allen J (2017) Reactive design patterns. Manning Publications Company

  34. Lin SW, Ying KC (2019) Makespan optimization in a no-wait flowline manufacturing cell with sequence-dependent family setup times. Comput Ind Eng 128(1):1–7

    Article  Google Scholar 

  35. Linthicum DS (2000) Enterprise application integration. Addison-Wesley Professional, Boston

    Google Scholar 

  36. Linthicum DS (2017) Cloud computing changes data integration forever: what’s needed right now. IEEE Cloud Comput 4(3):50–53

    Article  Google Scholar 

  37. Manasrah AM, Ali HB (2018) Workflow scheduling using hybrid GA-PSO algorithm in cloud computing. Wirel Commun Mobile Comput 2018:1–16

    Article  Google Scholar 

  38. Manikas K (2016) Revisiting software ecosystems research: a longitudinal literature study. J Syst Softw 117:84–103

    Article  Google Scholar 

  39. Pereira JL, o Varajão J, (2019) The temporal dimension of business processes: requirements and challenges. Int J Comput Appl Technol 59(1):74–81

  40. Pietri I, Chronis Y, Ioannidis Y (2019) Fairness in dataflow scheduling in the cloud. Inf Syst 83:118–125

    Article  Google Scholar 

  41. Pinto G, Castor F, Liu YD (2014) Understanding energy behaviors of thread management constructs. ACM SIGPLAN Not 49:345–360

    Article  Google Scholar 

  42. Qureshi K, Shah SMH, Manuel P (2011) Empirical performance evaluation of schedulers for cluster of workstations. Cluster Comput 14(2):101–113

    Article  Google Scholar 

  43. Ritter D, May N, Rinderle-Ma S (2017) Patterns for emerging application integration scenarios: a survey. Inf Syst 67:36–57

    Article  Google Scholar 

  44. Ritter D, Rinderle-Ma S, Montali M, Rivkin A, Sinha A (2018) Formalizing application integration patterns. In: International Enterprise Distributed Object Computing Conference, pp 11–20

  45. Ritter D, Rinderle-Ma S, Montali M, Rivkin A (2019) Formal foundations for responsible application integration. Inf Syst, p 101439

  46. Rodriguez MA, Buyya R (2014) Deadline based resource provisioning and scheduling algorithm for scientific workflows on clouds. Trans Cloud Comput 2(2):222–235

    Article  Google Scholar 

  47. Rodriguez MA, Buyya R (2018) Scheduling dynamic workloads in multi-tenant scientific workflow as a service platforms. Future Gener Comput Syst 79:739–750

    Article  Google Scholar 

  48. Russell J, Cohn R (2012) Jitterbit integration server. Book on Demand

  49. Saifullah A, Li J, Agrawal K, Lu C, Gill C (2013) Multi-core real-time scheduling for generalized parallel task models. Real-Time Syst 49(4):404–435

    Article  Google Scholar 

  50. Shoukry A, Khader J, Gani S (2019) Improving business process and functionality using IoT based E3-value business model. Electron Mark 1:1–10

    Google Scholar 

  51. Stavrinides GL, Karatza HD (2018) A hybrid approach to scheduling real-time IoT workflows in fog and cloud environments. Multimed Tools Appl 78:24639–24655

    Article  Google Scholar 

  52. Sun D, Yan H, Gao S, Liu X, Buyya R (2018) Rethinking elastic online scheduling of big data streaming applications over high-velocity continuous data streams. J Supercomput 74(2):615–636

    Article  Google Scholar 

  53. Surhone LM, Timpledon MT, Marseken SF (2010) Petals ESB. Betascript Publishing

  54. Thakur V, Kumar S (2018) A pragmatic study and analysis of load balancing techniques in parallel computing. In: Information and Decision Sciences, pp 447–45400

  55. Verma A, Kaushal S (2015) Cost minimized pso based workflow scheduling plan for cloud computing. Int J Inf Technol Comput Sci 7(8):37–43

    Google Scholar 

  56. Wang C, Zhang L, Liu C (2018) Adaptive scheduling method for dynamic robotic cell based on pattern classification algorithm. Int J Model Simul Sci Comput 9(5):1850040–1–1850040–18

  57. Wohlin C, Runeson P, Höst M, Ohlsson MC, Regnell B, Wesslén A (2012) Experimentation in soft. engineering. Springer, Berlin

  58. Wood DC, Forman EH (1971) Throughput measurement using a synthetic job stream. In: Fall Joint Computer Conference, pp 51–56

  59. Xie Y, Zhu Y, Wang Y, Cheng Y, Xu R, Sani AS, Yuan D, Yang Y (2019) A novel directional and non-local-convergent particle swarm optimization based workflow scheduling in cloud-edge environment. Future Gener Comput Syst 97:36–378

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the Research Support Foundation of the State of Rio Grande do Sul (FAPERGS) under grant 17/2551-0001206-2; and the National Council for Scientific and Technological Development (CNPq) under grant 309315/2020-4. We would like to thank Dra. Maria do Rosário Laureano and Dr. Sancho Oliveira from the Instituto Universitário de Lisboa (Portugal) for their helpful comments in earlier versions of this article.

Author information

Authors and Affiliations

  1. Instituto Universitário de Lisboa (ISCTE-IUL) ISTAR-IUL, Lisboa, Portugal

    Daniela L. Freire & Vitor Basto-Fernandes

  2. Department of Exact Sciences and Engineering, Unijuí University, Ijuí, Brazil

    Rafael Z. Frantz & Fabricia Roos-Frantz

Authors
  1. Daniela L. Freire
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rafael Z. Frantz
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fabricia Roos-Frantz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Vitor Basto-Fernandes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rafael Z. Frantz.

Ethics declarations

Conflict of Interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1: Additional metrics

Appendix 1: Additional metrics

The averages of processed messages in the execution of the Coffee shop integration process is our case study (CS) are shown in Figs. 7 and 8. The averages of remained messages in the execution of the CS are shown in Figs. 9 and 10. The makespan in the execution of the CS is shown in Figs. 11 and 12.

Fig. 7
figure 7

Average processed messages—low workload

Full size image
Fig. 8
figure 8

Average processed messages—high workload

Full size image

As for the number of messages processed, the FIFO heuristic performance degrades from 1,500,000 messages when it only processes approximately 35% of the total messages (Fig. 8). Figure 10 shows that 1,359,833 messages remained accumulated in the integration process, without processing, in the case of the FIFO heuristic. Figure 12 shows that from a workload of 1,500,000 messages arriving in a continuous flow in the integration process, the heuristic consumes all the simulation time and cannot fully process the messages that entered the integration process. The most important metric is the throughput, which shows the processing rate of the heuristics, cf. Fig. 6. For this metric, in all workloads, the qPrior heuristic performed better than FIFO, and with the workload of 1,500,000 messages, FIFO’s performance was only 7% of the performance obtained by qPrior. Thus, the qPrior heuristic is an option for scenarios of low workloads (<1,500,000 messages) and which continues to perform well for high volumes of messages (\(\ge 1,500,000\) messages). At the same time, FIFO is only an option for low workloads and still under-performing.

Fig. 9
figure 9

Average remained messages—low workload

Full size image
Fig. 10
figure 10

Average remained messages—high workload

Full size image
Fig. 11
figure 11

Average makespan—low workload

Full size image
Fig. 12
figure 12

Average makespan—high workload

Full size image

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Freire, D.L., Frantz, R.Z., Roos-Frantz, F. et al. Queue-priority optimized algorithm: a novel task scheduling for runtime systems of application integration platforms. J Supercomput 78, 1501–1531 (2022). https://doi.org/10.1007/s11227-021-03926-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-03926-x

Keywords

Search

Navigation