Skip to main content

Advertisement

SpringerLink
Log in

Using memetic algorithm for robustness testing of contract-based software models

  • Published:
Artificial Intelligence Review Aims and scope Submit manuscript

Abstract

Graph Transformation System (GTS) can formally specify the behavioral aspects of complex systems through graph-based contracts. Test suite generation under normal conditions from GTS specifications is a task well-suited to evolutionary algorithms such as Genetic and Particle Swarm Optimization (PSO) metaheuristics. However, testing the vulnerabilities of a system under unexpected events such as invalid inputs is essential. Furthermore, the mentioned global search algorithms tend to make big jumps in the system’s state-space that are not concentrated on particular test goals. In this paper, we extend the HGAPSO approach into a cost-aware Memetic Algorithm (MA) by making small local changes through a proposed local search operator to optimize coverage score and testing costs. Moreover, we test GTS specifications not only under normal events but also under unexpected situations. So, three coverage-based testing strategies are investigated, including normal testing, robustness testing, and a hybrid strategy. The effectiveness of the proposed test generation algorithm and the testing strategies are evaluated through a type of mutation analysis at the model-level. Our experimental results show that (1) the hybrid testing strategy outperforms normal and robustness testing strategies in terms of fault-detection capability, (2) the robustness testing is the most cost-efficient strategy, and (3) the proposed MA with the hybrid testing strategy outperforms the state-of-the-art global search algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  • Aichernig BK, Lorber F, Nickovic D (2013) Model-based mutation testing with timed automata. Graz University of Technology, Graz

    MATH  Google Scholar 

  • Anand S et al (2013) An orchestrated survey of methodologies for automated software test case generation. J Syst Softw 86(8):1978–2001

    Article  Google Scholar 

  • Araujo W, Briand LC, Labiche Y (2014) On the effectiveness of contracts as test oracles in the detection and diagnosis of functional faults in concurrent object-oriented software. IEEE Trans Software Eng 40(10):971–992

    Article  Google Scholar 

  • Arqub OA (2017) Adaptation of reproducing kernel algorithm for solving fuzzy Fredholm–Volterra integrodifferential equations. Neural Comput Appl 28(7):1591–1610

    Article  Google Scholar 

  • Arqub OA et al (2016) Numerical solutions of fuzzy differential equations using reproducing kernel Hilbert space method. Soft Comput 20(8):3283–3302

    Article  Google Scholar 

  • Arqub OA et al (2017) Application of reproducing kernel algorithm for solving second-order, two-point fuzzy boundary value problems. Soft Comput 21(23):7191–7206

    Article  Google Scholar 

  • Balci O (1998) Verification, validation, and testing. Handbook of Simulation 10:335–393

    Article  Google Scholar 

  • Belli F et al (2016) Model-based mutation testing—approach and case studies. Sci Comput Program 120:25–48

    Article  Google Scholar 

  • Böhmer K, Rinderle-Ma S (2015) A systematic literature review on process model testing: approaches, challenges, and research directions. ArXiv preprint arXiv:1509.04076

  • de Bruijn V (2013) Model-based testing with graph grammars, Master's thesis, University of Twente, Enschede, The Netherlands

  • Di Nardo D et al (2015) Evolutionary robustness testing of data processing systems using models and data mutation (T). In: 2015 30th IEEE/ACM international conference on automated software engineering (ASE). IEEE

  • Dias Neto AC et al (2007) A survey on model-based testing approaches: a systematic review. In: Proceedings of the 1st ACM international workshop on empirical assessment of software engineering languages and technologies: held in conjunction with the 22nd IEEE/ACM international conference on automated software engineering (ASE) 2007. ACM

  • Eberhart R, Kennedy J (1995) Particle swarm optimization. In: Proceedings of the IEEE international conference on neural networks. Citeseer

  • Ehrig H, Ehrig K (2006) Overview of formal concepts for model transformations based on typed attributed graph transformation. Electron Notes Theor Comput Sci 152:3–22

    Article  Google Scholar 

  • Ehrig H, Rozenberg G, Kreowski H-J (1999) Handbook of graph grammars and computing by graph transformation, vol 3. World Scientific, Singapore

    Book  Google Scholar 

  • Ehrig H, Prange U, Taentzer G (2004) Fundamental theory for typed attributed graph transformation. In: International conference on graph transformation. Springer

  • Engels G, Güldali B, Lohmann M (2006) Towards model-driven unit testing. In: International conference on model driven engineering languages and systems. Springer

  • Ferrer J et al (2015) Search based algorithms for test sequence generation in functional testing. Inf Softw Technol 58:419–432

    Article  Google Scholar 

  • Fraser G, Arcuri A (2013) Whole test suite generation. IEEE Trans Software Eng 39(2):276–291

    Article  Google Scholar 

  • Fraser G, Arcuri A, McMinn P (2015) A memetic algorithm for whole test suite generation. J Syst Softw 103:311–327

    Article  Google Scholar 

  • Gönczy L, Heckel R, Varró D (2007) Model-based testing of service infrastructure components. In: Testing of software and communicating systems. Springer, pp 155–170

  • Grechanik M, Devanla G (2016) Mutation integration testing. In: 2016 IEEE international conference on software quality, reliability and security (QRS). IEEE

  • Harman M, Jia Y, Zhang Y (2015) Achievements, open problems and challenges for search based software testing. In: 2015 IEEE 8th international conference on software testing, verification and validation (ICST). IEEE

  • Haupt RL, Ellen Haupt S (2004) Practical genetic algorithms, 2nd edition. Wiley, New Jersey

  • Heckel R (2006) Graph transformation in a nutshell. Electron Notes Theor Comput Sci 148(1):187–198

    Article  Google Scholar 

  • Heckel R, Mariani L (2004) Component integration testing by graph transformations. In: International conference on computer science, software engineering, information technology, e-business, and applications, Cairo. Citeseer

  • Heckel R, Khan TA, Machado R (2011) Towards test coverage criteria for visual contracts. In: Proceedings of the Tenth International Workshop on Graph Transformation and Visual Modeling Techniques, Berlin

  • Hutchison C et al (2018) Robustness testing of autonomy software. In: Proceedings of the 40th international conference on software engineering: software engineering in practice. ACM

  • ISC Committee (1990) IEEE standard glossary of software engineering terminology (IEEE Std 610.12-1990). Los Alamitos, CA: IEEE Computer Society, 169

  • Jonsson MBB, Leucker J-PKM, Pretschner A (2005) Model-based testing of reactive systems. Springer, Berlin

    MATH  Google Scholar 

  • Kalaee A, Rafe V (2018) Model-based test suite generation for graph transformation system using model simulation and search-based techniques. Inf Softw Technol 108:1-29

    Article  Google Scholar 

  • Khan TA, Runge O, Heckel R (2012) Testing against visual contracts: model-based coverage. In: International conference on graph transformation. Springer

  • Khari M, Kumar P (2017) An extensive evaluation of search-based software testing: a review. Soft Comput 23(6):1933–1946

    Article  Google Scholar 

  • König B et al (2018) A tutorial on graph transformation. In: Graph transformation, specifications, and nets. Springer, pp 83–104

  • Liu S (2016) Validating formal specifications using testing-based specification animation. In: Proceedings of the 4th FME workshop on formal methods in software engineering. ACM

  • Lobo FG, Lima CF (2007) Adaptive population sizing schemes in genetic algorithms. In: Parameter setting in evolutionary algorithms. Springer, pp 185–204

  • Lobo F, Lima CF, Michalewicz Z (2007) Parameter setting in evolutionary algorithms, vol 54. Springer, Berlin

    Book  Google Scholar 

  • Louzaoui K, Benlhachmi K (2017) A robustness testing approach for an object oriented model. JCP 12(4):335–353

    Article  Google Scholar 

  • Ma Y-S, Offutt J, Kwon Y-R (2006) MuJava: a mutation system for Java. In: Proceedings of the 28th international conference on software engineering. ACM

  • Machado R, Ribeiro L, Heckel R (2015) Rule-based transformation of graph rewriting rules: towards higher-order graph grammars. Theoret Comput Sci 594:1–23

    Article  MathSciNet  Google Scholar 

  • Mattiello-Francisco F et al (2012) InRob: an approach for testing interoperability and robustness of real-time embedded software. J Syst Softw 85(1):3–15

    Article  Google Scholar 

  • Meyer B (1988) Object-oriented software construction, vol 2. Prentice Hall, New York

    Google Scholar 

  • Offutt AJ (1992) Investigations of the software testing coupling effect. ACM Trans Softw Eng Methodol (TOSEM) 1(1):5–20

    Article  Google Scholar 

  • Rafe V (2013) Scenario-driven analysis of systems specified through graph transformations. J Vis Lang Comput 24(2):136–145

    Article  Google Scholar 

  • Rafe V, Rahmani M, Rashidi K (2013) A survey on coping with the state space explosion problem in model checking. Int Res J Appl Basic Sci 4(6):1379–1384

    Google Scholar 

  • Rensink A (2003) The GROOVE simulator: a tool for state space generation. In: International workshop on applications of graph transformations with industrial relevance. Springer

  • Runge O, Khan TA, Heckel R (2013) Test case generation using visual contracts. In: Proceedings of the Tenth International Workshop on Graph Transformation and Visual Modeling Techniques, Berlin

  • Satpathy M et al (2007) Automatic testing from formal specifications. In: International conference on tests and proofs. Springer

  • Savary A et al (2015) Model-based robustness testing in event-B using mutation. In: SEFM 2015 collocated workshops. Springer

  • Schmidt A, Varró D (2003). CheckVML: a tool for model checking visual modeling languages. In: International conference on the unified modeling language. Springer

  • Strug J (2016) Mutation testing approach to negative testing. J Eng

  • Taentzer G (2003) AGG: a graph transformation environment for modeling and validation of software. In: International workshop on applications of graph transformations with industrial relevance. Springer

  • Vos TE et al (2013) Evolutionary functional black-box testing in an industrial setting. Software Qual J 21(2):259–288

    Article  MathSciNet  Google Scholar 

  • Whitley D, Gordon VS, Mathias K (1994) Lamarckian evolution, the Baldwin effect and function optimization. In: International conference on parallel problem solving from nature. Springer

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Faculty of Engineering, Arak University, Arak, 3815688349, Iran

    Anvar Bahrampour & Vahid Rafe

Authors
  1. Anvar Bahrampour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vahid Rafe
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vahid Rafe.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bahrampour, A., Rafe, V. Using memetic algorithm for robustness testing of contract-based software models. Artif Intell Rev 54, 877–915 (2021). https://doi.org/10.1007/s10462-020-09881-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10462-020-09881-y

Keywords

Search

Navigation