Skip to main content
SpringerLink
Log in

LSTM-based deep learning for spatial–temporal software testing

  • Published:
Distributed and Parallel Databases Aims and scope Submit manuscript

Abstract

Continuous integration (CI) software development practice has become more and more popular. Regression testing occurs very frequently in CI. Test case suites constantly change since new test cases are inserted and obsolete test case are removed in each cycle. The software developer hunts for quick-feedback of faults because of time constraint. An embedded software usually includes the spatial–temporal data in CI. The efficiency of regression testing for the embedded software is related to the space–time. To achieve ideal regression testing goals for the embedded software in CI, this paper proposes a novel test case prioritization approach using LSTM-Based (Long short-term memory) deep learning. LSTM is a time series prediction model. It can predict the probability of each test case detection fault in the next cycle according to the testing history information of all the previous CI cycles. The priority of test case can be obtained dynamically under the guidance of the probability. The experiments are conducted on two industrial data sets. The results verify that compared with some exiting test case prioritization approaches, our approach has better performance for embedded software as follows: (1) improve the prioritization effectiveness, (2) increase the fault detection rate in CI environment, and (3) decrease the testing execution time through automatic reduction the obsolete test cases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. http: //martinfowler.com/articles/continuousIntegration.html

  2. Orso, A., Rothermel, G.: Software testing: a research travelogue (2000–2014). In Proc. Future Softw. Eng. 117–132 (2014)

  3. Stolberg S.: Enabling agile testing through continuous integration. In Agile Conf.360–374 (2009)

  4. Rothermel, G., Untch, R., Chu, C., et al.: Prioritizing test cases for regression testing. IEEE Transact. Softw. Eng. 27(10), 929–948 (2001)

    Article  Google Scholar 

  5. https://www.51testing.com/html/48/n-3721848.html

  6. Rothermel, G., Harrold, M.: A safe, efficient regression test selection technique. ACM Trans. Softw. Eng. Method. 6(2), 173–210 (1997)

    Article  Google Scholar 

  7. Wong, W., Hogan. J., London S., et al.: A study of effective regression testing in practice. Proceedings of IEEE international symposium on software reliability engineering. 264–274 (1997)

  8. Yoo, S., Harman, M.: Regression testing minimization, selection and prioritization: a survey. Softw.Test. Verif. Reliab. 22(2), 67–120 (2015)

    Article  Google Scholar 

  9. Regehr, J., Chen, Y., Cuoq, P., et al.: Test-case reduction for c compiler bugs. ACM SIGPLAN Notices 47(6), 335–346 (2012)

    Article  Google Scholar 

  10. Rothermel, G., Untch, R.H., Chu, C., Harrold, M.J.: Test case prioritization: an empirical study. In: Proc. Int. Conf. Softw. Maint. 179–188 (1999)

  11. Spieker, H., Gotlieb, A, et al.: Reinforcement learning for automatic test case prioritization and selection in continuous integration. ACM Sigsoft Int. Symp. Softw. Test. Anal. 12–22 (2017)

  12. Jones, J., Harrold, M.: Test-Suite reduction and prioritization for modified condition/decision coverage. IEEE Trans. on Softw. Eng. 29(3), 195–209 (2003)

    Article  Google Scholar 

  13. Kim, J., Porter, A.: A history-based test prioritization technique for regression testing in resource constrained environments. Int. Conf. Softw. Eng. IEEE, 119–129 (2002)

  14. Marijan, D., Gotlieb, A., Sen, S.: Test case prioritization for continuous regression testing: an industrial case study. IEEE Int. Conf. Soft. Maint. 540–543 (2103)

  15. Carlson, R., Do, H., Denton, A.: A clustering approach to improving test case prioritization: an industrial case study. IEEE Int. Conf. Softw. Maint. 382–391 (2011)

  16. Busjaeger, B., Xie, T.: Learning for test prioritization: an industrial case study. ACM Sigsoft Int. Symp. Foundat Softw. Eng. 975–980 (2016).

  17. Li, Z., Huang, J., Zhou, Z., et al.: LSTM-based deep learning models for answer ranking. IEEE Int. Conf. Data Sci. Cyberspace. 90–97 (2017)

  18. Tao, Y., Chen, H., Qiu, C.: Wind power prediction and pattern feature based on deep learning method. IEEE Pes Asia-Pacific Power Energy Eng. Conf. 1–4 (2014)

  19. Wang, X., Xu, L.: Short-term traffic flow prediction based on deep learning. J. Transport Syst Eng. Informat. Technol. 18(1), 81–88 (2018)

    MathSciNet  Google Scholar 

  20. Pham, T., Tran, T., Phung, D., et al.: Predicting healthcare trajectories from medical records: a deep learning approach. J. Biomed. Inform. 69, 218–229 (2017)

    Article  Google Scholar 

  21. Tahat, L., Korel, B., Harman, M., et al.: Regression test suite prioritization using system models [J]. Softw. Test. Verif. Reliab. 22(7), 481–506 (2012)

    Article  Google Scholar 

  22. Korel, B., Tahat, L., Harman, M.: Test prioritization using system models. IEEE Int. Conf. Softw. Maint. 559−568 (2005)

  23. Noor, T., Bin, T., Hemmati, H.: A similarity-based approach for test case prioritization using historical failure data, pp. 58–68. Int. Symp. Softw. Reliab. Eng. IEEE Comput. Soc., IEEE (2015)

    Google Scholar 

  24. Strandberg, P., Afzal, W., Ostrand, T., et al.: Automated system-level regression test prioritization in a nutshell. IEEE Softw. 34(4), 30–37 (2017)

    Article  Google Scholar 

  25. Srikanth, H., Williams, L., Osborne, J.: System test case prioritization of new and regression test cases. Int. Sym. Empir Softw. Eng.1–10 (2005)

  26. Wang, X., Zeng, H., Lin, W.: Techniques for regression testing in agile development environment. Chin. J. Comput. 43(2), 1–16 (2019)

    Google Scholar 

  27. Catal, C., Mishra, D.: Test case prioritization: a systematic mapping study. Softw. Qual. J. 21(3), 445–478 (2013)

    Article  Google Scholar 

  28. Bian, Y., Li, Z., Zhao, R., et al.: Epistasis based ACO for regression test case prioritization [J]. IEEE Trans. Emerg. Top. Comput. Intell. 1(3), 213–223 (2017)

    Article  Google Scholar 

  29. Noguchi, T., Washizaki, H., Fukazawa, Y.: History-based test case prioritization for black box testing using ant colony optimization. IEEE 8th Int. Conf. Softw. Test. Verif. Validat. 1–2 (2015)

  30. De, S., Luciano, S., et al.: Search based constrained test case selection using execution effort. Expert Syst. Appl. 40(12), 4887–4896 (2013)

    Article  Google Scholar 

  31. Zhang, L., Hou, S., Guo, C., et al.: Time-aware test-case prioritization using integer linear programming. Eighteenth Int. Sym. Softw. Test. Anal. 213–224 (2009)

  32. Walcott, K.R. et al.: Time aware test suite prioritization. Acm/sigsoft International Symposium on Software Test. Anal. 1–12 (2006)

  33. Elbaum, S., Rothermel, G., Penix, J.: Techniques for improving regression testing in continuous integration development environments, pp. 235–245. ACM Sigsoft Int. Symp., The (2014)

    Google Scholar 

  34. Gensler, A., Henze, J., Sick, B., et al.: Deep learning for solar power forecasting -an approach using autoencoder and LSTM neural networks. IEEE Int. Conf. Syst. Man Cybernet. 002858–002865 (2016)

  35. Zhao, Z., Chen, W., Wu, X., et al.: LSTM network: a deep learning approach for short-term traffic forecast. Intell. Trans. Syst. 1(2), 68–75 (2017)

    Article  Google Scholar 

  36. Wang, X., Wu, J., Liu, C., et al.: Exploring LSTM based recurrent neural network for failure time series prediction. J. Beijing Univ. Aeronaut. Astronaut. (4):772–784 (2018)

  37. Elbaum, S., Malishevsky, A., Rothermel, G.: Test case prioritization: a family of empirical studies. IEEE Trans. Softw. Eng. 28(2), 159–182 (2002)

    Article  Google Scholar 

  38. Elbaum, S., Malishevsky, A., Rothermel, G.: Incorporating varying test costs and fault severities into test case prioritization. Int. Conf. Softw. Eng. 329–338 (2001)

  39. Abadi, M., Barham, P., Chen, J., et al.: Tensorflow: a system for large-scale machine learning.12th Symp. Operat. Syst. Des. Implement. 2016: 265–283

  40. https://github.com/tgjeon/TensorFlow-Tutorials-for-Time-Series

  41. Kingma, D, Ba J.A.: A method for stochastic optimization. Int. Conf. Learn. Represent. 1–15 (2015)

Download references

Acknowledgements

This research work is supported by National Natural Science Foundation of China (61572306).

Author information

Authors and Affiliations

  1. School of Computer Engineering and Science, Shanghai University, Shanghai, 200444, China

    Lei Xiao & Huaikou Miao

  2. College of Computer and Information Engineering, Xiamen University of Technology, Xiamen, 361024, China

    Lei Xiao & Tingting Shi

  3. Shanghai Key Laboratory of Computer Software Testing & Evaluating, Shanghai, 201112, China

    Lei Xiao

  4. Fujian Yifei Information Technology Co., Ltd., Xiamen, 361000, China

    Yu Hong

Authors
  1. Lei Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huaikou Miao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tingting Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yu Hong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Xiao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiao, L., Miao, H., Shi, T. et al. LSTM-based deep learning for spatial–temporal software testing. Distrib Parallel Databases 38, 687–712 (2020). https://doi.org/10.1007/s10619-020-07291-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10619-020-07291-1

Keywords

Search

Navigation