Skip to main content
SpringerLink
Log in

Development and Validation of an Instrument to Measure STEM Undergraduate Students’ Comprehensive Educational Process

  • Research Article
  • Published:
Frontiers of Education in China

Abstract

This study reports the development and validation of an instrument, “Undergraduate STEM Educational Process Questionnaire” (USEPQ) to assess the learning experiences of STEM (science, technology, engineering and mathematics)-major undergraduates from the perspective of a conceptualized comprehensive STEM educational process framework that consists of key essentials such as course learning and instruction, educational environment, assessment methods, and unique STEM attributes. Exploration Factor Analysis and Confirmatory Factor Analysis performed on independent samples indicate that the finalized 9-factor, 41-item USEPQ possesses a stable factor structure and sound psychometric properties. The factors of the validated scale are Classroom Instruction of Courses, Support from Faculty Members, Alternative Assessment Methods, Cooperative Learning, Resources and Service, Academic Competition, Cross-discipline Features of Program Courses, Connectedness of Program Courses, and Examination Difficulty.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Allan, J., Clarke, K., & Jopling, M. (2009). Effective teaching in higher education: Perceptions of first year undergraduate students. International Journal of Teaching and Learning in Higher Education, 21(3), 362–372.

    Google Scholar 

  • Avery, Z. K., & Reeve, E. M. (2013). Developing effective STEM professional development programs. Journal of Technology Education, 25(1), 55–69.

    Google Scholar 

  • Bartlett, M. S. (1950). Tests of significance in factor analysis. British Journal of Statistical Psychology, 3(2), 77–85. https://doi.org/10.1111/j.2044-8317.1950.tb00285.x

    Google Scholar 

  • Bedggood, R. E., & Donovan, J. D. (2012). University performance evaluations: What are we really measuring? Studies in Higher Education, 37(7), 825–842. https://doi.org/10.1080/03075079.2010.549221

    Google Scholar 

  • Bentler, P. M. (1990). Comparative fit indexes in structural models. Psychological Bulletin, 107(2), 238–246. https://doi.org/10.1037/0033-2909.107.2.238

    Google Scholar 

  • Biggs, J., Kember, D., & Leung, D. Y. (2001). The Revised Two-Factor Study Process Questionnaire: R-SPQ-2F. British Journal of Educational Psychology, 71(1), 133–149.

    Google Scholar 

  • Browne, M. W., & Cudeck, R. (1989). Single sample cross-validation indices for covariance structures. Multivariate Behavioral Research, 24(4), 445–455. https://doi.org/10.1207/s15327906mbr2404_4

    Google Scholar 

  • Bryson, C., & Hand, L. (2007). The role of engagement in inspiring teaching and learning. Innovations in Education and Teaching International, 44(4), 349–362. https://doi.org/10.1080/14703290701602748

    Google Scholar 

  • Case, J. M. (2013). Researching student learning in higher education: A social realist approach. Abingdon, UK: Routledge.

    Google Scholar 

  • Chan, Y.-K. (2016). Investigating the relationship among extracurricular activities, learning approach and academic outcomes: A case study. Active Learning in Higher Education, 17(3), 223–233. https://doi.org/10.1177/1469787416654795

    Google Scholar 

  • Chen, X. L. (2013). STEM attrition: College students’ paths into and out of STEM fields (NCES 2014-001). Washington, DC: National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education.

    Google Scholar 

  • Costello, A. B., & Osborne, J. W. (2005). Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Practical Assessment, Research & Evaluation, 10(7), 1–9.

    Google Scholar 

  • Crawley, E. F., Malmqvist, J., Östlund, S., Brodeur, D. R., & Edström, K. (2014). Rethinking engineering education (2nd ed.). Cham, Switzerland: Springer.

    Google Scholar 

  • CSU. (2016). Program improvement process: Electrical/electronic engineering. (4th ed.) Retrieved from http://www.csuchico.edu/eece/_assets/documents/CMPE_Program_Improvement_Process_S16.pdf

  • Davim, J. P. (Ed.). (2012). Mechanical engineering education. London, UK: ISTE & Hoboken, NJ: John Wiley & Sons.

    Google Scholar 

  • Diseth, Á. (2011). Self-efficacy, goal orientations and learning strategies as mediators between preceding and subsequent academic achievement. Learning and Individual Differences, 21(2), 191–195. https://doi.org/10.1016/j.lindif.2011.01.003

    Google Scholar 

  • Do Ba, K., Do Ba, K., Lam, Q. D., Le, D. T. B. A., Nguyen, P. L., Nguyen, P. Q., & Pham, Q. L. (2016). Student plagiarism in higher education in Vietnam: An empirical study. Higher Education Research & Development, 36(5), 934–946. https://doi.org/10.1080/07294360.2016.1263829

    Google Scholar 

  • Dorman, J. P. (1998). The development and validation of an instrument to assess institutional-level environment in universities. Learning Environments Research, 1(3), 333–352. https://doi.org/10.1023/A:1009965621296

    Google Scholar 

  • Douglas, J., Douglas, A., & Barnes, B. (2006). Measuring student satisfaction at a UK university. Quality Assurance in Education, 14(3), 251–267. https://doi.org/10.1108/09684880610678568

    Google Scholar 

  • Dunkin, M. J. (2002). Novice and award-winning teachers’ concepts and beliefs about teaching in higher education. In N. Hativa & J. Goodyear (Eds.), Teacher thinking, beliefs and knowledge in higher education (pp. 41–57). Dordrecht, The Netherlands: Springer.

    Google Scholar 

  • Elhadary, O. (2016). Student satisfaction in STEM: An exploratory study. American Journal of Educational Research, 4(2), 195–199. doi: https://doi.org/10.12691/education-4-2-7

    Google Scholar 

  • Elliott, K. M., & Shin, D. (2002). Student satisfaction: An alternative approach to assessing this important concept. Journal of Higher Education Policy and Management, 24(2), 197–209. https://doi.org/10.1080/1360080022000013518

    Google Scholar 

  • Finelli, C. J., & Johnson, A. W. (2017). Work in progress: Flipping the circuits classroom: The impact of pre-class reading and in-class active learning on student and instructor. In 2017 IEEE Global Engineering Education Conference (EDUCON 2016) (pp. 437–440). Red Hook, NY: Curran Associates.

    Google Scholar 

  • Hair, J. F., Jr., Black, W. C., Babin, B. J., & Anderson, R. E. (2014). Multivariate data analysis (7th ed.). Harlow, UK: Pearson.

    Google Scholar 

  • Henderson, C., Beach, A., Finkelstein, N., & Larson, R. S. (2008). Facilitating change in undergraduate STEM: Initial results from an interdisciplinary literature review. AIP Conference Proceedings, 1064(1), 131–134.

    Google Scholar 

  • Husband, T. (2013). Improving the quality of instruction through a service teaching Framework. Journal of Effective Teaching, 13(2), 73–82.

    Google Scholar 

  • Ives, G., & Rowley, G. (2005). Supervisor selection or allocation and continuity of supervision: Ph.D. students’ progress and outcomes. Studies in Higher Education, 30(5), 535–555. https://doi.org/10.1080/03075070500249161

    Google Scholar 

  • Justicia, F., Pichardo, M. C., Cano, F., Berbén, A. B. G., & De la Fuente, J. (2008). The Revised Two-Factor Study Process Questionnaire (R-SPQ-2F): Exploratory and confirmatory factor analyses at item level. European Journal of Psychology of Education, 23(3), 355–372. https://doi.org/10.1007/BF03173004

    Google Scholar 

  • Kaiser, H. F., & Rice, J. (1974). Little jiffy, mark IV. Educational and Psychological Measurement, 34(1), 111–117. https://doi.org/10.1177/001316447403400115

    Google Scholar 

  • Kober, N. (2015). Reaching students: What research says about effective instruction in undergraduate science and engineering. Washington, DC: The National Academic Press.

    Google Scholar 

  • Law, D. C., & Meyer, J. H. (2011). Adaptation and validation of the course experience questionnaire in the context of post-secondary education in Hong Kong. Quality Assurance in Education, 19(1), 50–66. https://doi.org/10.1108/09684881111107753

    Google Scholar 

  • Lizzio, A., Wilson, K., & Simons, R. (2002). University students’ perceptions of the learning environment and academic outcomes: Implications for theory and practice. Studies in Higher Education, 27(1), 27–52. https://doi.org/10.1080/03075070120099359

    Google Scholar 

  • Lu, G. S. (2007). 优化学生学习经验提高高校教学质量: 基于学生视角的高校教学质量改 进途径与方法探讨 [Improving the quality of university students’ learning by optimizing students’ learning experiences: Discussion on the ways and methods of improving teaching quality in colleges and universities based on students’ perspectives]. 复旦教育论坛 [Fudan Education Forum], 5(2), 8–11.

    Google Scholar 

  • Lynd-Balta, E., Erklenz-Watts, M., Freeman, C., & Westbay, T. D. (2006). Professional development using an interdisciplinary learning circle. Journal of College Science Teaching, 35(4), 18–24.

    Google Scholar 

  • Marsh, H. W., Ginns, P., Morin, A. J., Nagengast, B., & Martin, A. J. (2011). Use of student ratings to benchmark universities: Multilevel modeling of responses to the Australian Course Experience Questionnaire (CEQ). Journal of Educational Psychology, 103(3), 733–748.

    Google Scholar 

  • Navarro, M., M., Iglesias, M. P., & Rivera, P. R. (2005). A new management element for universities: Satisfaction with the offered courses. International Journal of Educational Management, 19(6), 505–526. https://doi.org/10.1108/09513540510617454

    Google Scholar 

  • Nunnally, J. C., & Bernstein, I. (1994). Psychometric theory (3rd ed.). New York, NY: McGraw-Hill.

    Google Scholar 

  • O’Donovan, B. (2017). How student beliefs about knowledge and knowing influence their satisfaction with assessment and feedback. Higher Education, 74(4), 617–633. https://doi.org/10.1007/s10734-016-0068-y

    Google Scholar 

  • O’Flaherty, J., & Phillips, C. (2015). The use of flipped classrooms in higher education: A scoping review. The Internet and Higher Education, 25, 85–95. https://doi.org/10.1016/j.iheduc.2015.02.002

    Google Scholar 

  • Ortiz, A. M., & Sriraman, V. (2015). Exploring faculty insights into why undergraduate college students leave STEM fields of study-A three-part organizational self-study. American Journal of Engineering Education, 6(1), 43–60. https://doi.org/10.19030/ajee.v6i1.9251

    Google Scholar 

  • Price, L., Richardson, J. T. E., Robinson, B., Ding, X., Sun, X. L., & Han, C. L. (2011). Approaches to studying and perceptions of the academic environment among university students in China. Asia Pacific Journal of Education, 31(2), 159–175. https://doi.org/10.1080/02188791.2011.566996

    Google Scholar 

  • Richardson, J. T. (2005). Students’ perceptions of academic quality and approaches to studying in distance education. British Educational Research Journal, 31(1), 7–27. https://doi.org/10.1080/0141192052000310001

    Google Scholar 

  • Schreiber, J. B., Nora, A., Stage, F. K., Barlow, E. A., & King, J. (2006). Reporting structural equation modeling and confirmatory factor analysis results: A review. The Journal of Educational Research, 99(6), 323–338. https://doi.org/10.3200/JOER.99.6.323-338

    Google Scholar 

  • Schwab, J. J. (1973). The practical 3: Translation into curriculum. School Review, 81(4), 501–522. https://doi.org/10.1086/443100

    Google Scholar 

  • Seymour, E., & Hewitt, N. M. (1997). Talking about leaving: Why undergraduates leave the sciences. Boulder, CO: Westview Press.

    Google Scholar 

  • Singer, S., & Smith, K. A. (2013). Discipline-based education research: Understanding and improving learning in undergraduate science and engineering. Journal of Engineering Education, 102(4), 468–471. https://doi.org/10.1002/jee.20030

    Google Scholar 

  • Sithole, A., Chiyaka, E. T., McCarthy, P., Mupinga, D. M., Bucklein, B. K., & Kibirige, J. (2017). Student attraction, persistence and retention in STEM programs: Successes and continuing challenges. Higher Education Studies, 7(1), 46–59. https://doi.org/10.5539/hes.v7n1p46

    Google Scholar 

  • Slavin, R. E. (2008). Cooperative learning, success for all, and evidence-based reform in education. Éducation et Didactique, 2(2), 149–157. https://doi.org/10.4000/educationdidactique.334

    Google Scholar 

  • Song, G. T., Kwan, C.-Y., Bian, Z., Tai, B. J., & Wu, Q. (2005). Perspectives: Exploratory thoughts concerning educational reform with problem-based learning in China. Teaching and Learning in Medicine, 17(4), 382–384. https://doi.org/10.1207/s15328015tlm1704_12

    Google Scholar 

  • The National Academies of Sciences, Engineering, and Medicine. (2017). Undergraduate research experiences for STEM students: Successes, challenges, and opportunities. Washington, DC: The National Academies Press.

    Google Scholar 

  • The National Academies of Sciences, Engineering, and Medicine. (2018). Indicators for monitoring undergraduate STEM education. Washington, DC: The National Academies Press.

    Google Scholar 

  • The National Research Council. (2003). Evaluating and improving undergraduate teaching in science, technology, engineering, and mathematics. Washington, DC: The National Academies Press.

    Google Scholar 

  • Umbach, P. D. (2007). Faculty cultures and college teaching. In R. P. Perry & J. C. Smart (Eds.), The scholarship of teaching and learning in higher education: An evidence-based perspective (pp. 263–317). Dordrecht, The Netherlands: Springer.

    Google Scholar 

  • van Leeuwen, T., & Tijssen, R. (2000). Interdisciplinary dynamics of modern science: Analysis of cross-disciplinary citation flows. Research Evaluation, 9(3), 183–187. https://doi.org/10.3152/147154400781777241

    Google Scholar 

  • Watted, A., & Barak, M. (2018). Motivating factors of MOOC completers: Comparing between university-affiliated students and general participants. The Internet and Higher Education, 37, 11–20. https://doi.org/10.1016/j.iheduc.2017.12.001

    Google Scholar 

  • Wilson, K. L., Lizzio, A., & Ramsden, P. (1997). The development, validation and application of the Course Experience Questionnaire. Studies in Higher Education, 22(1), 33–53. https://doi.org/10.1080/03075079712331381121

    Google Scholar 

  • Worthington, R. L., & Whittaker, T. A. (2006). Scale development research: A content analysis and recommendations for best practices. The Counseling Psychologist, 34(6), 806–838. https://doi.org/10.1177/0011000006288127

    Google Scholar 

  • Xu, Y. H. J. (2016). Attention to retention: Exploring and addressing the needs of college students in STEM majors. Journal of Education and Training Studies, 4(2), 67–76. https://doi.org/10.11114/jets.v4i2.1147

    Google Scholar 

  • Yin, H. B., & Lu, G. S. (2014). Development and validation of an instrument for assessing mathematics classroom environment in tertiary institutions. The Asia-Pacific Education Researcher, 23(3), 655–669. https://doi.org/10.1007/s40299-013-0138-1

    Google Scholar 

  • Yin, H. B., & Wang, W. L. (2015). Assessing and improving the quality of undergraduate teaching in China: The Course Experience Questionnaire. Assessment & Evaluation in Higher Education, 40(8), 1032–1049. https://doi.org/10.1080/02602938.2014.963837

    Google Scholar 

  • Yin, H. B., Wang, W. L., & Han, J. Y. (2016). Chinese undergraduates’ perceptions of teaching quality and the effects on approaches to studying and course satisfaction. Higher Education, 71(1), 39–57. https://doi.org/10.1007/s10734-015-9887-5

    Google Scholar 

  • Zepke, N., & Leach, L. (2010). Improving student engagement: Ten proposals for action. Active Learning in Higher Education, 11(3), 167–177. https://doi.org/10.1177/1469787410379680

    Google Scholar 

  • Zhang, L.-F. (2000). University students’ learning approaches in three cultures: An investigation of Biggs’s 3P model. The Journal of Psychology, 134(1), 37–55. https://doi.org/10.1080/00223980009600847

    Google Scholar 

Download references

Acknowledgements

Our gratitude goes to the anonymous reviewers for their invaluable comments on the previous version of this manuscript. The fourth author of this paper Prof. Yipeng Tang thanks National Natural Science Foundation of China (NNSFC) for collaboratively funding the research (No. 71704115).

Author information

Authors and Affiliations

  1. Department of Educational Administration & Policy, Faculty of Education, The Chinese University of Hong Kong, Hong Kong, China

    Tengteng Zhuang & Alan C. K. Cheung

  2. Department of Curriculum and Instruction, Faculty of Education, The Chinese University of Hong Kong, Hong Kong, China

    Wilfred W. F. Lau

  3. College of Education, Capital Normal University, Beijing, 100089, China

    Yipeng Tang

Authors
  1. Tengteng Zhuang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alan C. K. Cheung
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wilfred W. F. Lau
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yipeng Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tengteng Zhuang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhuang, T., Cheung, A.C.K., Lau, W.W.F. et al. Development and Validation of an Instrument to Measure STEM Undergraduate Students’ Comprehensive Educational Process. Front Educ China 14, 575–611 (2019). https://doi.org/10.1007/s11516-019-0028-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11516-019-0028-2

Keywords

Search

Navigation