Construct Validity of Assessing Interest in STEM Content Scale

Srikoon, Sanit; Apaivatin, Ronnachit; Monsang, Piyapong; Khamngoen, Sujinta; Malaitao, Thararat

doi:https://doi.org/10.1155/2020/1696494

Education Research International

On this page

Abstract Introduction Literature Review Conclusion and Discussion Data Availability Conflicts of Interest Acknowledgments References Copyright Related Articles

Research Article | Open Access

Volume 2020 | Article ID 1696494 | https://doi.org/10.1155/2020/1696494

Construct Validity of Assessing Interest in STEM Content Scale

Sanit Srikoon,¹Ronnachit Apaivatin,¹Piyapong Monsang,¹Sujinta Khamngoen,¹and Thararat Malaitao¹

Academic Editor: Gwo-Jen Hwang

Received31 Oct 2019

Accepted02 Sept 2020

Published16 Sept 2020

Abstract

The purpose of this research was to validate the assessing interest in STEM content scale. The sample in this research was 552 students in upper secondary school at Fangchanupathum School, Chiang Mai, Thailand, who validated the assessing interest in STEM content scale, which used a rating scale of four factors consisting of (1) science, (2) mathematics, (3) laboratorial skills, and (4) technology and 20 items. Confirmatory factor analysis was used to confirm the construct validity of the assessing interest in STEM content scale adapted from the study by Tyler-Wood, Knezek, and Christensen. The results confirmed that the construct validity of this assessing interest in STEM content scale had an excellent fit. They showed that the fitness index of validating the assessing interest in STEM content scale was X² statistic of 163.679 (degree of freedom = 141, value = 0.0928) and the X²/df ratio having a value of 1.160 indicating a good fit. The comparative fit index (CFI) was 0.991, and Tucker–Lewis coefficient (TLI) was 0.988. The root mean square error approximation (RMSEA) was 0.017. The standardized root mean residual (SRMR) was 0.047. All the indicators indicated that there was a good fit between the empirical data and the hypothetical measurement model.

1. Introduction

In recent years, STEM education has been a learning and teaching approach, integrating science, technology, engineering, and mathematics knowledge and skills [1]. This interdisciplinary field of study both supports and provides a positive contribution to students’ skills in the 21^st century such as STEM literacy, problem solving, critical thinking, and creativity [2, 3]. In addition, STEM education is able to help students develop their research-questioning, logical reasoning and working behaviours in collaborative ways and let the students find solutions among simple or complex problems, local or global problems, and real-life situations. Hence, STEM education uses 21^st century skills by assisting students to adapt or adjust themselves by applying their new knowledge through their perceived experience to new situations [4].

The interest does not only mean that if someone has an interest in learning about something it becomes easy and even enjoyable to learn about that topic seeing that the students’ interest is the primary influence on perceived learning [5]. To know students’ interest in STEM education is vital for teachers to create a learning environment which suits students perfectly. Using an accurate assessment of interest in STEM education, we are able to identify each of the subject areas which are science, technology, engineering, and mathematics; thus, the teachers can use the results to assess students’ interest to design and fulfill their classroom activities or students’ learning experience. This also confirms the expectation that students learn more in environments in which instructional methods are congruent with their preferences [6].

Because of the shortage of the assessing interest in STEM content scale in Thailand and the loss of interest in STEM field, this was the most frequently cited reason that students gave for switching out of STEM education [7]. It was essential to develop the instrument to assess the students’ interest in STEM content; therefore, the researchers studied the appropriate instrument for assessing interest in STEM content adapted from the study by Tyler-Wood et al. which was analysed and it was shown that it had excellent internal consistency reliability as well as good content, construct, and criterion-related validity [8]. In addition, the researchers adjusted the instrument to assess the students’ interest in STEM content based on the educational context of Thailand. So to know how well the measured variables represent the construct validation of this questionnaire scale, the researchers use the confirmatory factor analysis (CFA) to examine the reliability and validity and verify the data reliability of this instrument. The confirmatory factor analysis (CFA) provides a standardized loading factor of each indicator where the research variable is quantified from a series of statements known as manifest variable. The standardized loading factor indicates the contribution of each indicator to the respective research latent variable. This model or hypothesis specifies which variables will be correlated with which factors and which factors are correlated. The hypothesis is based on a strong theoretical and/or empirical foundation [9].

The adapted assessing interest in STEM content scale had four factors incorporated as descriptors for target statements reflecting perceptions of the students’ interest in science, mathematics, laboratorial skill, and technology separately. Each of the four scales consists of five semantic perception adjective pairs used on this assessing which represented STEM interests. The purpose of this study was to test whether the factor structure of the assessing interest in STEM content scale could be reiterated as an independent sample using confirmatory factor analysis (CFA) with maximum likelihood estimates following the conceptual framework in Figure 1.

2. Literature Review

STEM (science, technology, engineering, and mathematics) education is a modern educational approach that endeavours to train students to develop high problem-solving skills and improve their critical thinking as well as fostering collaborative working skills to establish a connection between real-life problems and the integrated contents. In place of learning the disciplines involved separately, STEM education focuses on skills such as problem solving, research, effective communication, and design [10]. It also aims for students to look at problems with some perspective in connection with other fields and acquire current information skills with an educational approach holistically [11]. It can contribute to students’ sustainable growth by improving their skills in science and technology.

To enhance students’ performance, teachers have to know their interests so as to motivate them. The interest was defined as relatively stable preferences that focused on objects, activities, or experience [12]. Consequently, the interest in STEM education was satisfaction, performance, and persistence incorporated within STEM education. If they feel better about learning in STEM education, they will be more successful and more productive. The students have worked on problem solving activities in groups gaining more freedom and flexibility, and these activities can lead students to think about STEM disciplines and allow them to develop skills and collaborate for lifelong learning. As a result of their studies, these activities help them develop their skills and learn from one another and develop their interest towards STEM education [4]. Moreover, the interest in STEM education directs effort towards learning in a domain and correlates with a multitude of academic and occupational outcomes, including course selection, achievement, and persistence in a given field of study or career [13]. Hence, the interest in STEM content can be assessed by using the affective judgment related to how much an individual likes the domain and the tasks one enjoys.

Getting the right students’ information by using the proper assessing tools is the most important thing for all teachers to know the students’ interest upon their perceptions in (1) science, (2) mathematics, (3) laboratorial skills, and (4) technology, and it can result in students’ successful outcome. To earn the accurate students’ information using the construct validity which is generally viewed as a unifying form of validity for psychological measurements, one should subsume both content and criterion validity. The construct validity had been traditionally treated as distinct forms of validity [14]. Due to the fact that this method can be said to be well-accepted to accumulate the evidence by taking a unified definition of construct validity, using content analysis, correlation coefficients, factor analysis, and ANOVA, the researchers are able to demonstrate the differences between groups or pretest and posttest intervention studies and multitrait/multimethod studies and so on. [15]. Thereby, the construct validity is an experimental demonstration that the questionnaire uses to measure the construct it claims to be measuring. It can reveal that the questionnaire is a representative sample of the content of either objectives or specifications it was originally designed to measure. In order to test all the variables of the questionnaire, the researchers should use the appropriate analysis to investigate whether the questionnaire used has the construct validity. The confirmatory factor analysis (CFA) is a statistical procedure for examining the hypotheses about the commonality among the variables. As a multivariate procedure, the confirmatory factor analysis (CFA) is used to simultaneously test multiple hypotheses that collectively constitute a measurement model [16]. As a result, it attempts to validate hypothesized factors that the researchers expect to emerge from data collected from test questions or questionnaire responses. This method can be used to validate the questionnaire of the assessing interest in STEM content scale.

3. Objective

This research was to validate the assessing interest in STEM content scale adapted from the study by Tyler-Wood et al. [8] by using confirmatory factor analysis (CFA).

4. Research Design

4.1. The Sample Group

The sample group was 552 students in upper secondary at Fangchanupathum School, Chiang Mai, Thailand. The purposive sampling was used in this research. It consisted of 148 grade 10 students (26.81%), 168 grade 11 students (30.43%). and 236 grade 12 students (42.75%). It could be divided into two programs of study which included 203 students being in a science and mathematics program (36.77%) and 349 students being in an arts (languages, social studies, arts, or physical education) program (63.23%).

4.2. Variable

The variables used were the students’ interest in STEM content consisting of reflecting perceptions appearing while the students were studying (1) science, (2) mathematics, (3) laboratorial skills, and (4) technology and whether they had an interest in STEM education which was assessed through their satisfaction, performance, and persistence.

4.3. Research Instrument

The instrument used in the research was the questionnaire with five rating scales divided into four sections having five questions for each section following the four variables which were adapted from the study by Tyler-Wood et al. [8]. The first section was about the reflecting perceptions of students while studying science. The second section was about the reflecting perceptions of students during mathematics class. The third section was about students’ reflecting perceptions when they were doing experiments in the laboratory. The last section was about students’ reflecting perceptions as they were studying computer studies or technology. The Index of Item-Objective Congruence (IOC) was used to find the content validity. This questionnaire was checked by three experts. Obviously, the results showed that all items in this questionnaire received the value of +1 which meant that this questionnaire was congruent. The assessing interest in STEM content scale is shown in Table 1.

In Table 2, the assumptions in statistics of the four factors of the assessing interest in STEM content scale showed that the mean (x̅) was from 3.125 to 5.005, the deviation (SD) was from 1.504 to 1.694, the skewness was from −0.021 to 0.506, and the kurtosis was from −0.162 to −0.736.(2)The results of the examination of the goodness of fit index of confirmatory factor analysis (CFA) in the assessing interest in STEM content scale showed the following:(2.1)The chi-square: X² was 163.679 (degree of freedom = 141, value = 0.0928), which meant that the model fit indices are based on the empirical data [18],(2.2)The X²/df ratio had a value of 1.160 being in the ratio of 2 : 1, which indicates that the model of assessing interest in STEM content scale had a good fit [19],(2.3)The comparative fit index (CFI) was 0.991; the value was above 0.95 indicated that the model of assessing interest in STEM content scale was correlated with the factors [20],(2.4)Tucker–Lewis coefficient (TLI) was 0.988; the value was above 0.95 indicated that the model of assessing interest in STEM content scale was correlated with the factors [20],(2.5)The root mean square error approximation (RMSEA) was 0.017 which was less than 0.03, and it indicated that the model of assessing interest in STEM content scale was correlated with the factors [20],(2.6)The standardized root mean residual (SRMR) was 0.047 which was less than 0.08, and it indicated that the model of assessing interest in STEM content scale was correlated with the factors [20].

4.4. Data Collection

The data was collected as follows:(1)Asking permission to survey the students’ interest in STEM content by using the qualified assessing interest in STEM content scale from the school director and planning to collect the data together(2)Administering to assist students to understand the research objective, letting a total of 552 students at Fangchanupathum School, Chiang Mai, Thailand, realize how important this questionnaire was and answering it truly(3)Collecting the data following the scheduled plan during 8–12 July, 2019

4.5. Data Analysis

This research was to examine the construct validity of the assessing interest in STEM content scale in order to examine the item objective congruence and whether it was a good fit using the confirmatory factor analysis (CFA) by MPlus program [17].

5. Research Result

The results of the data analysis showed the following:(1)The results of the assumptions in statistics of the four factors were obtained using 20 questions which involved students’ interest in STEM content through reflecting perceptions in (1) science (five items), (2) mathematics (five items), (3) laboratorial skills (five items), and (4) technology (five items). The results are shown in Table 2.

Hence, these results confirmed that the construct validity of this assessing interest in STEM content scale had an excellent fit. The detailed data analysis result is shown in Table 3.

Table 3 of the first-order confirmatory factor analysis model of the assessing interest in STEM content scale shows that the estimate (B) of 20 items was from 0.095 to 1.722, the standard error (SE) was from 0.00 to 0.397, the standard error of estimate (T) was from 1.209 to 22.348, the estimate of STDYX standardization () was from 0.069 to 0.902, the factor score information (FS) was from −0.035 to 0.375, and the R-square (R²) was from 0.005 to 0.814. The second-order confirmatory factor analysis showed that the estimate (B) of four factors was from 0.305 to 1.090, the standard error (SE) was from 0.00 to 0.339, the standard error of estimate (T) was from 1.975 to 3.277, the estimate of STDYX standardization () was from 0.145 to 0.520, the factor score information (FS) was from 0.004 to 0.155, and the R-square (R²) was from 0.105 to 0.270.

6. Conclusion and Discussion

The research of confirmatory factor model analysis of the assessing interest in STEM content scale adjusted from Tyler-Wood et al. [8] appropriate to the educational context in Thailand consisted of four factors composing of the reflecting perceptions in (1) science, (2) mathematics, (3) laboratorial skills, and (4) technology because most of Thai schools generally focus on these four disciplines and teach the engineering skill through doing the projects or experiments in the laboratory. The results of the examining the goodness of fit index of confirmatory factor analysis (CFA) found that the model fit indexes based on the empirical data were X² = 163.679; df = 141; value = 0.0928; X²/df ratio = 1.160; CFI = 0.991; TLI = 0.988; RMSEA = 0.017; SRMR = 0.047. These represented that this developed assessing interest in STEM content scale can be used to measure students’ interest in STEM content. It was related to the questionnaire of Tyler-Wood et al. [8] using the Exploratory Factor Analysis (Principle Component Extraction, Varimax Rotation, and Suppressed Display of Loadings were less than 0.5) which indicated that in every case the items loaded on the hypothesized factors meant that this questionnaire was most strongly associated with intended construct in every case. Consequently, the teachers are able to use the results of the students’ interest in STEM content to create or develop their classroom activities properly to carry out the goal of 21^st century learning skills.

Frequently cited reasons for inspiring students to enjoy STEM content were good teaching that involved capable teachers who could create splendid learning activities [21–23]. Therefore, the teachers have to create an exceptional learning environment not only focusing on students’ achievement; otherwise, the students might become turned off by STEM subjects due to the relative difficulty of receiving high grades in STEM classes in comparison to other subjects, and the students became often discouraged from taking STEM subjects. Hence, the students’ emphasis on achieving high grades outweighs their interest in taking subjects related to science, mathematics, technology, and engineering and result in them abandoning their interest in STEM [24]. Thus, this assessing interest in STEM content scale is able to be the accurate tool that the teachers could use to assist them to know the students’ interest and find out ways to encourage both their interest and attitudes towards STEM content and boost their classroom activities. The teachers know clearly which factors in relation to STEM content should be promoted and used to enhance students’ interest [25]. The student’s performance is an essential part of their success, and teachers must consider how to best help them achieve that success by determining what interests or motivates them to live in modern society based on the goal of education in the 21^st century which is the mastery of information, engaged knowledge, and understanding and the advanced use of technology to solve complex problems they will face in their real lives.

In conclusion, the teachers can use this questionnaire in their courses which are related to STEM content in order to evaluate the students’ interests based on the educational context in Thailand. Furthermore, it is better to adjust the questionnaire further to assess both the students’ interests and attitudes towards STEM content.

Data Availability

The data from the questionnaire used to support the findings of this study are usable and can also be made available from the corresponding author upon request via email.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

The authors would like to extend their gratitude to all participants for giving invaluable responses in this study. This work was supported by the Unit of Excellence in Research Methodology of Innovations and Learning Sciences based-on Educational Neurosciences Research Fund (ISEN University of Phayao), University of Phayao, Thailand.

References

Maryland State STEM Standards of Practice, Maryland Stem: Innovation Today to Meet tomorrow’s Global Challenges, Maryland State STEM Standards of Practice, Annapolis, MD, USA, 2012.
A. Bagiati and D. Evangelou, “Engineering curriculum in the preschool classroom: the teacher’s experience,” European Early Childhood Education Research Journal, vol. 23, no. 1, pp. 112–118, 2015.
View at: Publisher Site | Google Scholar
R. W. Bybee, “Advancing stem education: a 2020 vision,” Technology and Engineering Teacher, vol. 70, no. 1, p. 30, 2010.
View at: Google Scholar
J. Bellanca and R. Brandt, 21^st Century Skills: Rethinking How Students Learn, Solution Tree Press, Bloomington, IN, USA, 2010.
J. L. Abrantes, C. Seabra, and L. F. Lages, “Pedagogical affect, student interest, and learning performance,” Journal of Business Research, vol. 60, no. 9, pp. 960–964, 2007.
View at: Publisher Site | Google Scholar
M. R. Young, B. R. Klemz, and J. W. Murphy, “Enhancing learning outcomes: the effects of instructional technology, learning styles, instructional methods, and student behaviour,” Journal of Marketing Education, vol. 25, no. 2, pp. 130–142, 2003.
View at: Publisher Site | Google Scholar
E. Seymour, “The loss of women from science, mathematics, and engineering undergraduate major: an explanatory account,” Science Education, vol. 79, no. 4, pp. 437–473, 1995.
View at: Publisher Site | Google Scholar
T. Tyler-Wood, G. Knezek, and R. Christensen, “Instruments for assessing interest in stem content and careers,” Journal of Technology and Teacher Education, vol. 18, no. 2, pp. 341–363, 2010.
View at: Google Scholar
H. W. Marsh, J. R. Balla, and R. P. McDonald, “Goodness of fit indexes in confirmatory factor analysis: The effect of sample size,” Psychological Bulletin, vol. 103, no. 3, pp. 391–410, 1988.
View at: Publisher Site | Google Scholar
P. Siripattrachai, “Stem education and 21^st century skills development,” Exclusive Journal, vol. 33, pp. 49–56, 2013.
View at: Google Scholar
H. B. Gonzalez and J. J. Kuenzi, “Science, technology, engineering, and mathematics (STEM) education, 2012 education: a primer congressional research service,” 2012.
View at: Google Scholar
S. Hidi, “Interest and its contribution as a mental resource for learning,” Review of Educational Research, vol. 60, no. 4, pp. 549–571, 1990.
View at: Publisher Site | Google Scholar
P. L. Ackerman, “A theory of adult intellectual development: Process, personality, interests, and knowledge,” Intelligence, vol. 22, no. 2, pp. 227–257, 1996.
View at: Google Scholar
F. J. Landy, “Stamp collecting versus science: validation as hypothesis testing,” American Psychologist Journal, vol. 41, pp. 1183–1192, 1986.
View at: Publisher Site | Google Scholar
J. D. Brown, “Questions and answers about language testing statistics: what is construct validity?” Shiken: JALT Testing & Evaluation SIG Newsletter, vol. 4, no. 2, pp. 8–12, 2000.
View at: Google Scholar
R. H. Hoyle, “Confirmatory factor analysis,” Encyclopedia of Social Science Research Methods, vol. 1, pp. 169–175, 2004.
View at: Google Scholar
K. L. Muthen and O. B. Muthen, Mplus Statistical Analysis with Latent Variables User Guided, Muthén & Muthén, Los Angeles: CA, USA, 6th edition, 2013.
D. Hooper, J. Coughlan, and M. R. Mullen, “Structural equation modelling: guidelines for determining model fit,” Electronic Journal of Business Research Methods, vol. 6, no. 1, pp. 53–59, 2008.
View at: Google Scholar
J. F. Hair, Multivariate Data Analysis, Prentice-Hall, Upper Saddle River, NJ, USA, 6th edition, 2006.
L. T. Hu and P. M. Bentler, “Cut off criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives,” Structural Equation Modeling, vol. 6, no. 1, pp. 1–55, 1999.
View at: Publisher Site | Google Scholar
A. Kline, B. M. Aller, C. Crumbaugh, P. Vellom, and E. Tsang, “Work in progress: western, Michigan university partnership with K-12 teachers to improve stem education,” in Proceddings of the 38th Annual Frontiers in Education Conference, pp. 7-8, San Diego, CA, USA, October 2006.
View at: Publisher Site | Google Scholar
B. H. Wells, H. A. Sanchez, and J. M. Attridge, “Modeling student interest in science, technology, engineering and mathematics,” in Proceedings of the 2007 IEEE Conference: Meeting the Growing Demand for Engineers and Their Educators 2010–2020 International Summit, pp. 1–17, Munich, Germany, 2007.
View at: Publisher Site | Google Scholar
I. P. Solomides and B. L. Button, “Improving the quality of student learning: a system approach,” Engineering Science and Education Journal, vol. 3, no. 3, pp. 131–136, 1994.
View at: Publisher Site | Google Scholar
J. M. B. Fonseca and J. E. Conboy, “Secondary student perceptions of factors effecting failure in science in Portugal,” Eurasia Journal of Mathematics, Science and Technology Education, vol. 2, no. 2, pp. 82–95, 2006.
View at: Publisher Site | Google Scholar
C. Diaz-Rubin, “Reading interests of students,” Reading Improvement, vol. 33, pp. 169–175, 1996.
View at: Google Scholar

Copyright

Copyright © 2020 Sanit Srikoon et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

PDF Download Citation

Download other formats

Order printed copies

Views

904

Downloads

978

Citations