A validation program for the Self-Beliefs, Writing-Beliefs, and Attitude Survey: A measure of adolescents' motivation toward writing
Introduction
Teachers acknowledge that typical writing instruction for adolescents may not always inspire motivation (Troia & Maddox, 2004). In early grades, although instruction focuses primarily on mechanics, students are often allowed to write on topics of their choice (Katz & Assor, 2007). However, by middle school, writing autonomy diminishes as the focus shifts and students are required to produce discipline-specific texts. At the same time, in secondary grades, most students complete little extended or complex writing allowing minimal opportunity for creativity and expression (Applebee & Langer, 2009; Graham & Harris, 2012). This situation has contributed to 74% of 8th graders and 73% of 12 graders scoring at the “basic” or “below basic” level on national writing tests. We maintain that writing instruction that does not emphasize motivation may be contributing to students’ low performance (National Center for Education Statistics, 2012).
Researchers are still exploring what motivates adolescents to write, but recent findings reveal clear evidence that students’ performance on writing tasks is related to motivation (Zumbrunn, Marrs, & Mewborn, 2016). As motivation for writing can vary depending upon the situation and task, teachers and scholars require valid assessments to determine whether writing skills are primarily a result of cognitive ability and knowledge, affective issues (e.g., motivation), or both. If students are not motivated to write, they may be viewed as having poor writing skills, even if this is not true (Johnston & Costello, 2005). Being identified as having underdeveloped skills may place students on a trajectory of specific coursework and lowered expectations. However, without a valid tool for measuring students’ motivation towards writing, these effects cannot be quantified.
Researchers also know that writing instruction and interventions have varying effects on adolescents’ outcomes (Graham & Perin, 2007; Washburn, Sielaff, & Golden, 2016), which are likely mediated by motivation. For example, Webb, Vandiver, and Jeung (2016) found that a course focused on developing the writing process supported students’ self-efficacy for writing better than courses focused creative or analytical writing. As such, a deeper understanding of adolescent students’ writing affect, in the context of educational interventions, will help identify practices that produce gains in skills while preventing (or reversing) the development of negative self-concepts. Furthermore, a child who has had negative experiences with writing will likely develop a poor attitude toward that experience (Fishbein & Ajzen, 1975) and potentially believe there is great personal cost in engaging in the task (Eccles & Wigfield, 2002). If we extrapolate these attitudes to writing, we can conclude that if a child has had negative experiences with writing, he or she may have a poor attitude toward writing and believe that writing has little value.
In this study we present the results of our two-study validation program for the Self-Beliefs, Writing-Beliefs, and Attitude Survey (SWAS). This measure is designed for monitoring students' motivation towards writing, as well as identify variables that mediate student achievement. To date, existing tools contain important limitations for measuring adolescent students’ affect towards writing. For example, select measures only consider one aspect of writing affect (e.g., Kear, Coffman, McKenna, & Ambrosio, 2000), yet we know that motivation is a complex construct. Therefore, we worked to create a multidimensional instrument. Furthermore, other measures present relatively low reliability (e.g., Troia, Harbaugh, Shankland, Wolbers, & Lawrence, 2013). As reliability is an essential component of validity (Thompson, 2003), we enacted a multistep process for assessing reliability. Most importantly, few measures have undergone rigorous, multi-step validation programs to ensure their scores measure the intended writing constructs.
Validation is a critical component of measurement development, because only through this process do scores take on meaning. Most contemporary researchers report statistical validity (such as factor analyses and/or structural equation modeling) and include reliability estimates (e.g., see Collie, Martin, & Curwood, 2016; Troia et al., 2013). These are essential steps towards validation, but further work is required. Benson (1998) proposed that a strong validation program has three distinct components: the substantive, the structural, and the external. Substantive refers to the collection of existing theoretical and empirical information to define constructs, which we addressed through a thorough review of research. The structural piece entails determining how the observed variables relate to one another and the main construct of interest, so we conducted multiple types of factor analyses (Study 1). Finally, the external component requires the researcher to determine how the measure relates to other expected constructs (Benson, 1998), which we addressed by comparing students’ SWAS scores to other measures of writing (Study 2).
Beyond instrument development, through the process of theoretical reviews and empirical validation, we aim to propose a model of writing motivation for adolescents. Specifically, we present the findings of two studies aimed at establishing the validity of the SWAS. Our research is guided by the following questions:
- 1
How valid are the constructs measured by the Self-Beliefs, Writing-Beliefs, and Attitude Survey?
- 2
What aspects of students’ affect towards writing are measured by the Self-Beliefs, Writing-Beliefs, and Attitude Survey?
Section snippets
Prior research and theoretical foundations
Writing is a skill that “allows people to communicate with others removed in both distance and time” (Graham & Perin, 2007, p. 445). Furthermore, following the work of Myers et al. (2016) writing is conceptualized as a “complex cognitive, physical, affective, and social” process, requiring writers to “use multiple skills and strategies as they move through the stages of planning, drafting, revising, editing publishing, and presentation” (p. 312). Therefore, we define writing as the skills and
SWAS instrument development & substantive validation
After examining existing tools for measuring writing affect, we selected the Motivation for Reading Questionnaire (MRQ) (Wigfield & Guthrie, 1995) to provide an initial structure for our measure. While the MRQ does not pertain to writing, it provides a theoretically-supported model for assessing student motivation. This tool has been repeatedly used to measure students’ (including adolescents) motivation towards reading, and consistently produces valid scores (e.g., Guthrie et al., 2006; Mason,
Survey administration
We administered the SWAS to 517 students at a public middle school (grades six through eight), in the southwestern United States. Students completed paper-based versions of the survey, and total administration took about 15 min. Teachers read the entire survey to students to ensure that results were not impacted by students’ reading ability.
Both the university and local school-level Institutional Review Boards approved our plan to administer this survey without seeking parental permission if
Survey administration
We administered the SWAS to a second, independent, group of participants. Students completed a paper-based version of the survey in approximately 15 min. We used this administration to complete the external stage of our construct validation program. In this study, we solicited parental consent and collected more information about individual students, including teacher ratings and writing samples.
Participants and school context
The sample consisted of 53 students in grades six through 11 who attended a private, independent
General discussion & model of writing motivation
Regarding our overarching theoretical question, the model that best fit the data is based upon a combination of theory and psychometrics (see Fig. 1) with attitudes and belief system representing two unique contributions to motivation, which was predicted by our review of the literature.
This model overlaps with aspects of Conradi et al.’s (2014) proposed hierarchy of reading motivation-related constructs. Their hierarchy conceptualized two categories related to beliefs – those about the self
Limitations & future directions
Despite these rigorous validation procedures, we recognize that limitations remain. Unfortunately, we were not able to collect data about students in Study 1 and therefore could not analyze class-level and individual characteristics, such as language-learning status or gifted and talented enrollment. In particular, future research should conduct measurement invariance analyses to assess whether the constructs of the SWAS function equivalently across gender and skill level. Additionally, we
Conclusions
While teachers certainly agree that an adolescents’ motivation for writing greatly enhances (or reduces) their growth, such discussions of writing motivation has often remained mired at the level of anecdote. Therefore, to make the previously “intangible” into a tangible construct that we can more systematically work to improve, we first need tools to help us measure these constructs. We need to move writing motivation from anecdote to science.
To develop such a measure, especially for an
Declarations of interest
None.
Dr. Katherine LandauWright is an Assistant Professor and Director of the Literacy Lab at Boise State University. Dr. Wright is a former middle school English and social studies teacher. Her research aims to motivate and support middle and high school readers and writers in their content-area classes.
References (67)
- et al.
Measuring cost: The forgotten component of expectancy-value theory
Contemporary Educational Psychology
(2015) - et al.
Reading motivation and reading comprehension growth in the later elementary years
Contemporary Educational Psychology
(2007) - et al.
Developing an individual and collective self-efficacy scale for the teaching of writing in high schools
Assessing Writing
(2016) - et al.
Variation in situation-specific engagement among lower secondary school students
Learning and Instruction
(2018) - et al.
What is happening in the teaching of writing?
The English Journal
(2009) Developing a strong program of construct validation: A test anxiety example
Educational Measurement: Issues and Practice
(1998)- et al.
Academic self-concept and self-efficacy: How different are they really?
Educational Psychology Review
(2003) - et al.
Assessing teachers’ beliefs about social and emotional learning
Journal of Psychoeducational Assessment
(2012) - et al.
Developing motivation to write
Educational Psychologist
(2000) - et al.
Self-efficacy beliefs and motivation in writing development
Examining dimensions of self-efficacy for writing
Educational Psychology
I think I know what my teachers want now: Gender and writing motivation
The English Journal
Multidimensional motivation and engagement for writing: Construct validation with a sample of boys
Educational Psychology
Motivation terminology in reading research: A conceptual review
Educational Psychology Review
Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis
Motivation and education: The self-determination perspective
Educational Psychologist
Motivation in action: Towards a process-oriented conceptualization of student motivation
The British Journal of Educational Psychology
How executive functions predict development in syntactic complexity of narrative writing in the upper elementary grades
Reading and Writing
Expectancy-value theory
Paper presented at the meeting of the American Educational Research Association
Motivation, beliefs, values, and goals
Annual Review of Psychology
Expectancies, values, and academic behaviors
Clarifying an elusive construct: A systematic review of writing attitudes
Educational Psychology Review
Belief, attitude, intention and behavior
From “what is reading?” to “what is literacy?”
Journal of Education
Disciplinary literacy: Adapt not adopt
Journal of Adolescent & Adult Literacy
A writer(s) within community model of writing
The role of strategies, knowledge, will, and skills in a 30 year program of research (with homage to Hayes, Fayol, and Boscolo)
A meta-analysis of writing instruction for adolescent students
Journal of Educational Psychology
The relationship among strategic writing behavior, writing motivation, and writing performance with young, developing writers
The Elementary School Journal
Effect sizes for research: Univariate and multivariate applications
From spark to fire: Can situational reading interest lead to long-term reading motivation?
Reading Research and Instruction
Examining shifts in preservice teachers’ beliefs and attitudes toward writing instruction
Journal of Early Childhood Teacher Education
Test of written language
Cited by (33)
Enhancing English writing and higher-order thinking skills through computational thinking
2024, Computers and EducationReflexive writing dialogues: Elementary students’ perceptions and performances as writers during classroom experiences
2022, Assessing WritingCitation Excerpt :These less visible elements of the writing process need to involve reflexive self-assessment and focused feedback from expert teachers. Reflexive self-assessment should go beyond grammatical form to include attitudes, beliefs, and priorities in any given writing situation (Gadd et al., 2019; Wright, Hodges, & McTigue, 2019). Students’ writing attitudes and beliefs are important indicators of their writing achievement (Bulut, 2017; Pajares, Johnson, & Usher, 2007).
Relationship between second language English writing self-efficacy and achievement: A meta-regression analysis
2021, Journal of Second Language WritingThe Preservice Teacher Self-Efficacy for Writing Inventory (PTSWI): A tool for measuring beliefs about writing
2021, Assessing WritingCitation Excerpt :This definition aligns with Benson’s (1998) structural piece of validation programs, and mirrors the work done by others when validating measures of writing affect. We also use current research on writing motivation and survey development to approach measuring the specific construct of self-efficacy (e.g., Abdel Latif, 2019; Ling et al., 2021; Wright, Hodges, & McTigue, 2019). Consulting the Standards for Educational and Psychological Testing (2014) published jointly by the American Educational Research Association (AERA), American Psychological Association (APA), and National Council on Measurement in Education (NCME), we followed an inferential program to create and determine evidence toward the validity and reliability of scores for the PTSWI.
Dr. Katherine LandauWright is an Assistant Professor and Director of the Literacy Lab at Boise State University. Dr. Wright is a former middle school English and social studies teacher. Her research aims to motivate and support middle and high school readers and writers in their content-area classes.
Dr. Tracey S. Hodges is an Assistant Professor at the University of Alabama. Dr. Hodges is a former English teacher, which influenced her interest in developing high quality writing teachers. She researches preservice teacher beliefs, writing interventions and instruction in K-12 settings, and student motivation for reading and writing.
Dr. Erin M. McTigue is a Research Scientist and Associate Professor II, in Norway’s National Centre for Reading Education and Reading Research at the University of Stavanger. She focuses on the needs of striving readers, from which she has authored over 50 manuscripts. Previously, Dr. McTigue was a classroom teacher and faculty at Texas A&M University.