Elsevier

Assessing Writing

Volume 39, January 2019, Pages 64-78
Assessing Writing

A validation program for the Self-Beliefs, Writing-Beliefs, and Attitude Survey: A measure of adolescents' motivation toward writing

https://doi.org/10.1016/j.asw.2018.12.004Get rights and content

Highlights

  • Summarizes validity of the Self-Beliefs, Writing-Beliefs, & Attitude Survey (SWAS).

  • The SWAS is a valid and reliable measure of adolescents’ writing motivation.

  • SWAS scores correlate with writing achievement and teacher ratings of writing.

  • Propose a new multi-dimensional model of writing motivation.

  • Grounded in Expectancy Value Theory and Theory of Attitude.

Abstract

Recent findings reveal clear evidence that students’ low performance on writing tasks is often related to problems with motivation. Writing curriculum and interventions produce varying effects on adolescents’ writing outcomes, and such variations may be mediated by motivation. However, without a valid tool for measuring students’ motivation towards writing, these effects cannot be quantified. In this study we present the results of our multi-study validation program for the Self-Beliefs, Writing-Beliefs, and Attitude Survey (SWAS). This measure is designed for monitoring students' motivation towards writing, as well as identifying variables that mediate student achievement. We first addressed substantive validation through a thorough review of research. Next, in Study 1, we established structural validity through multiple types of factor analyses and establishing reliability coefficients for the instrument scores. Finally, in Study 2, we provide evidence of external validity by comparing students’ SWAS scores to other measures of writing. Following these procedures, we were able to establish that the SWAS provides a valid measure of students’ writing motivation and is an instrument appropriate for a particularly important age group – adolescent learners. Additionally, through this process, we add to the theoretical base by proposing a new multi-dimensional model of writing motivation.

Introduction

Teachers acknowledge that typical writing instruction for adolescents may not always inspire motivation (Troia & Maddox, 2004). In early grades, although instruction focuses primarily on mechanics, students are often allowed to write on topics of their choice (Katz & Assor, 2007). However, by middle school, writing autonomy diminishes as the focus shifts and students are required to produce discipline-specific texts. At the same time, in secondary grades, most students complete little extended or complex writing allowing minimal opportunity for creativity and expression (Applebee & Langer, 2009; Graham & Harris, 2012). This situation has contributed to 74% of 8th graders and 73% of 12 graders scoring at the “basic” or “below basic” level on national writing tests. We maintain that writing instruction that does not emphasize motivation may be contributing to students’ low performance (National Center for Education Statistics, 2012).

Researchers are still exploring what motivates adolescents to write, but recent findings reveal clear evidence that students’ performance on writing tasks is related to motivation (Zumbrunn, Marrs, & Mewborn, 2016). As motivation for writing can vary depending upon the situation and task, teachers and scholars require valid assessments to determine whether writing skills are primarily a result of cognitive ability and knowledge, affective issues (e.g., motivation), or both. If students are not motivated to write, they may be viewed as having poor writing skills, even if this is not true (Johnston & Costello, 2005). Being identified as having underdeveloped skills may place students on a trajectory of specific coursework and lowered expectations. However, without a valid tool for measuring students’ motivation towards writing, these effects cannot be quantified.

Researchers also know that writing instruction and interventions have varying effects on adolescents’ outcomes (Graham & Perin, 2007; Washburn, Sielaff, & Golden, 2016), which are likely mediated by motivation. For example, Webb, Vandiver, and Jeung (2016) found that a course focused on developing the writing process supported students’ self-efficacy for writing better than courses focused creative or analytical writing. As such, a deeper understanding of adolescent students’ writing affect, in the context of educational interventions, will help identify practices that produce gains in skills while preventing (or reversing) the development of negative self-concepts. Furthermore, a child who has had negative experiences with writing will likely develop a poor attitude toward that experience (Fishbein & Ajzen, 1975) and potentially believe there is great personal cost in engaging in the task (Eccles & Wigfield, 2002). If we extrapolate these attitudes to writing, we can conclude that if a child has had negative experiences with writing, he or she may have a poor attitude toward writing and believe that writing has little value.

In this study we present the results of our two-study validation program for the Self-Beliefs, Writing-Beliefs, and Attitude Survey (SWAS). This measure is designed for monitoring students' motivation towards writing, as well as identify variables that mediate student achievement. To date, existing tools contain important limitations for measuring adolescent students’ affect towards writing. For example, select measures only consider one aspect of writing affect (e.g., Kear, Coffman, McKenna, & Ambrosio, 2000), yet we know that motivation is a complex construct. Therefore, we worked to create a multidimensional instrument. Furthermore, other measures present relatively low reliability (e.g., Troia, Harbaugh, Shankland, Wolbers, & Lawrence, 2013). As reliability is an essential component of validity (Thompson, 2003), we enacted a multistep process for assessing reliability. Most importantly, few measures have undergone rigorous, multi-step validation programs to ensure their scores measure the intended writing constructs.

Validation is a critical component of measurement development, because only through this process do scores take on meaning. Most contemporary researchers report statistical validity (such as factor analyses and/or structural equation modeling) and include reliability estimates (e.g., see Collie, Martin, & Curwood, 2016; Troia et al., 2013). These are essential steps towards validation, but further work is required. Benson (1998) proposed that a strong validation program has three distinct components: the substantive, the structural, and the external. Substantive refers to the collection of existing theoretical and empirical information to define constructs, which we addressed through a thorough review of research. The structural piece entails determining how the observed variables relate to one another and the main construct of interest, so we conducted multiple types of factor analyses (Study 1). Finally, the external component requires the researcher to determine how the measure relates to other expected constructs (Benson, 1998), which we addressed by comparing students’ SWAS scores to other measures of writing (Study 2).

Beyond instrument development, through the process of theoretical reviews and empirical validation, we aim to propose a model of writing motivation for adolescents. Specifically, we present the findings of two studies aimed at establishing the validity of the SWAS. Our research is guided by the following questions:

  • 1

    How valid are the constructs measured by the Self-Beliefs, Writing-Beliefs, and Attitude Survey?

  • 2

    What aspects of students’ affect towards writing are measured by the Self-Beliefs, Writing-Beliefs, and Attitude Survey?

Section snippets

Prior research and theoretical foundations

Writing is a skill that “allows people to communicate with others removed in both distance and time” (Graham & Perin, 2007, p. 445). Furthermore, following the work of Myers et al. (2016) writing is conceptualized as a “complex cognitive, physical, affective, and social” process, requiring writers to “use multiple skills and strategies as they move through the stages of planning, drafting, revising, editing publishing, and presentation” (p. 312). Therefore, we define writing as the skills and

SWAS instrument development & substantive validation

After examining existing tools for measuring writing affect, we selected the Motivation for Reading Questionnaire (MRQ) (Wigfield & Guthrie, 1995) to provide an initial structure for our measure. While the MRQ does not pertain to writing, it provides a theoretically-supported model for assessing student motivation. This tool has been repeatedly used to measure students’ (including adolescents) motivation towards reading, and consistently produces valid scores (e.g., Guthrie et al., 2006; Mason,

Survey administration

We administered the SWAS to 517 students at a public middle school (grades six through eight), in the southwestern United States. Students completed paper-based versions of the survey, and total administration took about 15 min. Teachers read the entire survey to students to ensure that results were not impacted by students’ reading ability.

Both the university and local school-level Institutional Review Boards approved our plan to administer this survey without seeking parental permission if

Survey administration

We administered the SWAS to a second, independent, group of participants. Students completed a paper-based version of the survey in approximately 15 min. We used this administration to complete the external stage of our construct validation program. In this study, we solicited parental consent and collected more information about individual students, including teacher ratings and writing samples.

Participants and school context

The sample consisted of 53 students in grades six through 11 who attended a private, independent

General discussion & model of writing motivation

Regarding our overarching theoretical question, the model that best fit the data is based upon a combination of theory and psychometrics (see Fig. 1) with attitudes and belief system representing two unique contributions to motivation, which was predicted by our review of the literature.

This model overlaps with aspects of Conradi et al.’s (2014) proposed hierarchy of reading motivation-related constructs. Their hierarchy conceptualized two categories related to beliefs – those about the self

Limitations & future directions

Despite these rigorous validation procedures, we recognize that limitations remain. Unfortunately, we were not able to collect data about students in Study 1 and therefore could not analyze class-level and individual characteristics, such as language-learning status or gifted and talented enrollment. In particular, future research should conduct measurement invariance analyses to assess whether the constructs of the SWAS function equivalently across gender and skill level. Additionally, we

Conclusions

While teachers certainly agree that an adolescents’ motivation for writing greatly enhances (or reduces) their growth, such discussions of writing motivation has often remained mired at the level of anecdote. Therefore, to make the previously “intangible” into a tangible construct that we can more systematically work to improve, we first need tools to help us measure these constructs. We need to move writing motivation from anecdote to science.

To develop such a measure, especially for an

Declarations of interest

None.

Dr. Katherine LandauWright is an Assistant Professor and Director of the Literacy Lab at Boise State University. Dr. Wright is a former middle school English and social studies teacher. Her research aims to motivate and support middle and high school readers and writers in their content-area classes.

References (67)

  • R. Bruning et al.

    Examining dimensions of self-efficacy for writing

    Educational Psychology

    (2013)
  • L.M. Clearly

    I think I know what my teachers want now: Gender and writing motivation

    The English Journal

    (1996)
  • R.J. Collie et al.

    Multidimensional motivation and engagement for writing: Construct validation with a sample of boys

    Educational Psychology

    (2016)
  • K. Conradi et al.

    Motivation terminology in reading research: A conceptual review

    Educational Psychology Review

    (2014)
  • G. Cumming

    Understanding the new statistics: Effect sizes, confidence intervals, and meta-analysis

    (2012)
  • E.L. Deci et al.

    Motivation and education: The self-determination perspective

    Educational Psychologist

    (1991)
  • Z. Dörnyei

    Motivation in action: Towards a process-oriented conceptualization of student motivation

    The British Journal of Educational Psychology

    (2000)
  • E. Drijbooms et al.

    How executive functions predict development in syntactic complexity of narrative writing in the upper elementary grades

    Reading and Writing

    (2017)
  • J.S. Eccles et al.

    Expectancy-value theory

    Paper presented at the meeting of the American Educational Research Association

    (2017)
  • J.S. Eccles et al.

    Motivation, beliefs, values, and goals

    Annual Review of Psychology

    (2002)
  • J.S. Eccles et al.

    Expectancies, values, and academic behaviors

  • E. Ekholm et al.

    Clarifying an elusive construct: A systematic review of writing attitudes

    Educational Psychology Review

    (2017)
  • M. Fishbein et al.

    Belief, attitude, intention and behavior

    (1975)
  • K.K. Frankel et al.

    From “what is reading?” to “what is literacy?”

    Journal of Education

    (2016)
  • V. Gillis

    Disciplinary literacy: Adapt not adopt

    Journal of Adolescent & Adult Literacy

    (2014)
  • S. Graham

    A writer(s) within community model of writing

  • S. Graham et al.

    The role of strategies, knowledge, will, and skills in a 30 year program of research (with homage to Hayes, Fayol, and Boscolo)

  • S. Graham et al.

    A meta-analysis of writing instruction for adolescent students

    Journal of Educational Psychology

    (2007)
  • S. Graham et al.

    The relationship among strategic writing behavior, writing motivation, and writing performance with young, developing writers

    The Elementary School Journal

    (2017)
  • R.J. Grissom et al.

    Effect sizes for research: Univariate and multivariate applications

    (2012)
  • J.T. Guthrie et al.

    From spark to fire: Can situational reading interest lead to long-term reading motivation?

    Reading Research and Instruction

    (2006)
  • A.H. Hall

    Examining shifts in preservice teachers’ beliefs and attitudes toward writing instruction

    Journal of Early Childhood Teacher Education

    (2016)
  • D.D. Hammill et al.

    Test of written language

    (2009)
  • Cited by (33)

    • Reflexive writing dialogues: Elementary students’ perceptions and performances as writers during classroom experiences

      2022, Assessing Writing
      Citation Excerpt :

      These less visible elements of the writing process need to involve reflexive self-assessment and focused feedback from expert teachers. Reflexive self-assessment should go beyond grammatical form to include attitudes, beliefs, and priorities in any given writing situation (Gadd et al., 2019; Wright, Hodges, & McTigue, 2019). Students’ writing attitudes and beliefs are important indicators of their writing achievement (Bulut, 2017; Pajares, Johnson, & Usher, 2007).

    • The Preservice Teacher Self-Efficacy for Writing Inventory (PTSWI): A tool for measuring beliefs about writing

      2021, Assessing Writing
      Citation Excerpt :

      This definition aligns with Benson’s (1998) structural piece of validation programs, and mirrors the work done by others when validating measures of writing affect. We also use current research on writing motivation and survey development to approach measuring the specific construct of self-efficacy (e.g., Abdel Latif, 2019; Ling et al., 2021; Wright, Hodges, & McTigue, 2019). Consulting the Standards for Educational and Psychological Testing (2014) published jointly by the American Educational Research Association (AERA), American Psychological Association (APA), and National Council on Measurement in Education (NCME), we followed an inferential program to create and determine evidence toward the validity and reliability of scores for the PTSWI.

    View all citing articles on Scopus

    Dr. Katherine LandauWright is an Assistant Professor and Director of the Literacy Lab at Boise State University. Dr. Wright is a former middle school English and social studies teacher. Her research aims to motivate and support middle and high school readers and writers in their content-area classes.

    Dr. Tracey S. Hodges is an Assistant Professor at the University of Alabama. Dr. Hodges is a former English teacher, which influenced her interest in developing high quality writing teachers. She researches preservice teacher beliefs, writing interventions and instruction in K-12 settings, and student motivation for reading and writing.

    Dr. Erin M. McTigue is a Research Scientist and Associate Professor II, in Norway’s National Centre for Reading Education and Reading Research at the University of Stavanger. She focuses on the needs of striving readers, from which she has authored over 50 manuscripts. Previously, Dr. McTigue was a classroom teacher and faculty at Texas A&M University.

    View full text