Empirical Research
A psychometric comparison of psychological inflexibility measures: Discriminant validity and item performance

https://doi.org/10.1016/j.jcbs.2020.08.007Get rights and content

Highlights

  • Compared the AAQ-II, AAQ-3 (revised AAQ-II), BEAQ, and CompACT.

  • CompACT showed the strongest discriminant validity.

  • CompACT OE and VA subscales showed the most consistent item performance.

  • Utility of measures depends on the goals and context of assessment.

Abstract

Psychological inflexibility is a rigid behavioral pattern that interferes with engagement in personally meaningful activities; it is the hypothesized root of suffering in acceptance and commitment therapy (ACT). Thus, the quality of its measurement affects the research, theory, and practice of ACT. The current study aimed to evaluate the discriminant validity and item performance of four measures of psychological inflexibility: the Acceptance and Action Questionnaire—II (AAQ-II), a revised version of the AAQ-II (AAQ-3), the Brief Experiential Avoidance Questionnaire (BEAQ), and the Comprehensive assessment of Acceptance and Commitment Therapy processes (CompACT). We analyzed data from community (n = 253), student (n = 261), and treatment-seeking samples (n = 140) using exploratory factor analysis and multigroup graded-response models. The CompACT had the strongest discriminant validity followed by the AAQ-3, whereas items in the CompACT Behavioral Awareness and Valued Action subscales performed most consistently across groups. No single measure emerged as clearly superior to others; rather, appropriate selection of measures depends on the goals and context of assessment. Scientific and clinical implications are discussed.

Section snippets

Recruitment

Eligibility criteria for the present study were: (1) at least 18 years old and (2) ability to complete the letter of information and measures in English. Our current sample comprised three groups: undergraduate college students enrolled in psychology classes in the western United States, individuals currently seeking mental health treatment, and community members from Amazon Mechanical Turk (MTurk).

College students were recruited using fliers on campus and online postings on university websites

Psychological inflexibility

Acceptance and Action Questionnaire – II (AAQ-II; Bond et al., 2011). The AAQ-II is a seven-item measure designed to assess psychological inflexibility. Each item is rated on a seven-point scale ranging from 1 (never true) to 7 (always true). Items include “I'm afraid of my feelings” and “Emotions cause problems in my life.” Responses are summed for a total score ranging from 7 to 49; higher scores indicate higher levels of psychological inflexibility. The AAQ-II has been used to assess

Discriminant validity community sample

AAQ-II. Parallel analysis suggested extraction of four factors and model fit indices showed good fit for the four-factor model (TLI = 0.938, RMSR = 0.023, RMSEA = 0.068). Factor loadings from the EFA of the AAQ-II and DASS-21 are presented in Table 2. AAQ-II items loaded on to one factor with one AAQ-II item cross-loading (i.e., loading ≥ 0.30 for more than one factor) with a factor corresponding to the DASS-21 Depression subscale. That is, the latent constructs measured by the items from each

Discussion

The aims of the present study were to examine two aspects of psychometric validity of psychological inflexibility measures: discriminant validity and item-specific performance. The measures selected for the current study were the AAQ-II, a revised AAQ-II (AAQ-3), BEAQ, and CompACT.

Conclusions

Overall, we found that (1) the CompACT performed most consistently in terms of discriminant validity followed by the AAQ-3, and (2) the Behavioral Awareness and Valued Action subscales were most robust in terms of having consistent sensitivity to individual differences in psychological (in)flexibility. However, the CompACT produced a different factor structure from that in its psychometric development analyses (Francis et al., 2016), potentially pointing to poor structural validity, and showed

Declaration of competing interest

None.

References (42)

  • H.J. Ojalehto et al.

    Adherence to exposure and response prevention as a predictor of improvement in obsessive-compulsive symptom dimensions

    Journal of Anxiety Disorders

    (2020)
  • C. Rochefort et al.

    Experiential avoidance: An examination of the construct validity of the AAQ-II and MEAQ

    Behavior Therapy

    (2018)
  • R.D. Rogge et al.

    It's all about the process: Examining the convergent validity, conceptual coverage, unique predictive validity, and clinical utility of ACT process measures

    Journal of Contextual Behavioral Science

    (2019)
  • M. Thompson et al.

    Preliminary psychometric properties of the Everyday psychological inflexibility checklist

    Journal of Contextual Behavioral Science

    (2019)
  • M.P. Twohig et al.

    Exposure therapy for OCD from an acceptance and commitment therapy (ACT) framework

    Journal of Obsessive-Compulsive and Related Disorders

    (2015)
  • I. Tyndall et al.

    The Acceptance and Action Questionnaire-II (AAQ-II) as a measure of experiential avoidance: Concerns over discriminant validity

    Journal of Contextual Behavioral Science

    (2019)
  • M. Wolgast

    What does the Acceptance and Action Questionnaire (AAQ-II) really measure?

    Behavior Therapy

    (2014)
  • T. Asparouhov et al.

    2018. SRMR in mplus

  • T.S. Barrett et al.

    Furniture for quantitative scientists

    R Journal

    (2017)
  • M. Buhrmester et al.

    Amazon's mechanical Turk: A new source of inexpensive, yet high-quality data?

    (2016)
  • A.B. Feinstein et al.

    Pain intensity, psychological inflexibility, and acceptance of pain as predictors of functioning in adolescents with juvenile idiopathic arthritis: A preliminary investigation

    Journal of Clinical Psychology in Medical Settings

    (2011)
  • Cited by (50)

    • Acceptance and Commitment Therapy Processes and Mediation: Challenges and How to Address Them

      2023, Behavior Therapy
      Citation Excerpt :

      Conclusions about process variables in psychotherapy are only as valid and reliable as the measures themselves. As noted above, independent research groups have seriously challenged the validity and reliability of ACT’s most widely used process measures: the AAQ and AAQ-II (Allen, 2021; Ong et al., 2020; Rochefort et al., 2018; Tyndall et al., 2019; Wolgast, 2014). What guidelines can be applied to more rigorously develop and validate ACT process measures?

    View all citing articles on Scopus
    View full text