You are here

Article Text

Article menu

Download PDFPDF

Systematic review
How to measure mental pain: a systematic review assessing measures of mental pain
Free
  1. Camille Charvet1,
  2. Isabelle Boutron2,3,
  3. Yannick Morvan4,5,
  4. Catherine Le Berre2,3,
  5. Suzanne Touboul6,
  6. Raphaël Gaillard7,
  7. Eiko Fried8,
  8. Astrid Chevance2,3
  1. 1 Medical School, Sorbonne Université, FR-75006, Paris, France
  2. 2 CRESS U1153, Université Paris-Cité, Inserm, FR-75006, Paris, France
  3. 3 Service d'épidémologie clinique, APHP, GHU Cochin-Hôtel Dieu, FR-75005, Paris, France
  4. 4 CESP, Inserm, Maison de Solenn, FR-75005, Paris, France
  5. 5 Laboratoire CLIPSYD, EA4430, Université Paris-Nanterre, FR-92000, Nanterre, France
  6. 6 Person with Lived Experience, Paris, France
  7. 7 Department of Psychiatry, Service Hospitalo-Universitaire, GHU Paris Psychiatry &Neurosciences, FR-75014, Paris, France
  8. 8 Department of Clinical Psychology, Leiden University, Leiden, The Netherlands
  1. Correspondence to Dr Astrid Chevance, Public health, University of Paris, Paris, France; astrid.chevance{at}gmail.com

Abstract

Question Although mental pain is present in many mental disorders and is a predictor of suicide, it is rarely investigated in research or treated in care. A valid tool to measure it is a necessary first step towards better understanding, predicting and ultimately relieving this pain.

Study selection and analysis Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we performed a systematic review to identify all published standardised measures of mental pain. We used qualitative content analysis to evaluate the similarity of each measure, quantified via Jaccard Index scores ranging from no similarity (0) to full similarity (1). Finally, using the Consensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology, we evaluated each measure’s development (assessing 35 features), its content validity (31 features) and if the latter was rated at least adequate, its other psychometric properties.

Findings We identified 10 self-reported scales of mental pain in 2658 screened studies relying on diverse definitions of this construct. The highest average similarity coefficient for any given measure was 0.24, indicative of weak similarity (individual pairwise coefficients from 0 to 0.5). Little to no information was provided regarding the development and the content validity of all 10 scales. Therefore, their development and content validity were rated ‘inadequate’ or ‘doubtful’.

Conclusions and clinical implications There is not enough evidence of validity to recommend using one measure over others in research or clinical practice. Heterogeneous use of disparate measures across studies limits comparison and combination of their results in meta-analyses. Development by all stakeholders (especially patients) of a consensual patient-reported measure for mental pain is needed.

PROSPERO registration number CRD42021242679.

  • suicide & self-harm
  • adult psychiatry
  • depression & mood disorders
  • anxiety disorders
  • substance misuse

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information.

https://doi.org/10.1136/ebmental-2021-300350

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Summary box

    What is already known on this topic

    • Mental pain is a life-threatening outcome important to patients, found in a variety of mental disorders, yet rarely evaluated or treated in routine clinical practice.

    • A valid tool to measure mental pain is a necessary step to improve both research and clinical practice.

    What this study adds

    • Our systematic review identified 10 patient-reported measures of mental pain which vary substantially in their theoretical frameworks, definitions of mental pain, semantics and content.

    • We cannot currently recommend one scale over any other for research or clinical purposes, mainly because of the lack of available data to evaluate the tools’ development and content validity.

    How this study might affect research, practice or policy

    • The existing measures can however be understood as important first efforts to gain an understanding of mental pain paving the way towards improving the conceptualisation of mental pain and its measurement in research and practice.

    Introduction

    Mental pain is a risk factor for suicidal behaviour, independently of other constructs such as depression or hopelessness.1–5 In a recent qualitative study, 1912 patients, 627 healthcare professionals and 464 informal caregivers in 52 countries listed mental pain as the fifth most important outcome in depression, describing it as ‘unbearable’ and similar to ‘torture’.6 This stresses the importance of mental pain as a treatment target. Moreover, mental pain has been reported as part of other mental disorders, including obsessive-compulsive disorder, post-traumatic stress disorder and borderline personality disorder, as well as somatic conditions such as migraine.7–12 During the COVID-19 pandemic, mental pain was one of the negative psychological outcomes observed in general population settings.13 Neuroscientific investigations of mental pain tend to show common pathways with physical pain (eg, activation of the anterior cingulate cortex and the insula, mostly when confronted with social rejection, as well as involvement of opioid µ-receptors).14–17 Given the ubiquity of mental pain, it is no surprise that it has been proposed as a transdiagnostic construct that should be assessed via patient-reported outcome measures (PROMs).18 Despite growing evidence of the need for its routine assessment, it is rarely measured in clinical trials or evaluated in clinical practice.6 7

    Identifying a valid measure of mental pain is a first step in developing research to help address this outcome in clinical practice. Interestingly, like other constructs such as depression, no consensus exists for the definition of mental pain.19 In the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), mental pain is defined in the introductory section as a synonym for the ’distress associated with the symptoms’.20 This makes mental pain a relevant component of every diagnosis with a diagnostic criterion including ‘significant clinical distress’.20 The history of this construct reveals that mental pain emerged in the psychiatric literature early during the 19th century as one of the three core symptoms of melancholia with depressed mood and suicidal ideas/behaviours in the work of Krafft-Ebing, Séglas, Guislain, Clouston, Maudsley and Griesinger.21–23 These authors used different and not completely overlapping terminology, including ‘psychischer Schmerz’, ‘psychalgia’, ‘douleur morale’ and ‘phrénalgie’.22 23 In the 1990s, Shneidman qualitatively analysed hundreds of suicide notes and proposed a model for suicide based on frustrated psychological needs.24 He coined the term ‘psychache’ defined as ‘the hurt, anguish, soreness, aching, psychological pain in the psyche, the mind’.25 Other authors have proposed newer definitions, for example, ‘a lasting, unsustainable and unpleasant feeling resulting from negative appraisal of an inability or deficiency of the self’ (Meerwijk) or ‘stemming from the discrepancy between the ideal and the actual perception of self, accompanied by the awareness of one’s role in the experience of emotional pain’ (Tossani).7 26

    Simultaneously, there is no consensus about what measures should be used in scientific research or clinical practice. We found no study that systematically evaluated and compared measures of mental pain to help clinicians and researchers choose appropriate scales for specific uses. A few narrative reviews cover some measures of mental pain, but these studies have three limitations.2 7 27 First, they were not systematic and might have missed relevant measures. Second, the theoretical conceptualisations of mental pain underlying these definitions and hence their operationalisation in these measures may not be similar or even convertible, an important point because the lack of standardisation of measures across studies prevents their comparison and combination, thereby causing research waste.7 28–30 Third, previous reviews have not assessed the development of the measures, nor have they compared their psychometric properties. The frequency at which measurement instruments of psychological dimensions lack validity suggests that this may also be true of mental pain scales.19 31

    These issues call for a systematic effort to review mental pain measures, and the evidence of their validity, and collect the definitions that support them—the aim of this study.

    Objective

    The aim of this study was to (1) identify all existing measures stated by their authors to assess mental pain and the definitions that underlie them, (2) evaluate the similarities of the measures and (3) evaluate and compare their development and psychometric properties.

    Study selection and analysis

    Following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist, we conducted a systematic review to identify all existing tools measuring mental pain. We then determined the content overlap of each measure and evaluated their similarity by both qualitative and quantitative methods. Finally, we evaluated the development of each measure and its psychometric properties by applying the Consensus-based Standards for the selection of health Measurement Instruments (COSMIN).32 33 For a global overview of the study design, see online supplemental material 1. The study protocol is available on Zenodo (4603560) and PROSPERO (CRD42021242679).

    Supplemental material

    Systematic review to identify measures of mental pain

    Eligibility criteria

    We defined a measure of mental pain as any standardised tool whose authors claim specifically that it measures ‘mental pain’ or ‘psychological pain’ or ‘psychache’ or ‘psychic pain’. Since our aim was to identify tools to measure mental pain for the purpose of making recommendations about their use in clinical research or practice, we chose to focus on tools that have been used in peer-reviewed published studies. When there were several versions of a single tool (eg, shorter scales), we only included the first version that had at least one development and validation publication. We mention the other versions but we do not count them as distinct tools.

    We included peer-reviewed scientific publications (eg, clinical studies, comments, letters to editors and reviews) that either mentioned the names of standardised measures of mental pain OR reported mental pain as an outcome measured in the study, without restriction of date or language or target population (eg, age, gender, country or disorder). We included all mental and physical health conditions. We excluded studies not involving humans, not using standardised instruments and using measures that assessed mental pain only in a subscale or by a few items among a larger set of items measuring other constructs. We excluded studies using measures designed for contexts other than clinical research or practice (such as the determination of non-material damage in legal actions). We excluded studies that stated they measured mental pain but used tools purporting to measure other constructs such as distress or depression.

    Information sources and search strategy

    As recommended by Cochrane, we systematically searched three databases relevant for psychiatry and psychological studies: PubMed, EMBASE and PsychINFO, from their dates of inception through 8 February 2022.34 We searched for ‘mental pain’, as well as ‘psychological pain’, ‘psychache’ and ‘psychic pain’.7 Online supplemental material 2 presents the full search queries.

    Selection of the relevant scientific publications

    Two investigators (either CC or CLB and AC) independently screened the articles by title and abstract, using Rayyan, a free software program for systematic reviews. Disagreements were resolved by consensus. Online supplemental material 3 lists all the included papers.

    Data extraction process

    The full text of all studies selected by title and abstract was examined by one investigator (either CC or CLB) to identify potential measures of mental pain and their development/validation studies. A second investigator checked a randomly selected 30% of all screened studies (AC).

    Data extraction form

    The data charting form (online supplemental material 4) included all information recommended by COSMIN to evaluate the development, content validity and further measurement properties of each tool.33

    To provide a picture of the scientific use of the measures of mental pain, we also extracted the field of study, the study design (review, observational study, development and validation study, effectiveness study) and the population in which the measure was used. Finally, we evaluated whether mental pain was the main outcome of the study and if it involved suicidal behaviours.

    Analysis and synthesis of results

    All identified measures are presented in table 1 reporting their name with the first development and validation publication, whether it is a PROM, a brief description of the tool, any different versions of it, the underlying conceptual framework, its definition of mental pain and the initial purpose for which the measure was developed.

    View this table:
    Table 1

    Description of the 10 measures of mental pain

    Content overlap and similarity of the measures

    To investigate the similarity between the measures, we first conducted a semantic analysis to identify all the different words used to denominate the pain and its location, then performed a qualitative content analysis to identify the different domains assessed by each measure. Then we represented the content overlap graphically, and finally calculated a Jaccard Similarity Index. This method is explained extensively elsewhere and has been duplicated in several studies with, for example, depression scales, anxiety scales and mania scales.28 35 36

    Two researchers with professional experience with mental pain (AC and CC) and the patient with lived experience of mental pain (ST) conducted the semantic and qualitative content analyses. For the semantic analysis, they identified each word used to name or describe the pain and its location. In the qualitative content analysis, they coded all the different domains measured by each item of each scale by using their professional and experiential knowledge (each domain was coded with a specific number). Similarly worded items, as well as reverse worded items, were coded as the same domains. For example, “I cope with the pain even though it is difficult to bear” (Tolerance of Mental Pain Scale) and “I can’t take my pain anymore” (Psychache Scale) were classified as ‘unbearable’. Metaphors were coded according to their latent content; for instance, “my pain makes my life seem dark” (Psychache Scale) was coded as ‘hopelessness’ and “it feels like my heart crunched up” (Three-Dimensional Psychological Pain Scale) as ‘to be destroyed’. Some scale items were placed in multiple domains simultaneously (one item received several codes). For example, the item “When my feelings are intense, I can’t think straight” (Psychic Pain Scale) measures both ‘overflooded’ and ‘cognitive impairment’. Our conservative approach to avoid overestimating heterogeneity among and between scales assumed, when we hesitated between categories, that the items were similar rather than distinct.28

    We then represented the content overlap of the measures in figure 1.

    Figure 1
    Figure 1

    Content overlap of the different measures of mental pain. Each grey concentric circle represents a measurement tool (from outer circle to inner circle: Orbach and Mikulincer Mental Pain Scale (OMMPS), Psychological Pain Assessment Scale (PPAS), Tolerance of Mental Pain Scale (TMPS), Psychache Pain Scale (PPS), Three-Dimensional Psychological Pain Scale (TDPPS), Mental Pain Questionnaire (MPQ), Psychache Scale (PAS), Mee-Bunney Psychological Assessment Pain Scale (MBPPS), Physical and Psychological Pain Visual Analogue Scale (PPP-VAS) and Unbearable Psychache Scale (UP3)). The qualitative content analysis of the 112 items of the 10 measures retrieved 35 domains. A coloured dot on a grey concentric circle means that the domain is measured by the corresponding scale. For instance, 'anxiety' is measured by the PPP-VAS and the PPS. The 35 domains are presented anticlockwise around the circle, from the most frequently measured (seven measures assess pain intensity) to the least often measured (one measure assesses anger).

    Finally, to determine the similarity of the content of the measures, we calculated a Jaccard Similarity Index; this coefficient of similarity for binary variables ranges from 0 (no similarity) to 1 (full similarity). The index is estimated based on the formula s/(u1+u2+s), where s represents the number of shared domains assessed by two measures, and u1 and u2 are the number of domains assessed respectively in the first and the second measure.28 A rule of interpretation is that a Jaccard coefficient of 0.00–0.19 indicates very weak similarity, 0.20–0.39 weak similarity, 0.40–0.59 moderate similarity, 0.60–0.79 strong similarity and 0.80–1 very strong similarity.28

    Evaluation and comparison of the development, content validity and other psychometric properties of the measures of mental pain

    We evaluated each measure with COSMIN (online supplemental material 5a and 5b).33 This method applies a filter approach to evaluate successively: (1) the measure’s development, (2) its content validity and (3) its further psychometric properties, if and only if the content validity was rated ‘adequate’ or ‘very good’.33

    Information sources to evaluate the development and content validity

    We used three sources to evaluate the development and content validity of the measures. First, we used all studies tagged ‘development and validation study’ during the data extraction process of the systematic review. Second, we screened all references of these studies to identify further references.32 33 Third, we emailed each corresponding author of each tool to request all relevant further development/validation materials (published and unpublished studies).32 33 Only the authors of the Three-Dimensional Psychological Pain Scale (TDPPS) (Li et al) failed to respond. The final sample of studies is listed in online supplemental material 6.

    Evaluation of the measures

    Three investigators with different backgrounds and expertise (CC, AC and YM) independently rated development, content validity and, if content validity was adequate, all other measurement properties. Disagreements, if any, were resolved by consensus with experts. The investigators reported no conflict of interest with any of these measures.

    First, the investigators evaluated the 35 features of each measure’s development, classified in two categories: (1) 'design' (eg, the underlying conceptual framework, description of the target population and appropriateness of the qualitative data collection used to identify relevant items) and (2) ‘cognitive interview study or pilot test’ (eg, was a cognitive interview performed and were patients asked about the comprehensiveness of the measure?). Details of the 35 features can be found in box 1 of the COSMIN methodology for content validity user manual, available on their website and reproduced in online supplemental material 5a. Each of the 35 features is to be rated ‘very good’, ‘adequate’, ‘doubtful’, ‘inadequate’ or ‘not applicable’. Following the COSMIN recommendation that ‘poor methodological aspects of a study cannot be compensated by [different] good aspects’, the overall rating of each primary category is the lowest score of the features. However, COSMIN differentiates ‘fatal flaws’, which lead automatically to the rating of the entire category as ‘inadequate’, from other flaws that are rated adequate or doubtful, to lower their impact in global scoring.33 For instance, not using an appropriate qualitative data collection method to identify relevant items for a new PROM leads to the rating of the overarching ‘design’ category as ‘inadequate’.

    Second, we evaluated the content validity of each measure of mental pain. COSMIN defines three critical aspects of content validity: relevance (relevance of the items and response options for the target population, in the context of use), comprehensiveness (no key concepts missing) and comprehensibility (are the instructions, items and response options appropriately worded and understood by the target population). This definition is operationalised in 31 features, classified in 5 main categories: (1) ‘asking patients about the relevance of PROM items’, (2) ‘asking patients about comprehensiveness’, (3) ‘asking patients about comprehensibility’, (4) ‘asking professionals about relevance’ and (5) ‘asking professionals about comprehensiveness’. Details of the 31 features can be found in box 2 of the ‘COSMIN methodology for content validity’ user manual (online supplemental material 5b). These features were clarified by sample questions applied to each relevant item, such as ‘Was an appropriate method used for assessing the relevance/comprehensiveness/comprehensibility of the PROM?’ ‘Were skilled moderators used?’ ‘Was each item tested in an adequate number of patients?’ ‘Were professionals from all relevant disciplines included?’ Each of the 31 domains can be rated ‘very good’, ‘adequate’, ‘doubtful’, ‘inadequate’ or not applicable. Like the assessment of development, the overall rating is based on the lowest score of the features.

    Third, we evaluated the other psychometric properties (structural validity, internal consistency, cross-cultural validity, measurement invariance, reliability, measurement error, criterion validity, hypothesis testing for construct validity and responsiveness) if and only if the content validity was rated above the level of ‘inadequate’ in the previous step.33

    Findings

    Identification of the measures of mental pain and description

    Data extraction on 8 February 2022 retrieved 2658 publications, with 299 full texts searched resulting in the final inclusion of 167 unique studies (flow chart in online supplemental material 7). Among them, we identified 107 observational studies, 38 development and validation papers, 11 reviews, 1 meta-analysis and 10 intervention studies (3 of them randomised controlled trials). Among the 153 primary research projects involving participants, most took place in North America (n=49) and Europe (n=44). Studies including clinical populations (n=75) mostly involved people with mood disorders (n=51). Overall, 97 studies investigated the relation between mental pain and suicidal ideation and behaviour. See online supplemental material 8 for a more detailed description of the included studies.

    We identified 10 different standardised measures of mental pain (table 1). All of these measures are self-rated scales published between 1999 and 2021 (cited here from the oldest to the newest): the Psychological Pain Assessment Scale (PPAS), Psychache Scale (PAS), Orbach and Mikulincer Mental Pain Scale (OMMPS), Tolerance of Mental Pain Scale (TMPS), Physical and Psychological Pain Visual Analogic scale (PPP-VAS), Mee-Bunney Psychological Assessment Pain Scale (MBPPS), Three-Dimensional Psychological Pain Scale (TDPPS, Mental Pain Questionnaire (MPQ), Unbearable Psychache Scale (UP3) and Psychache Pain Scale (PPS) (table 1). The OMMPS has an original version composed of 44 items, and four shorter versions (41, 40, 31, 8 items), resulting from attempts to validate the scale in different samples (see references in online supplemental material 6). The original TMPS comprised 20 items, but a 10-item version was validated in three different samples (the USA, Turkey, Italy) (online supplemental material 6). The PPS has an original unpublished version of 20 items and a published version of 12 items (online supplemental material 6).

    Table 2 presents the underlying definitions of mental pain as reported by the authors of each measure. Five measures used Shneidman’s definition. Seven measures were initially developed to explore the relations between mental pain and suicide (PPAS, PAS, OMMP, TMPS, PPP-VAS, MPQ, PPS), and three were developed to predict and prevent suicide in routine clinical practice (MBPPS, TDPPS, UP3) (table 1).

    View this table:
    Table 2

    Definitions of mental pain underlying each measure of mental pain

    The measure cited most often in the studies included in our review is the PAS (85 of 167 identified studies), followed by the OMMPS (44 studies) and the PPP-VAS (21 studies) (table 1).

    Content overlap and similarity of the measures

    Semantic analysis of the 10 measures identified 7 different words to designate the pain: pain, hurt, ache, suffering, torment, misery and anguish. Four measures used only the word ‘pain’ (OMMPS, PPP-VAS, MPQ, UP3) in framing the items; four used two different words (TMPS, MBPPS, TDPPS, PPS); and two used three or more different words (PPAS, PAS). Across the measures, the pain was described as ‘psychological’, ‘emotional‘ or felt ‘inside’, ‘in the mind’, ‘in the heart’, ‘in the whole body’ or ‘everywhere’. The qualification of ‘psychological’ was used most frequently—in five measures (PPAS, PAS, PPP-VAS, MBPPS, TDPPS); ‘mind’ was used in four (PPAS, PAS, TMPS, TDPPS), ‘inside’ in four (PPAS, PAS, TMPS, PPS), ‘heart’ in three (PPAS, TDPPS, MPQ) and finally ‘emotional’ (TDPPS), ‘everywhere’ (MPQ) and ‘whole body’ (OMMPS) were used in only one measure each. Two tools specified the pain’s localisation ‘in the mind’, as ‘psychological’ or ‘emotional’ only (PPP-VAS and MBPPS), whereas three measures only mentioned locations that were not explicitly mental, such as ‘inside’, ‘everywhere’ or in the ‘heart’ or ‘whole body’ (OMMPS, MPQ, PPS). The UP3 mentions no location for the pain.

    The 10 measures together feature 112 items. The qualitative content analysis of all 112 retrieved 35 domains. For clarity, we present them in five broad categories: pain characterisation (10 domains, eg, ‘intensity’ and ‘frequency’), associated emotions (13 domains, eg, ‘emptiness’ and ‘sadness’), its functional consequences (7 domains, eg, ‘cognitive impairment’ and ‘suicidality’), destruction of the self (4 domains, eg, ‘self-estrangement’ and ‘feeling destroyed’) and a measure of physical pain (one domain: ‘current physical pain’).

    Figure 1 presents the content overlap between all measures. No domain was common to all 10 measures, and 42.9% (15/35 domains) appeared in only one or two measures. ‘Pain intensity’ was the domain of mental pain appearing in the most measures (7/10), followed by ‘feeling destroyed’, ‘pain tolerance’, ‘suicidality’, ‘helplessness’, ‘hopelessness’ and the qualification of ‘worse than physical pain’. While some measures focused on only a few domains, such as the UP3 and PPP-VAS, which capture 3 and 4 domains, respectively, others capture substantially more, such as the PPAS (17 domains) and OMMPS (18 domains).

    Our analysis of similarity resulted in 45 individual Jaccard Similarity Index scores (one for each pair of measures). All individual scores were 0.5 (PPP-VAS and MBPPS) or lower, that is, had at best; ‘moderate’ similarity. Indeed, 57.7% of the Jaccard scores were below 0.19, that is, had ‘very weak’ similarity (table 3).

    View this table:
    Table 3

    Jaccard indices for quantifying the similarity of the 10 measures of mental pain

    In four cases, the Jaccard Index was 0, indicating that not a single item overlapped across two measures. Assessing the average similarity of one measure with all the others, we found the highest average similarity coefficient was 0.24, for both PAS and TDPPS. That is, the highest average similarity of mental pain measures with all other measures was ‘weak’.

    Evaluation and comparison of the development and content validity of the measures of mental pain

    Following the COSMIN standards, we used the 43 development and validation studies listed in online supplemental material 6 to rate the quality of the measure’s development. It was rated ‘inadequate’ (PPAS, PAS, PPS, UP3, MPQ, TDPPS, PPP-VAS, OMMPS) or ‘doubtful’ (MBPPS, TMPS), mainly because of the paucity of data about the methods used to generate items (table 4). There was no published report dedicated to the development of any of these scales. Only for the OMMPS and the MBPPS were there a few lines in a validation paper about the measure’s development (qualitative analysis of interviews or notes of patients), but not enough information for a rigorous assessment.

    View this table:
    Table 4

    Assessment and comparison of the development of the 10 measures of mental pain

    In addition, we found no study evaluating the content validity, as defined by COSMIN, that is, no study involved patients and healthcare professionals in evaluating the relevance, comprehensiveness and comprehensibility of the measures. Accordingly, we rated the content validity of all measures as ‘insufficient’ or ‘doubtful’ (online supplemental material 9).

    Because the content validity of all measures was inadequate, we did not evaluate further psychometric properties. The COSMIN filter approach is, after all, based on the principle that a measure can be precise, reliable and consistent, yet fail to capture the intended construct.33

    Conclusions and clinical implications

    Our systematic review identified 10 measures of mental pain developed over the past 20 years, mainly for the purpose of predicting suicide and used mostly in the field of clinical psychology. Some scales, such as the OMMPS, have shorter versions that we did not review independently.37 The 10 measures identified vary substantially in their theoretical frameworks, definitions of mental pain, semantics and content (poor content overlap and weak to very weak similarity). This heterogeneity might well be a limitation in comparing or combining the results of studies using these measures, for example, for systematic reviews or meta-analyses.30

    We found no comprehensive report of the development of any of these 10 measures, nor any comprehensive assessment of their content validity. Therefore, as required by our study protocol that planned assessment according to COSMIN, the design of the development of these measures was rated ‘inadequate‘ or ‘doubtful’, while the content validity was rated as ‘inadequate’. The articles reporting on the scale validation tested these measures instead for several other psychometric properties.

    To our knowledge, this is the first review to comprehensively identify measures of mental pain. It is also the first study to analyse in detail their content and their similarity with both qualitative and quantitative approaches. Finally, it is the first study to evaluate and compare their development and content validity. We hope our work lays the foundations for similar papers about other constructs, to enhance the level of evidence of validity in psychiatry and psychology.

    One limitation of our study may be our choice of search terms. Although we included ‘psychache, psychic pain’, ‘psychological pain’ and ‘mental pain’, we omitted the term ‘emotional pain’, which is at times used in the more biological (and less clinical) literature. The reason for not including the term is our focus on clinical psychology and psychiatry.

    Furthermore, we chose not to evaluate further psychometric or clinimetric properties to comply with the COSMIN guidelines, in accordance with our protocol.33 38 However, these properties, including dimensionality and internal consistency, may provide further information about mental pain to inform scale construction and iteration.38 39

    This study leads to three conclusions. First, based on the COSMIN guidelines, we cannot currently recommend one scale over any other for research or clinical purposes, mainly because of the lack of available data to evaluate the tools’ development and content validity. This development must be reported in detail and their content validity must be investigated. Second, too few measures were developed with the input of persons with lived experience of mental pain. This absence is especially notable given that all the measures are patient-reported outcomes.33 Hence, there is a need to collect more empirical data about the subjective experience of mental pain to (1) evaluate the content validity of these tools, (2) better define the construct and (3) develop relevant, comprehensive and comprehensible items if the content validity proved insufficient.33 Third, learning about constructs such as mental pain requires iterative exchange between theory and measurement.19 40 The existing measures can therefore be understood as important first efforts to gain an understanding of mental pain. These efforts must now inform our conceptualisation of mental pain and thereby pave the way towards improving it.19 41

    In conclusion, because mental pain is an outcome that matters greatly to patients, considerably more attention and efforts are called for to assess it properly. One critical next step is therefore to develop a patient-reported routine outcome measure for mental pain for both research and clinical purposes.

    Data availability statement

    All data relevant to the study are included in the article or uploaded as supplementary information.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Not applicable.

    Acknowledgments

    We thank Professor Blynn Bunney, Dr Steven Mee, Professor Mario Mikulincer, Rui C Campos, Professor Emilie Olié, Dr Jane Tillman, Dr Katie Lewis, Professor Holden and Professor Fava for responding to our questions. We thank Jean Charvet for the data visualisation. We thank Jo Ann Cahn for the professional copyediting.

    References

    1. ↵
      2. Ducasse D ,
      3. Holden RR ,
      4. Boyer L , et al
      . Psychological pain in suicidality: a meta-analysis. J Clin Psychiatry 2018;79.doi:10.4088/JCP.16r10732 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28872267
    2. ↵
      2. Verrocchio MC ,
      3. Carrozzino D ,
      4. Marchetti D , et al
      . Mental pain and suicide: a systematic review of the literature. Front Psychiatry 2016;7:108.doi:10.3389/fpsyt.2016.00108 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27378956
      OpenUrlPubMed
    3. ↵
      2. Soumani A ,
      3. Damigos D ,
      4. Oulis P , et al
      . Mental pain and suicide risk: application of the Greek version of the mental pain and the tolerance of mental pain scale. Psychiatriki 2011;22:330–40.pmid:http://www.ncbi.nlm.nih.gov/pubmed/22271846
      OpenUrlPubMed
    4. ↵
      2. Troister T ,
      3. Davis MP ,
      4. Lowndes A , et al
      . A five-month longitudinal study of psychache and suicide ideation: replication in general and high-risk university students. Suicide Life Threat Behav 2013;43:611–20.doi:10.1111/sltb.12043 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23829705
      OpenUrlCrossRefPubMed
    5. ↵
      2. Orbach I ,
      3. Mikulincer M ,
      4. Gilboa-Schechtman E , et al
      . Mental pain and its relationship to suicidality and life meaning. Suicide Life Threat Behav 2003;33:231–41.doi:10.1521/suli.33.3.231.23213 pmid:http://www.ncbi.nlm.nih.gov/pubmed/14582834
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      2. Chevance A ,
      3. Ravaud P ,
      4. Tomlinson A , et al
      . Identifying outcomes for depression that matter to patients, informal caregivers, and health-care professionals: qualitative content analysis of a large international online survey. Lancet Psychiatry 2020;7:692–702.doi:10.1016/S2215-0366(20)30191-7 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32711710
      OpenUrlPubMed
    7. ↵
      2. Tossani E
      . The concept of mental pain. Psychother Psychosom 2013;82:67–73.doi:10.1159/000343003 pmid:http://www.ncbi.nlm.nih.gov/pubmed/23295405
      OpenUrlPubMed
    8. ↵
      2. Conejero I ,
      3. Olié E ,
      4. Courtet P , et al
      . Suicide in older adults: current perspectives. Clin Interv Aging 2018;13:691–9.doi:10.2147/CIA.S130670 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29719381
      OpenUrlPubMed
    9. ↵
      2. Zanarini MC ,
      3. Frankenburg FR ,
      4. DeLuca CJ , et al
      . The pain of being borderline: dysphoric states specific to borderline personality disorder. Harv Rev Psychiatry 1998;6:201–7.doi:10.3109/10673229809000330 pmid:http://www.ncbi.nlm.nih.gov/pubmed/10370445
      OpenUrlPubMed
    10. ↵
      2. Demirkol ME ,
      3. Tamam L ,
      4. Namli Z , et al
      . Association of Psychache and Alexithymia with suicide in patients with schizophrenia. J Nerv Ment Dis 2019;207:668–74.doi:10.1097/NMD.0000000000001023 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31318739
      OpenUrlPubMed
    11. ↵
      2. Korner A ,
      3. Gerull F ,
      4. Stevenson J , et al
      . Harm avoidance, self-harm, psychic pain, and the borderline personality: life in a "haunted house". Compr Psychiatry 2007;48:303–8.doi:10.1016/j.comppsych.2007.01.001 pmid:http://www.ncbi.nlm.nih.gov/pubmed/17445527
      OpenUrlCrossRefPubMedWeb of Science
    12. ↵
      2. Monson CM ,
      3. Price JL ,
      4. Rodriguez BF , et al
      . Emotional deficits in military-related PTSD: an investigation of content and process disturbances. J Trauma Stress 2004;17:275–9.doi:10.1023/B:JOTS.0000029271.58494.05 pmid:http://www.ncbi.nlm.nih.gov/pubmed/15253100
      OpenUrlPubMed
    13. ↵
      2. Landi G ,
      3. Grossman-Giron A ,
      4. Bitan DT , et al
      . Mental pain, psychological distress, and suicidal ideation during the COVID-19 emergency: the moderating role of tolerance for mental pain. Int J Ment Health Addict 2021:1–12.doi:10.1007/s11469-021-00646-4 pmid:http://www.ncbi.nlm.nih.gov/pubmed/34744529
    14. ↵
      2. Nummenmaa L ,
      3. Tuominen L
      . Opioid system and human emotions. Br J Pharmacol 2018;175:2737–49.doi:10.1111/bph.13812 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28394427
      OpenUrlPubMed
    15. ↵
      2. Rizvi SJ ,
      3. Iskric A ,
      4. Calati R , et al
      . Psychological and physical pain as predictors of suicide risk: evidence from clinical and neuroimaging findings. Curr Opin Psychiatry 2017;30:159–67.doi:10.1097/YCO.0000000000000314 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28067727
      OpenUrlPubMed
    16. ↵
      2. Meerwijk EL ,
      3. Ford JM ,
      4. Weiss SJ
      . Brain regions associated with psychological pain: implications for a neural network and its relationship to physical pain. Brain Imaging Behav 2013;7:1–14.doi:10.1007/s11682-012-9179-y pmid:http://www.ncbi.nlm.nih.gov/pubmed/22660945
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      2. Meerwijk EL ,
      3. Ford JM ,
      4. Weiss SJ
      . Resting-State EEG delta power is associated with psychological pain in adults with a history of depression. Biol Psychol 2015;105:106–14.doi:10.1016/j.biopsycho.2015.01.003 pmid:http://www.ncbi.nlm.nih.gov/pubmed/25600291
      OpenUrlPubMed
    18. ↵
      2. Fava GA ,
      3. Tomba E ,
      4. Brakemeier E-L , et al
      . Mental pain as a Transdiagnostic patient-reported outcome measure. Psychother Psychosom 2019;88:341–9.doi:10.1159/000504024 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31665739
      OpenUrlPubMed
    19. ↵
      2. Fried EI ,
      3. Flake JK ,
      4. Robinaugh DJ
      . Revisiting the theoretical and methodological foundations of depression measurement. Nat Rev Psychol 2022;1:358–68.doi:10.1038/s44159-022-00050-2
      OpenUrl
    20. ↵
      1. American Psychiatric Association
      . Diagnostic and statistical manual of mental disorders: DSM-5. 5th Revised edition. Washington, D.C: American Psychiatric Association Publishing, 2013.
    21. ↵
      2. Masson M ,
      3. Muirheid-Delacroix B
      . La douleur morale : historique et devenir d’un concept clinique. Ann Méd-Psychol Rev Psychiatr 2014;172:139–45.doi:10.1016/j.amp.2014.01.006
      OpenUrl
    22. ↵
      2. Kendler KS
      . The genealogy of major depression: symptoms and signs of melancholia from 1880 to 1900. Mol Psychiatry 2017;22:1539–53.doi:10.1038/mp.2017.148 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28785109
      OpenUrlPubMed
    23. ↵
      2. Jansson A
      . From melancholia to depression: disordered mood in nineteenth-century psychiatry. Cham: Springer International Publishing, 2021.
    24. ↵
      2. Shneidman ES
      . The suicidal mind. Oxford University Press, 1998.
    25. ↵
      2. Shneidman ES
      . Suicide as Psychache: a clinical approach to Self-destructive behavior. Jason Aronson 1993.
    26. ↵
      2. Meerwijk EL ,
      3. Weiss SJ
      . Toward a unifying definition: response to 'The concept of mental pain'. Psychother Psychosom 2014;83:62–3.doi:10.1159/000348869 pmid:http://www.ncbi.nlm.nih.gov/pubmed/24281693
      OpenUrlPubMed
    27. ↵
      2. Lewis KC ,
      3. Good EW ,
      4. Tillman JG , et al
      . Assessment of psychological pain in clinical and non-clinical samples: a preliminary investigation using the psychic pain scale. Arch Suicide Res 2021;25:552–69.doi:10.1080/13811118.2020.1729914 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32089105
      OpenUrlPubMed
    28. ↵
      2. Fried EI
      . The 52 symptoms of major depression: lack of content overlap among seven common depression scales. J Affect Disord 2017;208:191–7.doi:10.1016/j.jad.2016.10.019 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27792962
      OpenUrlCrossRefPubMed
    29. ↵
      2. Chalmers I ,
      3. Bracken MB ,
      4. Djulbegovic B , et al
      . How to increase value and reduce waste when research priorities are set. Lancet 2014;383:156–65.doi:10.1016/S0140-6736(13)62229-1 pmid:http://www.ncbi.nlm.nih.gov/pubmed/24411644
      OpenUrlCrossRefPubMedWeb of Science
    30. ↵
      2. Williamson P ,
      3. Altman D ,
      4. Blazeby J , et al
      . Driving up the quality and relevance of research through the use of agreed core outcomes. J Health Serv Res Policy 2012;17:1–2.doi:10.1258/jhsrp.2011.011131 pmid:http://www.ncbi.nlm.nih.gov/pubmed/22294719
      OpenUrlCrossRefPubMed
    31. ↵
      2. Flake JK ,
      3. Fried EI
      . Measurement Schmeasurement: questionable measurement practices and how to avoid them. Adv Methods Pract Psychol Sci 2020;3:456–65.doi:10.1177/2515245920952393
      OpenUrl
    32. ↵
      2. Prinsen CAC ,
      3. Mokkink LB ,
      4. Bouter LM , et al
      . COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res 2018;27:1147–57.doi:10.1007/s11136-018-1798-3 pmid:http://www.ncbi.nlm.nih.gov/pubmed/29435801
      OpenUrlCrossRefPubMed
    33. ↵
      2. Terwee C ,
      3. Prinsen C ,
      4. de Vet H
      . COSMIN methodology for assessing the content validity of patient-reported outcome measures (PROMs). In: User manual, 2018. Available: www.cosmin.nl
    34. ↵
      Chapter 4: searching for and selecting studies. Available: https://training.cochrane.org/handbook/current/chapter-04 [Accessed 15 May 2022].
    35. ↵
      2. Visontay R ,
      3. Sunderland M ,
      4. Grisham J , et al
      . Content overlap between youth OCD scales: heterogeneity among symptoms probed and implications. J Obsessive Compuls Relat Disord 2019;21:6–12.doi:10.1016/j.jocrd.2018.10.005
      OpenUrl
    36. ↵
      2. Chrobak AA ,
      3. Siwek M ,
      4. Dudek D , et al
      . Content overlap analysis of 64 (hypo)mania symptoms among seven common rating scales. Int J Methods Psychiatr Res 2018;27:e1737.doi:10.1002/mpr.1737 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30058102
      OpenUrlPubMed
    37. ↵
      2. Casanova MP ,
      3. Nelson MC ,
      4. Pickering MA , et al
      . Measuring psychological pain: psychometric analysis of the Orbach and Mikulincer mental pain scale. Meas Instrum Soc Sci 2021;3:7.doi:10.1186/s42409-021-00025-8
      OpenUrl
    38. ↵
      2. Carrozzino D ,
      3. Patierno C ,
      4. Guidi J , et al
      . Clinimetric criteria for patient-reported outcome measures. Psychother Psychosom 2021;90:222–32.doi:10.1159/000516599 pmid:http://www.ncbi.nlm.nih.gov/pubmed/34038901
      OpenUrlPubMed
    39. ↵
      1. American Psychological Association
      . The standards for educational and psychological testing. Available: https://www.apa.org. https://www.apa.org/science/programs/testing/standards [Accessed 15 May 2022].
    40. ↵
      2. Chang H
      . Inventing temperature: measurement and scientific progress. New York: Oxford University Press, 2004.
    41. ↵
      2. Rolffs JL ,
      3. Rogge RD ,
      4. Wilson KG
      . Disentangling components of flexibility via the Hexaflex model: development and validation of the multidimensional psychological flexibility inventory (MPFI). Assessment 2018;25:458–82.doi:10.1177/1073191116645905 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27152011
      OpenUrlPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @ChevanceAstrid

    • Contributors AC conceptualised the project, designed the methods, performed the analysis, critically reviewed and edited the draft and supervised the project. CC extracted the data, performed the analysis and the data visualisation and wrote the first draft. CLB extracted the data, performed the analysis and reviewed and edited the draft. YM performed the analysis and reviewed and edited the draft. ST performed the analysis and reviewed and edited the draft. RG reviewed and edited the draft. IB designed the project and reviewed and edited the draft. EF designed the methods and data visualisation and reviewed and edited the draft.

    • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

    • Competing interests CC, IB, CLB, YM, EF, ST and AC report no conflict of interest. RG has received compensation as a member of the scientific advisory board of Janssen, Lundbeck, Roche, SOBI, Takeda. He has served as consultant and/or speaker for AstraZeneca, Boehringer-Ingelheim, Pierre Fabre, Lilly, Lundbeck, LVMH, MAPREG, Novartis, Otsuka, Pileje, SANOFI, Servier and received compensation, and he has received research support from Servier. Co-founder and stock shareholder: Regstem.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.