https://orcid.org/0000-0001-6101-5925
Figures
Abstract
The death literacy index (DLI) was developed in Australia to measure death literacy, a set of experience-based knowledge needed to understand and act on end-of-life (EOL) care options but has not yet been validated outside its original context. The aim of this study was to develop a culturally adapted Swedish-language version of the DLI, the DLI-S, and assess sources of evidence for its validity in a Swedish context. The study involved a multi-step process of translation and cultural adaptation and two validation phases: examining first content and response process validity through expert review (n = 10) and cognitive interviews (n = 10); and second, internal structure validity of DLI-S data collected from an online cross-sectional survey (n = 503). The psychometric evaluation involved analysis of descriptive statistics on item and scale-level, internal consistency and test-retest reliability, and confirmatory factor analysis. During translation and adaptation, changes were made to adjust items to the Swedish context. Additional adjustments were made following findings from the expert review and cognitive interviews. The content validity index exceeded recommended thresholds (S-CVIAve = 0.926). The psychometric evaluation provided support for DLI-S’ validity. The hypothesized six-factor model showed good fit (χ2 = 1107.631 p<0.001, CFI = 0.993, TLI = 0.993, RMSEA = 0.064, SRMR = 0.054). High internal consistency reliability was demonstrated for the overall scale (Cronbach’s α = 0.94) and each sub-scale (α 0.81–0.92). Test-retest reliability was acceptable, ICC ranging between 0.66–0.85. Through a comprehensive assessment of several sources of evidence, we show that the DLI-S demonstrates satisfactory validity and acceptability to measure death literacy in the Swedish context. There are, however, indications that the sub-scales measuring community capacity perform worse in comparison to other sca and may function differently in Sweden than in the original context. The DLI-S has potential to contribute to research on community-based EOL interventions.
Citation: Johansson T, Olsson Å, Tishelman C, Noonan K, Leonard R, Eriksson LE, et al. (2023) Validation of a culturally adapted Swedish-language version of the Death Literacy Index. PLoS ONE 18(11): e0295141. https://doi.org/10.1371/journal.pone.0295141
Editor: Vilfredo De Pascalis, La Sapienza University of Rome, ITALY
Received: January 18, 2022; Accepted: November 14, 2023; Published: November 30, 2023
Copyright: © 2023 Johansson et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data contain potentially sensitive personal information and cannot be shared publicly because of limitations in the ethical approval. We are unable to share data sets due to GDPR restrictions in Sweden and the EU. Data are available upon reasonable request from prefekt@nvs.ki.se.
Funding: “This study was funded by Strategic Research Area Health Care Science (SFO‐V), Karolinska Institutet; the Doctoral School in Health Care Sciences, Karolinska Institutet; Forskningsrådet om Hälsa, Arbetsliv och Välfärd (Swedish Research Council for Health, Welfare and Working Life, grant number 2014-04071); Stockholm City Elder Care Bureau; and Stockholm Gerontology Research Center. The funders had no part in, nor influence on, the study design, data collection, analysis, interpretation, or writing of results.”.
Competing interests: The authors have declared that no competing interests exist.
Introduction and aim
Just as health literacy relates to the extent to which people can find, interpret, and use health information and services [1], death literacy (DL) is a newly-coined term denoting knowledge and skills needed to understand and act on options for end-of-life (EOL) and death care [2]. The concept of DL was developed after extensive qualitative research involving carers and networks involved in caring for a person dying at home. Building on new public health approaches that highlight individual and community capacity-building and empowerment to handle issues of dying, death, and loss [3], DL is theorized to develop from engaging with EOL care and learning from those experiences. It is defined as “a set of knowledge and skills that make it possible to gain access to, understand, and act upon end-of-life and death care options” [4].
The death literacy index (DLI), an operationalization of DL, is a questionnaire developed from research in Australia about people’s experiences of EOL care [5]. The DLI evolved through an iterative mixed-methods approach to content development, refinement, and testing, and is intended to measure group levels of DL and evaluate EOL-related educational initiatives [6]. As such, the DLI may constitute a promising tool for appraising impact of health promoting activities in relation to EOL issues. To date, the DLI has primarily been used among adults in the Australian general public, although the questionnaire has garnered increasing interest internationally, and been validated in the United Kingdom [7], Turkey [8], and China [9]. However, the extent to which DL and its operationalization are comparable across cultures is unknown. The aim of this study was to develop and culturally adapt Swedish-language version of the DLI (DLI-S) and assess its validity in a Swedish context.
Materials and methods
Study design
This validation study [10] used a multi-step approach combining qualitative and quantitative methods [11] to translate and culturally adapt the Death Literacy Index (DLI) into Swedish and to assess the validity of the resulting DLI-S [12]. The multi-step process comprised: forward translation and adaptation by initial translators; translation revision through negotiated consensus in the research team; review by a panel of experts; pre-testing using cognitive interviews; literacy reviews by an external consultant; and psychometric evaluation using online testing of the DLI-S with a survey panel (see Fig 1).
Steps in the translation and adaptation of the DLI-S. Sources of validity evidence.
The Death Literacy Index
The DLI is a multi-dimensional self-report questionnaire containing 29 items distributed over four dimensions of DL; Practical knowing (n of items = 8), Learning from Experience (n = 5), Factual knowledge (n = 7), and Community capacity (n = 9) [2]. The dimensions are represented in the DLI as four scales, two of which contain sub-scales capturing specific dimensional facets; Practical knowing has the sub-scales Talking support (n = 4) and Hands-on care (n = 4), whereas Community Capacity has the sub-scales Accessing help (n = 5) and Community support groups (n = 4). DLI items are in the form of statements with ordered category responses on a 5-point Likert-type scale, usually ranging from “do not agree at all” and “strongly agree”. Since the DLI covers various aspects of DL, scores are calculated for each scale and sub-scale, using transformed mean scores. Total DL is calculated as a higher-order composite score, which is said to reflect overall capacity gained from previous experiences [6]. The proposed six-factor model in the original English-language DLI demonstrated satisfactory psychometric properties in Australia indicating good model fit overall (CFI = 0.955, TLI = 0.95, RMSEA = 0.049), and high reliability of the scales and sub-scales (Cronbach’s α ranging 0.82 to 0.95) [2], and the United Kingdom (CFI = 0.94, RMSEA = 0.07, SRMR = 0.07; Cronbach’s α for sub-scales ranging 0.76 to 0.93) [7]. Moreover, translated DLI versions have demonstrated acceptable reliability and validity in Turkey (total CVI = 0.91; CFI = 0.93, RMSEA = 0.053; Cronbach’s α ranging 0.68–0.90) [8] and China (CFI = 0.956, RMSEA = 0.055, SRMR = 0.051, Cronbach’s α for sub-scales ranging 0.76–0.97) [9].
Translation and adaptation of the Swedish Death Literacy Index
After obtaining permission to translate the DLI from the original developers, co-authors RL and KN, the DLI-S development process was guided by recommendations for translation and cross-cultural adaptation [13, 14]. Back translation was not used in this study as it has been criticized for missing variation in linguistic meaning and cultural nuances [11, 15, 16]. The 29 items with corresponding instructions and response categories were independently translated from English to Swedish by co-authors TJ and ÅO, both native Swedish speakers proficient in English. The two initial forward translations were compared to identify discrepancies and discuss ambiguous wordings, resulting in a joint draft. This DLI-S draft was sent to the members of a multidisciplinary research group with both native English and Swedish speakers, who individually reviewed the draft, prior to continued revision through a process of negotiated consensus in a series of meetings [11] after which changes to content were decided so that the Swedish items would convey the same conceptual understanding as the original DLI [14, 17]. The original DLI and items in the DLI-S are presented in S1 and S2 Appendices respectively.
Validation
Validity is defined as a unitary concept derived from several sources of evidence that each contribute to the overall validity of a measure [18]. Validation involves the assessment of evidence to support interpretations of scores in relation to the intended use of a measure [19]. Validity is context-specific, and thus, validation is strongly recommended whenever a measure is to be used in a new, qualitatively different, population or context [20]. In this article we focus on three sources of validity evidence: 1) Evidence based on content, relating to the adequacy and relevance of items to represent and score the construct measured [21]; 2) Evidence based on response processes, which involves exploration of respondents’ actions and cognitive processes to identify possible sources of error, e.g., challenges with interpreting and answering items [18]; and 3) Evidence based on internal structure, which concerns how items relate to each other and to the overarching construct, often measured as reliability across items, time, or respondents [21]. These correspond with identified quality criteria for assessing psychometric properties of measures [12].
Data collection
Validity evidence based on content and response process.
Expert panel review. An expert review was conducted to evaluate the validity of DLI-S items to measure DL [22]. Since DLI targets the general population, there is no delimited area of expertise relevant for this step. We therefore made efforts to recruit ten panel members with varying ages, backgrounds, and personal and professional perspectives in relation to the EOL, e.g., palliative care, gerontology, ethnology, professional translation, clinical nursing, and from patient interest organizations. Email invitations were sent with information about the study purpose and methods. Inclusion criteria were proficiency in both Swedish and English. Each panel member reviewed the DLI-S independently, using an online survey, accessible only after providing informed consent. Following recommendations by Grant and Davis [22], panel members were first provided with a summary description of the conceptual DL model and provided with information about the intended use of the DLI. As shown in S3 Appendix, the review comprised two main sections: Translation review, in which the semantic and cultural equivalence between each Swedish item and the original corresponding English-language item was assessed on a four-point scale; and Content validity assessment, in which the DLI-S items’ relevance and clarity of content were rated on a four-point scale. Panel members could comment and suggest changes for every item throughout both sections of the survey.
Pre-testing using cognitive interviews. To determine whether the DLI-S items were acceptable, comprehensible, and able to generate information as intended, authors TJ and ÅO conducted cognitive interviews with a new sample from the target population [23], i.e., adults from the general public. Interviews combined think-aloud technique, in which the respondent describes their reasoning out loud as they read and respond to each item [24], and verbal probing, whereby the interviewer asks questions to clarify and further explore any issues [25]. Participants were recruited through convenience sampling in the researchers’ networks. No explicit inclusion or exclusion criteria were used but we strove for variation of age, gender, educational level, and professional background. Due to the Covid-19 pandemic, interviews were conducted online using Zoom. All participants received written information about the study in advance and consented orally to participate in audio-recorded interviews. While we assessed the risk of harm to participants as low, the interviewers were experienced in handling emotional reactions and could refer participants to other sources for support if needed.
Literacy review. To investigate linguistic accessibility, the lexical profile of the DLI-S was reviewed by an independent consultant on two occasions: after the first 5 cognitive interviews and again after the DLI-S had been finalized (Fig 1).
Validity evidence based on internal structure.
Online testing with survey panel. Data was collected from September–November 2021 through an online survey administered by an external data collection agency with a pre-existing panel of ca 100,000 Swedish residents, aged 15 or older, willing to partake in surveys on various topics for both market and scientific research.
Since this study aimed for theoretical generalization rather than making statistical inferences regarding population estimates of death literacy, a representative probability sample was not necessary [26]. Still, we strove for a sample reflective of the heterogeneity within the Swedish population. The inclusion criteria were being eighteen years old or over and residing in Sweden. No explicit exclusion criteria were used and no EOL experience was required to participate. Survey invitations were sent to a quota sample (n = 2991) in the agency’s existing panel, stratified by gender, age, and geographical region. The minimum sample size was set to 500 to have sufficient data and power for confirmatory factor analysis (CFA) [27]. The survey comprised the DLI-S items and questions about sociodemographic variables and EOL experiences and was only accessible to panel members who agreed to participate after being informed about the study. Participants were notified that they could exit the survey at any time and were provided with the researchers’ contact information in case they needed support or had questions or comments following participation. At the end of the survey, participants were asked whether they were willing to complete the survey a second time, to assess test-retest reliability. Of the 412 that agreed to participate in a follow-up, 82 were re-invited to the survey. The retest sample size was set to minimum 50 to provide sufficient for calculating intra-class correlation coefficients (ICC) [28]. The time interval was set to approximately 4 weeks, chosen to allow enough time to avoid rehearsal effects but short enough to minimize the risk of participants experiencing real change that might alter their responses [29].
Data analysis
Expert panel review.
Comments related to the translation and content of each item were reviewed and summarized by TJ. Quantitative ratings were collated in a matrix in Microsoft Excel to calculate the content validity index (CVI), i.e., inter-rater agreement at the item-level (I-CVI) and scale-level (S-CVI). I-CVI represents the proportion of panel members rating an item positively (e.g., 3 or 4) and is recommended to be at least 0.78 [30]. S-CVI was calculated using average proportion, recommended for panels of ≥8 [31].
Pre-testing using cognitive interviews.
Data were based on the interviewers’ field notes, using audio-recordings as back-up if needed, and compiled into a summary matrix, linking comments to the items and sub-scales to which they referred [32]. The matrix was reviewed in recurring consensus meetings to inform decisions regarding item retention, revision, or deletion and modifications to instructions or response categories.
Literacy review.
The literacy review was conducted using the software AntWordProfiler to examine the proportion of words among the 5,000 most common in Swedish.
Online testing with survey panel.
Statistical analyses were conducted using IBM SPSS Statistics 28 and the lavaan package [33] in R (version 4.1.1). Descriptive statistics were used to explore socio-demographic characteristics of the sample and analyze central tendencies and dispersion on item- and scale-level. Response variation was examined at item-level by investigating if all five response categories were used and at scale-level by identifying whether there were floor or ceiling effects, i.e., ≥15% of responses in the maximum and/or minimum category [34]. Inter-item correlations were calculated for all items.
Corrected item-total correlations were examined and values between 0.2–0.7 were considered good discrimination within a scale [35]. Internal consistency reliability was assessed by calculating Cronbach’s α coefficients and confidence intervals of items, sub-scales and the full DLI. Values ≥0.8 were considered as demonstrating good reliability [36]. However, since the appropriateness of Cronbach’s α as a sole measure of reliability has been questioned, due to the effects of the number of scale items and assumptions of unidimensionality and tau equivalence [37, 38], we also calculated average inter-item correlation (AIC), a more robust indicator of internal consistency [39]. The acceptable range for AIC is considered to be 0.15–0.5 [36]. ICC was calculated to assess scale test-retest reliability, using the two-way mixed effects, single measure model (ICC type 3,1). ICC >0.9 indicates excellent reliability, whereas ICC >0.75 is considered good, >0.5 is moderate and <0.5 suggests poor reliability [40].
Since a hypothesized model had been identified and tested during the development of the original DLI and the current study sought to validate this, a CFA was considered more appropriate to assess the fitness of the DLI factor structure than an exploratory factor analysis [41, 42]. Three factor models were tested: with DLI treated as one universal factor; as four factors (corresponding to the original main DL dimensions); and as six factors (corresponding to four sub-scales (1.1, 1.2, 4.1, 4.2) and two dimension-scales (2 and 3). As the 5-point response range used in the DLI was relatively short, data were modelled as ordinal rather than continuous. Diagonally weighted least squares (DWLS) were used as model estimator, since this has been shown to perform better than robust maximum likelihood estimation with ordinal data [41]. Tucker-Lewis index (TLI) and Comparative Fit Index (CFI) >0.95, root mean square error of approximation (RMSEA) >0.06, Standardized Root Mean Square Residual (SRMR) >.08, and χ2 p value >.05 were used as indices of good model fit [43].
Ethical considerations of the study
The study process described below was approved by the Swedish Ethics Review Authority (reference number 2021–00915) and conducted according to the ethical principles of the Helsinki Declaration [44]. All study participants received written information about the nature of the study, including its subject, purpose, and procedure, as well as their right to withdraw. Informed consent to participate was obtained from all participants. The DLI asks about experiences related to the EOL, a potentially sensitive topic. However, previous research has shown that questions addressing dying or death may have an effect on immediate mood but unlikely to cause harm [45]. Furthermore, it should be noted that participants were not persons known to be at the EOL themselves.
Results
Translation and adaptation of the DLI-S
Minor changes were made to item content to ensure conceptual equivalence to the original items and better adapt items to the Swedish context. Detailed examples of changes made at various stages of the item revision process during both translation/adaptation and validation phases are provided in S1 Table, using two DLI-S items as examples.
Validity based on content and response process
Expert panel review.
All ten individuals contacted agreed to participate, of whom seven were women. The qualitative data, e.g., potential concerns identified and suggestions for improvement, were addressed in discussions about item revision during consensus meetings with the research group, thereby informing the continued process of revising the DLI-S. Overall, the expert review identified words and terms that were awkward or unclear in Swedish and raised questions regarding content relevance in Sweden. To address these issues, minor changes to content were made for several items (see S1 Table). I-CVI scores ranged between 0.837–0.987, indicating that each item was considered relevant for the DL construct. The full DLI-S demonstrated good content validity with an S-CVIAve = 0.926. All CVI scores are presented in S2 Table.
Cognitive interviews.
In total, ten people (seven women) participated in the cognitive interviews. Overall, the cognitive interviews showed that some items could raise memories of past experiences. Nevertheless, DLI-S content was generally acceptable to participants, i.e., not distressing, interesting, and with items of varying difficulty. Table 1 presents six types of issues requiring minor changes to the DLI-S and affected items and/or scales, based on participants’ response processes, comments, and suggestions for improvement (see S1 Table for more detailed examples of item revision). For example, we found that participants’ responses to the question in sub-scale Talking Support (1.1), “how difficult or easy you would find the following [items]”, indicated that the question did not sufficiently prompt participants to think about their self-perceived competence or preparedness for engaging in conversations about EOL issues when answering, as was intended. Instead, they often mentioned a combination of values, perceived social taboos, or relation to the conversation partner when thinking aloud about their responses to the items. Consequently, the question was reformulated to “how prepared would you be to talk about the following [conversations about EOL issues]?”, to better guide respondents to consider their readiness to engage in conversations about the EOL when answering items in this sub-scale.
Literacy review.
The first review found that 84.7% of the words used in the DLI-S were among the 5,000 most common in Swedish. The consultant suggested more common or easy-to-read alternatives to difficult or uncommon terms, which formed the basis for another round of item revision through consensus meetings in the research group. This revised DLI-S version was then used in the five subsequent cognitive interviews. A second literacy review was conducted on the final DLI-S, in which the rate had increased to 93.3%, which was deemed sufficient.
Validity based on internal structure
In total, 503 people completed the survey, giving a response rate of 17%. At the retest, 55 participants completed the survey, giving a second response rate of 67%. Socio-demographic characteristics of participants are presented in Table 2. Since the online survey used a mandatory response procedure, requiring all items to be answered to proceed, there were no missing values in the data. Item-level descriptives are presented in Table 3. Inter-item correlations within sub-scales were generally high, e.g. >0.5, and presented in S3 Table.
Scale descriptives.
Scale descriptives, i.e., mean scores, standard deviations, and floor and/or ceiling effects, are presented in Table 4. Total DLI scores and sub-scale scores were normally distributed in the sample, except for the sub-scale Accessing help, which demonstrated a floor effect, i.e., a negatively skewed distribution.
Reliability.
The DLI-S demonstrated high internal consistency reliability, Cronbach’s α = 0.94 for the overall scale and between 0.81–0.92 for the sub-scales. Test-retest reliability was moderate to good, with scale-level ICC ranging 0.66–0.84. Reliability estimates for the full DLI-S and each sub-scale are presented in Table 4.
Confirmatory factor analysis.
Fit indicators for all tested models are presented in Table 5. The one-factor model was tested first, demonstrating adequate fit. The four-factor model, corresponding to the main dimensions of DL, generated a better fit, though still not meeting all recommended thresholds. The six-factor model showed good fit, performing best of the three models tested. Factor loadings in this model were generally high, ranging between 0.57–0.95, and correlations between factors were moderate to strong, between 0.40–0.86 (Fig 2).
Arrow Factor loadings. Line Correlation coefficients between factors.
Discussion
In this mixed-methods study we used an iterative multi-step process of translation, adaptation, and validation that generated both qualitative and quantitative data for assessing several sources of validity evidence for the DLI-S with regard to culture, language, and care organization and provision. The results show evidence of cross-cultural validity of the DLI and confirm fit of the proposed six-factor model of DL. This supports other recent validation studies [7–9] as well as the potential of the DLI to be used to measure DL internationally.
Both I-CVI and S-CVI ratings exceed the recommended minimum set out by Polit et al. [30], suggesting support for DLI-S’ validity in terms of item clarity and relevance for the DL construct, which aligns with Turkish results [8]. Despite high CVI ratings, several potential issues were raised by experts regarding item meaning and suitability in a Swedish context, highlighting the need for qualitative data to make meaningful assessments of content validity for translated items. Likewise, qualitative findings from the cognitive interviews were instrumental for guiding the researchers in addressing problematic items and unclear instructions. The cognitive interviews showed that the DLI-S could be completed by people with varying EOL experiences, from those with who describe themselves as having very limited EOL experiences to experts in the field. Importantly, the cognitive interviews also demonstrated that the DLI-S was not perceived as too sensitive or distressing, although it was noted that certain questions could bring up emotional memories. Several participants mentioned that they thought the items were interesting and thought-provoking, suggesting that completing the DLI-S could constitute a positive self-reflective experience. This is important since death education often emphasizes the role of reflection and sharing of experiences as part of the learning process [46]. In addition, the high proportion of survey participants who were willing to complete the survey a second time further illustrate DLI-S’ acceptability. This is a significant finding since death is often described as a taboo topic in Sweden [47]. Still, as Arthur et al. [48] state, cognitive interviews about measures of a sensitive nature can be challenging: it may be difficult to raise concerns about intrusive or insensitive questions in a face-to-face situation, where a participant might feel obliged to justify their opinion.
One challenge regarding content validity concerns the definition of “community”used in item 21 (Accessing help). There is no Swedish word for “community”, which could be translated with an emphasis on either social, geographical, or cultural connotations. To guide the translation process, the Swedish research team discussed the intention of the term with the original DLI developers. Based on this discussion, we used a translation that highlight location, i.e., neighborhood. Even if this is a common interpretation of the term and no major problems were identified during the cognitive interviews, it is possible that this translation was too narrow and influenced how the question functioned in the Swedish setting, as it is more specific than the English term.
The DLI-S was found to have satisfactory psychometric properties, with support for validity evidence based on internal structure, which corresponds to previous findings in validations of other language-versions of the DLI [2, 7–9]. However, our findings also identify some potential issues with the DLI-S that are important to consider. High Cronbach’s α for all scales and sub-scales indicate that items are inter-related but does not necessarily mean that the scale is unidimensional [36]. Scale-level AICs further confirm strong item inter-relatedness, with values exceeding the recommended range. This finding raises questions of item redundancy, i.e., presence of items that do not sufficiently contribute with new information to measure the construct. The inter-item correlations suggest that the DLI-S might benefit from having one or several items removed: in particular the sub-scales comprising the dimensional scale Community capacity (Accessing help and Community support groups). These sub-scales consistently performed worse in comparison to the other scales and the DLI-S overall. For example, the floor effect in Accessing help indicates that the sub-scale has limited ability differentiating between responses at low levels, which might reduce reliability [12]. This finding also points to differences between the Swedish and Australian context that appear as variation in item difficulty for these items [49]. The distinctiveness of the Swedish context is further emphasized as the Community capacity scale functions well in China and Turkey [8, 9]. In addition, the confirmatory factor analysis showed that two factors (Existing knowledge and Accessing help) were highly correlated [41], suggesting that items in these scales may measure one, underlying, factor rather than two distinct dimensions of DL. Further studies are thus warranted to explore the performance of a shorter DLI-S version in the Swedish context and to investigate if a five-factor model constitutes a better fit for the DLI-S and the extent to which this might be relevant in other contexts. The Turkish and Chinese DLI validations similarly suggest that correlations between these two are the highest of the DLI scales, though not exceeding the cut-off off 0.85. This could indicate that knowledge about formal and informal systems and processes for care provision and support are not easily distinguished.
In sum, the validity evidence for internal structure show that the DLI-S performs well psychometrically, although the comparatively worse performance of the Community capacity sub-scales may indicate difference in function or meaning in Sweden compared to Australia, Turkey, China, and the United Kingdom. This finding could be an accurate reflection of differences in context, particularly in how care systems are organized and people’s expectations of and interactions with them. In Sweden, public awareness of palliative and EOL care has been found to be generally low [46]. Unlike many other countries, Swedish palliative and EOL care is not dependent on public involvement such as volunteerism and charitable donations [50, 51]. Instead, Sweden has a long history of tax-funded universal welfare and high levels of trust in health care providers and institutions [52], which has remained stable even during the Covid-19 pandemic [53]. Indeed, participants in the cognitive interviews who gave low ratings for items in these sub-scales described feeling confident in their belief that if needed, they could turn to their primary care clinic for support or contact the national hub for information about health and healthcare services in Sweden that is accessible round-the-clock by phone or chat.
Strengths and limitations
There are several methodological limitations that should be acknowledged. Participants in the online survey comprise a non-probability quota sample that was recruited from an existing national panel. Although the composition is balanced to that of the Swedish population in terms of age, gender, and place of residence; the educational level of our sample is slightly positively skewed compared to the population average [54] and underrepresented concerning place of birth, as 19.7% of the Swedish population are born outside Sweden, compared to only 7.2% of our participants [55]. More importantly, non-probability sampling raises concerns of possible self-selection and disproportion in unmeasured characteristics that may produce biased results, particularly if the purpose is making population estimates and representativeness [56, 57]. However, as theoretical rather than statistical generalization was the aim of this validation study, a representative and random sample was not required. Similarly, our 17% response rate may be considered low, but a high response rate is not necessary for the purpose of validation. Still, additional studies using larger and representative samples may allow further examination of the generalizability of the DLI-S’ validity in the Swedish population. In addition, it was not possible to examine convergent validity in this study since Swedish translations of other validated measures of comparable constructs are lacking.
Despite these limitations, the study complied with recommended practice for translation, cultural adaptation, and validation, applying a rigorous process to assess validity and reliability from numerous sources of evidence [12, 18, 19]. In comparison to earlier DLI validations, we investigate the factor structure of DL in more detail, adding to the understanding of the fit between theoretical and factor model. It should be highlighted that since the aim of the study was to assess the validity of the DLI-S and not to develop a modified DLI version, no items were removed even if there were some indications of items that could be challenging in terms of comprehensibility (in phase 1) or may be redundant for measuring a DL dimension (in phase 2). Using bilingual field researchers instead of a professional translator during forward translation can be considered a strength, as proposed by Nolte et al. [58] who point out that professional translators often focus on the accuracy of the linguistic translation rather than general readability and conceptual meaning. We also made efforts to address previously identified issues of transparency in validation, e.g., providing full instructions for the expert panel review (S3 Appendix) to increase clarity regarding the basis of ratings [22]. An additional strength in this study is the use of literacy reviews, which is imperative for identifying possible unfair and unintended advantages or disadvantages to certain groups in the target population that might otherwise affect a measure’s usefulness [59].
Implications
Rather than measuring knowledge and skills alone, DL seems to represent a more overarching familiarity with the dying process, as recently suggested by Hayes et al. [60], that also encompasses attitudes and self-efficacy. This perspective seems fitting in the Swedish context, as our findings suggest that alongside gauging the extent of knowledge gained from prior EOL experiences, the DLI appears to capture perceived capacity to handle EOL-related issues and confidence in abilities to learn. The demonstrated acceptability and satisfactory psychometric properties of the DLI-S suggest that it is suitable to measure DL in Sweden. Nevertheless, more research is needed to better understand the DL construct, particularly across cultures, and to determine whether the DLI is appropriate to capture individual and community capacity for EOL in a comparable manner. Furthermore, the suitability of the DLI as an evaluation tool for EOL-related educational initiatives, both within and outside formal care settings, needs to be examined in Sweden and elsewhere.
Lack of public awareness of EOL care and civic preparedness for engaging with issues related to death and dying has been identified as hindering people’s access to high-quality care [61]. Internationally, there is a growth of community based EOL interventions, such as compassionate communities, which are intended to encourage people to assist and support those at the EOL within their community. With further validation, the DLI has potential to be a multifaceted measure appropriate for continued cross-cultural research and better understanding of impact of such initiatives [62]. This is increasingly pertinent as it is expected that EOL care provision will progressively take place outside formal care settings, e.g., aging populations, both internationally [63, 64] and in Sweden [65]. Additional research can also shed light on whether the DLI may be useful in the care context to measure overarching competence for EOL care among staff, especially to evaluate more integrated and comprehensive EOL education interventions [66, 67].
Conclusion
This study provides empirical evidence supporting the validity of the Swedish translation and adaptation of the 29-item DLI to measure death literacy in the adult general public. The DLI-S was shown to be acceptable and feasible to answer regardless of the extent of respondents’ prior EOL experiences. Our results support previous findings, indicating that the theoretical model operationalized in the DLI is stable across countries despite differences in language, culture, and organisation of institutions that all might influence local or regional death systems. In a time with growing interest in building community preparedness for EOL issues, the DLI-S constitutes a promising measure with good properties to capture overall capacity to engage with EOL care. Even though the six-factor model of the DLI yielded a good fit, our results show some characteristics that could potentially impact its measurement properties in a Swedish context.
Supporting information
S1 Appendix. Original English-language Death Literacy Index, with scale and sub-scale headings.
https://doi.org/10.1371/journal.pone.0295141.s001
(DOCX)
S2 Appendix. Online survey, comprising the Swedish Death Literacy Index items and sociodemographic questions (in English).
https://doi.org/10.1371/journal.pone.0295141.s002
(DOCX)
S3 Appendix. Instructions for expert panel review.
https://doi.org/10.1371/journal.pone.0295141.s003
(DOCX)
S1 Table. Matrix with detailed examples of the revision process of two DLI-S items throughout instrument adaptation and validation.
Notes: DLI-S items 4 and 19, which were found to be problematic in terms of clarity, relevance, and/or language, are presented in Swedish and English. Item revision is marked in bold, with reasoning presented in English.
https://doi.org/10.1371/journal.pone.0295141.s004
(DOCX)
S2 Table. Ratings of relevance and clarity and calculated overall content validity index (CVI).
Notes: *Positive rating = number of experts rating the item 3 or 4 on a 4-point scale. I-CVI calculated as (n of raters rating 3 or 4/total n of raters). Scale-level CVI was calculated using average proportion of I-CVI values.
https://doi.org/10.1371/journal.pone.0295141.s005
(DOCX)
S3 Table. Inter-item correlation matrix of DLI-S items.
**. Correlation is significant at the 0.01 level (2-tailed). *. Correlation is significant at the 0.05 level (2-tailed).
https://doi.org/10.1371/journal.pone.0295141.s006
(PDF)
Acknowledgments
The authors would like to acknowledge the generous contributions by Aleksandra Sjöström-Bujacz, Leo Kowalski, and Sara Runesdotter to the statistical analyses. We also acknowledge the contribution of Terese Stenfors during the translation of the DLI-S.
References
- 1.
Kindig DA, Panzer AM, Nielsen-Bohlman L. Health literacy: a prescription to end confusion. In: Io Medicine, editor. Washington DC: The National Academies Press; 2004.
- 2. Leonard R, Noonan K, Horsfall D, Kelly M, Rosenberg JP, Grindrod A, et al. Developing a death literacy index. Death Stud. 2021;46(9):2110–22. Epub 2021/06/22. pmid:34152939.
- 3. Sallnow L, Tishelman C, Lindqvist O, Richardson H, Cohen J. Research in public health and end-of-life care–Building on the past and developing the new. Progress in Palliative Care. 2016;24(1):25–30.
- 4. Noonan K, Horsfall D, Leonard R, Rosenberg J. Developing death literacy. Progress in Palliative Care. 2016;24(1):31–5.
- 5. Horsfall D, Noonan K, Leonard R. Bringing our dying home: How caring for someone at end of life builds social capital and develops compassionate communities. Health Sociology Review. 2012;21(4):373–82. WOS:000316259300002.
- 6. Leonard R, Noonan K, Horsfall D, Psychogios H, Kelly M, Rosenberg J, et al. Death Literacy Index: A Report on its Development and Implementation. Sydney: Western Sydney University: 2020.
- 7. Graham-Wisener L, Toner P, Leonard R, Groarke JM. Psychometric validation of the death literacy index and benchmarking of death literacy level in a representative uk population sample. BMC palliative care. 2022;21(1):1–15.
- 8. Semerci V, Sönmez Sari E, Seven A. Validity and Reliability of the Turkish Version of the Death Literacy Index. OMEGA-Journal of Death and Dying. 2022:00302228221144672.
- 9. Che SL, Li X, Zhu M, Ng WI. The Death Literacy Index: translation, cultural adaptation, and validation of the Chinese version. Frontiers in Public Health. 2023;11. pmid:37250081
- 10. Arafat SY. Validation study can be a separate study design. Int J Med Sci Public Health. 2016;5(11):2421–2.
- 11. McGreevy J, Orrevall Y, Belqaid K, Bernhardson BM. Reflections on the process of translation and cultural adaptation of an instrument to investigate taste and smell changes in adults with cancer. Scand J Caring Sci. 2014;28(1):204–11. Epub 2013/02/07. pmid:23383751.
- 12. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34–42. Epub 2006/12/13. pmid:17161752.
- 13. Sousa VD, Rojjanasrirat W. Translation, adaptation and validation of instruments or scales for use in cross-cultural health care research: a clear and user-friendly guideline. Journal of Evaluation in Clinical Practice. 2011;17(2):268–74. WOS:000288217700009. pmid:20874835
- 14. Behr D, Shishido K. The translation of measurement instruments for cross-cultural surveys. The SAGE handbook of survey methodology. 2016;55:269–87.
- 15. Geisinger KF. Cross-cultural normative assessment: translation and adaptation issues influencing the normative interpretation of assessment instruments. Psychological assessment. 1994;6(4):304–12.
- 16. Behr D. Assessing the use of back translation: the shortcomings of back translation as a quality testing method. International Journal of Social Research Methodology. 2016;20(6):573–84.
- 17. Beck CT, Bernal H, Froman RD. Methods to document semantic equivalence of a translated scale. Res Nurs Health. 2003;26(1):64–73. Epub 2003/01/18. pmid:12532368.
- 18.
American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Standards for educational and psychological testing. Washington DC: American Educational Research Association; 2014.
- 19.
Zumbo BD, Chan EK. Setting the stage for validity and validation in social, behavioral, and health sciences: Trends in validation practices. Validity and validation in social, behavioral, and health sciences: Springer; 2014.
- 20. Gjersing L, Caplehorn JR, Clausen T. Cross-cultural adaptation of research instruments: language, setting, time and statistical considerations. BMC Med Res Methodol. 2010;10(1):13. Epub 2010/02/11. pmid:20144247; PubMed Central PMCID: PMC2831007.
- 21. Cook DA, Zendejas B, Hamstra SJ, Hatala R, Brydges R. What counts as validity evidence? Examples and prevalence in a systematic review of simulation-based assessment. Advances in Health Sciences Education. 2014;19(2):233–50. pmid:23636643
- 22. Grant JS, Davis LL. Selection and use of content experts for instrument development. Res Nurs Health. 1997;20(3):269–74. pmid:9179180
- 23. Beatty PC, Willis GB. Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly. 2007;71(2):287–311. WOS:000250350500006.
- 24. Fonteyn ME, Kuipers B, Grobe SJ. A Description of Think Aloud Method and Protocol Analysis. Qualitative Health Research. 2016;3(4):430–41.
- 25. Willis GB, Miller K, Willis GB, Miller K. Cross-Cultural Cognitive Interviewing. Field Methods. 2011;23(4):331–41.
- 26. Pasek J. When will Nonprobability Surveys Mirror Probability Surveys? Considering Types of Inference and Weighting Strategies as Criteria for Correspondence. International Journal of Public Opinion Research. 2016;28(2):269–91.
- 27. Mundfrom DJ, Shaw DG, Ke TL. Minimum Sample Size Recommendations for Conducting Factor Analyses. International Journal of Testing. 2005;5(2):159–68.
- 28. Shoukri MM, Asyali MH, Donner A. Sample size requirements for the design of reliability study: review and new results. Statistical Methods in Medical Research. 2016;13(4):251–71. WOS:000223201100001.
- 29. Polit DF. Getting serious about test-retest reliability: a critique of retest research and some recommendations. Qual Life Res. 2014;23(6):1713–20. Epub 2014/02/08. pmid:24504622.
- 30. Polit DF, Beck CT, Owen SV. Is the CVI an acceptable indicator of content validity? Appraisal and recommendations. Res Nurs Health. 2007;30(4):459–67. Epub 2007/07/27. pmid:17654487.
- 31. Polit DF, Beck CT. The content validity index: are you sure you know what’s being reported? Critique and recommendations. Res Nurs Health. 2006;29(5):489–97. pmid:16977646
- 32. Willis GB. Analysis of the cognitive interview in questionnaire design: Oxford University Press; 2015.
- 33. Rosseel Y. lavaan: AnRPackage for Structural Equation Modeling. J Stat Softw. 2012;48(2):1–36. WOS:000305117100001.
- 34. McHorney CA, Tarlov AR. Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res. 1995;4(4):293–307. Epub 1995/08/01. pmid:7550178.
- 35. Streiner DL, Norman GR, Cairney J. Health measurement scales: a practical guide to their development and use: Oxford University Press, USA; 2015.
- 36. Clark LA, Watson D. Constructing validity: New developments in creating objective measuring instruments. Psychol Assess. 2019;31(12):1412–27. Epub 2019/03/22. pmid:30896212; PubMed Central PMCID: PMC6754793.
- 37. Sijtsma K. On the use, the misuse, and the very limited usefulness of Cronbach’s alpha. Psychometrika. 2009;74(1):107. pmid:20037639
- 38. Dima AL. Scale validation in applied health research: tutorial for a 6-step R-based psychometrics protocol. Health Psychol Behav Med. 2018;6(1):136–61. Epub 2018/05/10. pmid:34040826; PubMed Central PMCID: PMC8133536.
- 39. Cortina JM. What is coefficient alpha? An examination of theory and applications. Journal of applied psychology. 1993;78(1):98.
- 40. Koo TK, Li MY. A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. J Chiropr Med. 2016;15(2):155–63. Epub 2016/06/23. pmid:27330520; PubMed Central PMCID: PMC4913118.
- 41. Brown TA. Confirmatory factor analysis for applied research: Guilford publications; 2015.
- 42. Knekta E, Runyon C, Eddy S. One size doesn’t fit all: Using factor analysis to gather validity evidence when using surveys in your research. CBE—Life Sciences Education. 2019;18(1):rm1. pmid:30821600
- 43. Lt Hu, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal. 1999;6(1):1–55. WOS:000208063500001.
- 44. Council for International Organizations of Medical Sciences. International ethical guidelines for biomedical research involving human subjects. Bulletin of medical ethics. 2002;(182):17.
- 45. Labott SM, Johnson TP, Fendrich M, Feeny NC. Emotional risks to respondents in survey research. J Empir Res Hum Res. 2013;8(4):53–66. Epub 2013/10/31. pmid:24169422; PubMed Central PMCID: PMC3996452.
- 46. McClatchey IS, King S. The Impact of Death Education on Fear of Death and Death Anxiety Among Human Services Students. Omega (Westport). 2015;71(4):343–61. Epub 2015/12/17. pmid:26665963.
- 47. Westerlund C, Tishelman C, Benkel I, Furst CJ, Molander U, Rasmussen BH, et al. Public awareness of palliative care in Sweden. Scand J Public Health. 2018;46(4):478–87. Epub 2018/01/06. pmid:29301481; PubMed Central PMCID: PMC5989249.
- 48. Arthur EK, Menon U, Browning K, Overcash J, Wills CE. Challenges of Cognitive Interviewing in Sensitive Health Topic Research. Nurs Res. 2021;70(5):376–82. Epub 2021/06/27. pmid:34173378.
- 49. Reeve BB, Fayers P. Applying item response theory modeling for evaluating questionnaire item and scale properties. Assessing quality of life in clinical trials: methods of practice. 2005;2:55–73.
- 50. Groeneveld EI, Cassel JB, Bausewein C, Csikos A, Krajnik M, Ryan K, et al. Funding models in palliative care: Lessons from international experience. Palliat Med. 2017;31(4):296–305. Epub 2017/02/06. pmid:28156188; PubMed Central PMCID: PMC5405831.
- 51. Lindqvist O, Tishelman C. Going public: reflections on developing the DoBra research program for health-promoting palliative care in Sweden. Prog Palliat Care. 2016;24(1):19–24. Epub 2016/05/03. pmid:27134417; PubMed Central PMCID: PMC4832378.
- 52. Mohseni M, Lindström M. Social capital, trust in the health-care system and self-rated health: the role of access to health care in a population-based study. Soc Sci Med. 2007;64(7):1373–83. Epub 2007/01/05. pmid:17202025.
- 53. Nielsen JH, Lindvall J. Trust in government in Sweden and Denmark during the COVID-19 epidemic. West Eur Polit. 2021;44(5–6):1180–204. WOS:000651797900001.
- 54. Statistics Sweden. Befolkningens utbildning 2020 [Educational attainment of the population 2020]. 2021.
- 55. Sweden Statistics. Befolkningsstatistik i sammandrag 1960–2020 [A summary of population statistics 1960–2020]. 2021.
- 56. Mercer AW, Kreuter F, Keeter S, Stuart EA. Theory and practice in nonprobability surveys: parallels between causal inference and survey inference. Public Opinion Quarterly. 2017;81(S1):250–71.
- 57. Valliant R. Comparing Alternatives for Estimation from Nonprobability Samples. J Surv Stat Methodol. 2020;8(2):231–63. WOS:000553465900003.
- 58. Nolte S, Osborne RH, Dwinger S, Elsworth GR, Conrad ML, Rose M, et al. German translation, cultural adaptation, and validation of the Health Literacy Questionnaire (HLQ). PLoS One. 2017;12(2):e0172340. Epub 2017/02/25. pmid:28234987; PubMed Central PMCID: PMC5325258.
- 59. Han K, Colarelli SM, Weed NC. Methodological and statistical advances in the consideration of cultural diversity in assessment: A critical review of group classification and measurement invariance testing. Psychol Assess. 2019;31(12):1481–96. Epub 2019/11/26. pmid:31763873.
- 60. Hayes B, Fabri AM, Coperchini M, Parkar R, Austin-Crowe Z. Health and death literacy and cultural diversity: insights from hospital-employed interpreters. BMJ Support Palliat Care. 2020;10(1):e8. Epub 2017/07/06. pmid:28676498.
- 61. Collins A, Brown JEH, Mills J, Philip J. The impact of public health palliative care interventions on health system outcomes: A systematic review. Palliat Med. 2021;35(3):473–85. Epub 2020/12/24. pmid:33353507.
- 62. Sallnow ES. Collective social capital: a study of new public health and end-of-life care. University of Edinburgh 2018.
- 63.
Albers G, Pereira S, Onwuteaka-Philipsen B, Deliens L, Pasman R, Van den Block L. A public health perspective on palliative care for older people: an introduction. In: Van den Block L, Albers G, Pereira S, Onwuteaka-Philipsen B, Pasman R, Deliens L, editors. Palliative care for older people: A public health perspective. Oxford: Oxford University Press; 2015. p. 3–15.
- 64. Bone AE, Gomes B, Etkind SN, Verne J, Murtagh FEM, Evans CJ, et al. What is the impact of population ageing on the future provision of end-of-life care? Population-based projections of place of death. Palliat Med. 2018;32(2):329–36. Epub 2017/10/11. pmid:29017018; PubMed Central PMCID: PMC5788077.
- 65. Swedish Government Official Reports. God och nära vård: En reform för ett hållbart hälso-och sjukvårdssystem [Good and close care: A reform for a sustainable health care system] (SOU 2020:19) Stockholm: Ministry of Health and Social Affairs, 2020.
- 66. Iida K, Ryan A, Hasson F, Payne S, McIlfatrick S. Palliative and end-of-life educational interventions for staff working in long-term care facilities: An integrative review of the literature. Int J Older People Nurs. 2021;16(1):e12347. Epub 2020/09/13. pmid:32918793.
- 67. Johansson T, Tishelman C, Cohen J, Eriksson LE, Goliath I. Continuums of Change in a Competence-Building Initiative Addressing End-of-Life Communication in Swedish Elder Care. Qual Health Res. 2021;31(10):1904–17. Epub 2021/05/14. pmid:33980082; PubMed Central PMCID: PMC8446900.