Dos and Don'ts in Designing School-Based Awareness Programs for Suicide Prevention
Results of a Three-Stage Delphi Survey
Abstract
Abstract.Background: Despite the promising evidence for the effectiveness of school-based awareness programs in decreasing the rates of suicidal thoughts and suicide attempts in young people, no guidelines on the targets and methods of safe and effective awareness programs exist. Aims: This study intends to distill recommendations for school-based suicide awareness and prevention programs from experts. Method: A three-stage Delphi survey was administered to an expert panel between November 2018 and March 2019. A total of 214 items obtained from open-ended questions and the literature were rated in two rounds. Consensus and stability were used as assessment criteria. Results: The panel consisted of 19 participants in the first and 13 in the third stage. Recommended targets included the reduction of suicide attempts, the enhancement of help-seeking and peer support, as well as the promotion of mental health literacy and life skills. Program evaluation, facilitating access to healthcare, and long-term action plans across multiple levels were among the best strategies for the prevention of adverse effects. Limitations: The study is based on opinions of a rather small number of experts. Conclusion: The promotion of help-seeking and peer support as well as facilitating access to mental health-care utilities appear pivotal for the success of school-based awareness programs.
The incidence of suicidal thoughts and suicide attempts increases steeply from the age of 12 (Cha et al., 2018). Although the suicide rate in young people is about half the suicide rate in older adults (Bertolote & Fleischmann, 2002), suicide is nevertheless the third most common cause of death for adolescents aged 15–19 years worldwide (World Health Organization [WHO], 2019). School-based prevention programs are often considered among the key strategies of prevention for adolescents (Surgenor et al., 2016).
School-based suicide prevention programs can be categorized as awareness/education programs, gatekeeper education, and screening programs (Robinson et al., 2013). Awareness programs are universal prevention strategies that aim to raise awareness of suicidality and provide education regarding recognition and help-seeking. Gatekeeper education and screening programs aim at identifying young people who may be suicidal through trained individuals or screening instruments, and referring those in need to appropriate services (Gould et al., 2003; Robinson et al., 2013). Wasserman et al. (2015) demonstrated in a multi-centered, randomized controlled trial (RCT) that the Youth Aware of Mental Health program led to reduced suicide thoughts and attempts after 12 months compared with a minimal intervention. A gatekeeper education program for school personnel and a screening program had no such effect. Similarly, the authors of the program Signs of Suicide reported a reduction of suicide attempts in several RCTs (Katz et al., 2013). In a recent systematic review of reviews, Platt and Niederkrotenthaler (2020) reported that awareness programs seem to be the type of school-based prevention with the highest level of evidence in suicide prevention. However, further high-quality evidence is required to prove the effectivity and usefulness of these programs. This is especially important in light of the substantial amount of evidence currently supporting the use of other universal prevention strategies, such as restrictions on common methods of suicide.
Several studies have reported unexpected negative effects of awareness programs, such as higher rates of negative attitudes toward help-seeking (Kuiper et al., 2019). Research on awareness programs has been further hampered by the lack of consensus regarding the safest and most effective methods, the need for large sample sizes and follow-up periods, as well as ethical concerns (Kutcher et al., 2017; Nordentoft, 2011). Evidence-based guidelines could therefore help to inform the quality and safety of future programs. Surgenor et al. (2016) developed 10 general recommendations for school-based suicide prevention programs based on a scoping review. In this study, we used the expert Delphi survey method to formulate recommendations focusing on school-based awareness programs for suicide prevention, more specifically targeting: (1) which contents should be included and what would constitute favorable outcomes, (2) what precautions could prevent adverse effects, and (3) what formats and techniques would be particularly useful?
Method
A Delphi survey is a multi-stage survey incorporating multiple rounds of questionnaires administered consecutively to a panel of experts (Jones & Hunter, 1995; Jorm, 2015). After each round, participants are provided with individualized feedback containing the overall response of the panel. Experts are then asked to re-evaluate their initial responses. The Delphi process is repeated until a pre-arranged criterion is achieved. The most common criteria are consensus among participants or stable responses between subsequent rounds (von der Gracht, 2012). The Delphi survey in this report was conducted from November 2018 to March 2019 (Round 1: November 20 to December 6, 2018; Round 2: December 21, 2018 to January 9, 2019, Round 3: February 14 to March 3, 2019).
Formation of the Delphi Panel
We recruited international researchers in the fields of suicidality and suicide prevention. We aimed to reach a sample size of seven or more, as this number has been suggested to be the minimum required for stable group decisions (Häder, 2014). Researchers were considered to be experts if they contributed significantly to research on suicidality or had specific knowledge in awareness programs for suicide prevention in schools, which means:
- •They were the first authors of two or more research papers on suicidality or suicide prevention AND
- •Had a Web of Science H-index > 10 OR
- •Led at least one research project on a school-based awareness program.
To identify participants, one of the authors (K.He) conducted a literature search on GoogleScholar, Hogrefe, eLibrary, and PsychInfo with the search terms: [“suicide*” OR “suicide prevention”] AND [“youth” OR “adolescent*” OR “school”] AND [“awareness” OR “curriculum” OR “program”]. The same author screened the results for relevant articles and generated an initial list of 76 researchers active in suicide research. Overall, 63 researchers met the inclusion criteria and were invited via e-mail to participate in the survey. Asking the candidates to suggest further experts in the field did not produce any further names.
Qualitative Survey (Round 1)
In the first round of the survey, the expert panel received open-ended questions generated by the research team (Table 1). Participants were encouraged to record up to five answers per question. The panel members completed the questionnaire via the online platform SoSci Survey (Leiner, 2016).
Quantitative Survey (Round 2)
Two of the authors (L.G., K.He) reviewed the responses to Round 1. Similar answers were aggregated and duplicates were removed from the data. Disagreements were resolved through team discussions (L.G., K.He, J.H., S.Kn). Through use of this method, the research team created a quantitative questionnaire. To enhance its exhaustiveness, the authors supplemented the questionnaire with items derived from the literature (see Figure 1). We opted for items referring to widely accepted theories of suicidality (as described by Teismann & Dorrmann, 2014), as well the methods of successful programs evaluated with randomized controlled trials (RCTs; Aseltine & DeMartino, 2004; Wasserman et al., 2012). The questionnaire was administered online via SoSci Survey (Leiner, 2016). All 63 members of the original panel were invited to participate. Participants were asked to comment on the correctness and completeness of the questionnaire and to rate the items with regard to different criteria adapted from the ROAMER Expert Survey (Roadmap for Mental Health Research in Europe; Elfeddali et al., 2014) and the indications of Häder (2014) about the conception of expert Delphi surveys (see Table 1).
Statistical Analysis and Re-Rating (Round 3)
The research team analyzed survey responses (medians, interquartile range [IQR], kurtosis) using SPSS (IBM Corporation, version 25). Consensus was defined as an IQR of 1 or below (see von der Gracht, 2012). An IQR of ≤ 1 indicates that 50% of the experts’ responses are within an interval of one point of the Likert scale, whereby the median indicates which point of the scale contains the majority of answers. Participants received the questionnaire in PDF form via e-mail. Each participant received personalized feedback for each of the items where consensus was not reached, and was asked to re-rate those items on the same criteria and scales used in Round 2.
After completion of Round 3, IQRs and medians were computed again to determine consensus, while the Wilcoxon matched-pairs signed-ranks test was used to determine stability. This is a non-parametric equivalent of the paired t test and is used to test for a difference in the mean tendency (median) of paired observations (here: pairs of responses from Rounds 2 and 3; Trevelyan & Robinson, 2015). To avoid unnecessary repetitions, the iterative process was terminated when items reached either consensus or stability (cf. von der Gracht, 2012). Items for which the participants reached consensus and had a median higher than the middle value of the scale were used for content analysis (L.G., S.Kn) to delineate recommendations for school-based awareness programs.
Results
Panel Description
Of the 63 international experts approached, 36 did not respond to the invitation, and four declined participation due to lack of time or because they did not feel they had enough expertise. A total of 23 participants took part in at least one round of the survey: 19 of 63 (30.2%) participated in the first, 16 of 63 (84.2% of Round 1 participants) in the second, and 13 of 63 (81.3% of Round 2 participants) in the third round of the survey. Participants in the first round originated from 12 countries, while participants in the second and third rounds originated from eight (overall: Australia, Austria, Canada, Germany, Iran, Israel, Italy, Hungary, The Netherlands, Norway, Sweden, the United Kingdom, and the United States). Three experts were contacted because of their specific knowledge as leaders of at least one awareness program, and all of them participated in all three rounds. The other participants were authors of influential articles on suicidality and had an H-index > 10 on the Web of Science (range = 14–99).
Delphi Process
Figure 1 summarizes the Delphi process.
Given the small number of items that did not reach the criteria of stability or consensus in Round 3 (six items, 2.8% of total) we decided to terminate the survey, as a further round would not have added substantial findings.
Contents and Favorable Outcomes
In the quantitative survey, the experts were asked to rate the effectiveness of awareness programs when specific types of content were included (30 items). Only one item was rated as very effective for suicide prevention (Mdn = 5, IQR = 1), namely, “Information about where to find help (in the community, at school).” A further 27 items were considered somewhat effective (Table S1 in ESM 1).
We also asked participants to rate the importance, effectivity, and feasibility of outcomes of awareness programs. Three out of 38 outcomes related directly to suicide (reduction of severe suicidal thoughts, fatal and nonfatal suicide attempts); the others were secondary outcomes. The panel expected awareness programs to effect suicide-related outcomes to a small extent (Mdn = 2, IQR = 1). The five items chosen by at least one third of the participants as the most important outcomes included the prevention of suicide attempts and the increase of both help-seeking and peer support. Ten secondary outcomes were identified to be both effective and feasible for suicide prevention in the long term (Mdn ≥ 3, IQR ≤ 1; see Table S2 in ESM 1).
A content analysis of recommended types of content and outcomes resulted in eight central goals for awareness programs: (1) fostering help-seeking (attitudes, behaviors); (2) improving peer support for suicidal youth; (3) improving mental health literacy including knowledge on mental health, mental health disorders and their treatments; (4) education about suicidality (warning signs, real experience reports, false beliefs) and fostering helpful attitudes toward it; (5) informing about issues related to suicidality (e.g., bullying, risk-taking behavior); (6) improving life skills (coping with stress, communication, problem-solving); (7) reducing stigma toward mental health disorders and help-seeking; and (8) reducing risk factors for suicide attempts (hopelessness, social isolation).
Prevention of Adverse Effects
We initially formulated 16 general statements regarding the safety of awareness programs. Although participants argued in favor of school-based awareness programs, they highlighted the need for both adapting these programs to the audience and implementing plans to prevent adverse effects (see Table S3 in ESM 1). In the subsequent rounds, the panel agreed that, “The benefits of suicide prevention programs outweigh the unanticipated consequences” (Mdn = 3, IQR = 1). Of note, two items suggesting that talking about suicide with young people could increase suicidality did not reach consensus, although their median suggested rejection by most of the participants (Mdn = 2, IQR = 2). A further 27 items about precautions were rated as very or moderately important for preventing adverse effects (Mdn ≥ 3, IQR ≤ 1; see Table S4 in ESM 1). Interestingly, to describe suicidality as a symptom of psychopathology (Mdn = 2.5, IQR = 2) and to focus on biological causes of mental health disorders (Mdn = 2, IQR = 2) were rated as being not so important by many of the participants.
Overall, participants suggested to: (1) embed suicide prevention in more general prevention programs; (2) develop, evaluate, and choose programs according to adequate scientific standards; (3) follow guidelines for safe communication about suicide; (4) facilitate access to treatment for participants; (5) prepare long-term action plans to react to suicidality in the schools; (6) train school personnel and parents to recognize and react to suicidality; (7) create an appropriate setting (trained personnel, no punctual intervention); (8) include types of content considered effective for suicide prevention.
Formats and Techniques
The appropriate age range of the target population was estimated to be between 12 (M = 12.2, SD = 2.2) and 17 years (M = 16.5, SD = 6.9), with a group size between 8 (M = 8.4, SD = 5.8) and 20 individuals (M = 19.2, SD = 8.9). Among 17 intervention techniques, participants rated videos, skills trainings, signalizing the presence of gate-keepers, web-based self-management components, and group discussions as the most useful techniques (Mdn ≥ 4, IQR ≤ 1; see Tables S5 and S6 in ESM 1). The program should preferably be led by school psychologists, psychologists, or trained teachers (mean ranks: 1.9, 2.9, and 4; school social workers and school guidance counselors were not included in response formats; see Table S7 in ESM 1).
Discussion
Using a three-stage expert Delphi survey, we developed recommendations on content types, outcomes, formats, and techniques of awareness programs, as well as precautions to prevent negative effects. These findings aim to inform the development, refinement, and dissemination of school-based awareness programs for youth suicide prevention.
In this study, participants rated the prevention of suicide attempts and increasing help-seeking behavior as pivotal targets for prevention programs. This is in line with the idea that improving access to treatment for mental health disorders could be one of the most powerful suicide prevention strategies (Hegerl & Heinz, 2018). Mental health literacy, favorable attitudes toward help-seeking, and communication skills are antecedents of help-seeking behaviors (Rickwood et al., 2005). Hence, the inclusion of this kind of content could be an important step to lower the barriers for access to mental health care. However, further research is needed to determine how awareness programs can promote actual help-seeking behavior (Platt & Niederkrotenthaler, 2020).
The expert panel also suggested that awareness programs should educate young people to react helpfully when peers shows signs of suicidality. This seems particularly important since teenagers often disclose suicidal thoughts only to their friends (Rickwood et al., 2005). The “Teen Mental Health First Aid” program, a program teaching young people to react appropriately to peers with mental health problems, has already been shown to increase the quality of support intentions toward suicidal peers (Hart et al., 2020). Future studies could investigate whether programs focusing on peer support skills can also directly reduce suicide attempts and suicidal thoughts in participating schools. Furthermore, the expert panel recommended to “… embed suicide prevention in more general mental health fostering programs.” Future studies should consider the effects of more general prevention programs (e.g., programs educating about mental health) on suicide attempts and suicidal thoughts.
Overall, most of the survey participants supported the use of school-based awareness programs, stressing that the positive effects outweigh potential negative effects. Interestingly, some of the experts in our study did not support the suggestion to depict suicidality as a symptom of psychopathology, compared with a more normalizing explanation that suicide is the result of extreme stress. This may reflect the assumption that potential stigmas could dissuade young people from talking about their suicidal thoughts. However, Ciffone (2007) did not find any decrease in help-seeking attitudes among those who came to view suicide as a symptom of psychopathology. In fact, Ciffone (2007) asserts that associating suicide with psychopathology could increase the cognitive dissonance in people who were otherwise complacent with their own suicide thoughts, thereby increasing help-seeking. Future studies could investigate which of these two causal explanations is less adverse and more effective.
Nevertheless, the most common recommendation among panel members to minimize any potential adverse effects was to implement evidence-based prevention methodologies that are innocuously effective. The panel also suggested adopting multiple prevention strategies at schools, for example, developing school protocols and educating parents and teachers. Research shows multilevel interventions generally outperforming single interventions (Hofstra et al., 2019). Future studies could test whether this is also true for school-specific interventions.
According to the expert panel, the ideal age for awareness programs should be between 12 and 17 years. Since most prevention programs that have been implemented to date have been targeted at 14- to 16 year-olds, it may be necessary to additionally develop and test programs for younger people.
Limitations
Recommendations from this expert Delphi survey should be interpreted against some caveats. First, these results depend on panel formation and on information upon which participants based their judgment. The panel represents only a small number of experts in the field of suicide prevention. The research team also chose not to provide literature to the panel prior to the survey, mostly because the literature found failed to address the majority of our research questions. Thus, participants’ answers are based on their own knowledge and experience. Second, a systematic literature search or a focus group prior to the survey could have improved its exhaustiveness. Third, for economic reasons, the research team decided to let experts re-rate only those items for which participants did not reach consensus. It is possible that a new rating of all items would have changed the results. Fourth, the inclusion of practitioners and stakeholders with experience in suicide prevention could likely add to our findings.
Conclusion
This Delphi survey is, to our knowledge, the first of its kind to explore content, goals, and methods most suited for effective and safe awareness programs for youth suicide prevention. The improvement of help-seeking and peer support were recommended as key elements of awareness programs. A methodologically sound and long-term evaluation of programs, in combination with long-term interventions in schools that help facilitate access to mental health care for students in need, could further help to prevent the occurrence of negative effects.
Electronic Supplementary Material
The electronic supplementary material is available with the online version of the article at https://doi.org/10.1027/0227-5910/a000783
Luna Grosselli, MSc, is a clinical psychologist. As a scientific associate and doctoral candidate at Technische Universität Dresden she is involved in the evaluation of a school-based program for suicide prevention in the project Network for Suicide Prevention in Dresden.
Kristina Herzog, MSc, is a psychologist. During her master's degree, she worked on the project Network for Suicide Prevention in Dresden. She currently works at the university hospitals in Dresden and Leipzig and is conducting a multicenter study on illness perceptions in children and adolescents with cancer, aimed at improving psychosocial care.
Prof. Robert H. Aseltine Jr., PhD, is chair of the Division of Behavioral Sciences and Community Health and deputy director of the Center for Public Health and Health Policy at UCONN Health. Prof. Aseltine is a medical sociologist with diverse research interests (e.g., suicide prevention, health disparities, the development of public health information systems).
Prof. Judit Balazs, MD, DSc, is a child and adolescent psychiatrist and psychiatrist. Among other positions, she is the chair of the Department of Developmental and Clinical Child Psychology, Institute Psychology Eotvos Lorand University, Budapest, the president of the Hungarian Psychiatric Association, and the chair of the Child Psychiatry Section of the European Psychiatric Association.
Dr. Vladimir Carli, MD, PhD, is associate professor of psychiatry at the National Centre for Suicide Research and Prevention (NASP), at Karolinska Institutet. He is co-director of the WHO Collaborating Centre for Research, Training and Methods Development in Suicide Prevention, and Treasurer of the International Academy for Suicide Research (IASR).
Jerry Ciffone, LCSW, is a licensed clinical social worker. During his 32-year career as a school social worker he was directly involved in the assessment and treatment of over 10,000 troubled teens. Mr. Ciffone also implemented and evaluated his own suicide prevention program, which he personally provided to over 20,000 teens.
Prof. em. Diego De Leo, AO, DSc, MD, PhD, FRANZCP, was the director of the Australian Institute for Suicide Research and Prevention (AISRAP) and is a past president of the International Association for Suicide Prevention and the International Academy for Suicide Research. He has advised suicide prevention projects across the world, including working with the World Health Organization.
Prof. Christina Van der Feltz-Cornelis, MD, PhD, holds the position of Chair of Psychiatry and Epidemiology at the University of York. She is a member of the Mental Health and Addictions Research Group (MHARG) at the University of York's Department of Health Sciences and holds a joint appointment with the Hull York Medical School (HYMS).
Prof. Keith Hawton, DSc, FMedSci, DM, is director of the Centre for Suicide Research at Oxford University Department of Psychiatry and consultant psychiatrist with Oxford Health NHS Foundation Trust. His research group conducts investigations concerning the epidemiology, causes, treatment, prevention, and outcome of suicidal behavior.
Prof. Ulrich Hegerl, MD, PhD, is Distinguished Professor at the Department of Psychiatry, Psychosomatic, and Psychotherapy, Goethe University Frankfurt am Main. He is chair of the German Depression Foundation (Stiftung Depressionshilfe) and the German Alliance Against Depression e. V., as well as president of the European Alliance Against Depression e. V.
Dr. Kairi Kõlves, PhD, is a principal research fellow and course convener at the Australian Institute for Suicide Research and Prevention (AISRAP) and co-director of the WHO Collaborating Centre for Research and Training in Suicide Prevention, Griffith University. She has been working in suicide research and prevention since 1998.
Senator Stan Kutcher, ONS, MD, FRCPC, FCAHS, is professor emeritus of psychiatry at Dalhousie University. He was the Sun Life Financial Chair in Adolescent Mental Health and former director World Health Organization Collaborating Centre in Mental Health Policy and Training at Dalhousie University and IWK Health Centre in Canada.
Prof. Lars Mehlum, MD, PhD, is the founding director of the National Centre for Suicide Research and Prevention, University of Oslo. He is the immediate past president of the International Academy of Suicide Research (IASR) and the European Society for the Study of Personality Disorders and the past president of the International Association for Suicide Prevention (IASP).
Assoc. Prof. Thomas Niederkrotenthaler, MD, PhD, MMSC, is head of the Unit Suicide Research & Mental Health Promotion at the Medical University of Vienna, Austria. He is currently a board member of the International Association for Suicide Prevention, and chair of the Wiener Werkstaette for Suicide Research.
Prof. Mohsen Rezaeian, PhD, is professor of epidemiology at Rafsanjan University of Medical Sciences, Iran. He is one of the founders of the Iranian Scientific Society for Suicide Prevention and editor-in-chief of the Journal of Suicide Prevention. He is co-chair of the IASP SIG on the Development of Effective National Suicide Prevention Strategy and Practice.
Assoc. Prof. Johanne Renaud, MD, MSc, FRCPC, is a principal investigator at the McGill Group for Suicide Studies, Canada. Since 2011, she has been the chair of the Manulife Centre for Breakthroughs in Teen Depression and Suicide Prevention. She won the 2012 Klerman Prize Honorable Mention (Brain & Behavior Research Foundation) for her academic advancement as a clinician scientist.
Prof. Barbara Schneider is head physician of the Clinic for Addiction-Related Diseases at the LVR-Klinik Köln. She is the chair of the National Suicide Prevention Program for Germany (NaSPRO) and deputy chair of the Unit of Suicidology of the German Society of Psychiatry, Somatic Medicine and Neurology (dgppn).
Assoc. Prof. Ute Lewitzka, MD, is a psychiatrist at the Department of Psychiatry and Psychotherapy of the University Clinic Carl Gustav Carus, Dresden. She is the chair of the German Association for Suicide Prevention and chair of the Unit of Suicidology of the German Society of Psychiatry, Somatic Medicine and Neurology (dgppn).
Prof. Jürgen Hoyer is professor for behavioral psychotherapy at the Technische Universität Dresden (TUD). He is also head of the TUD outpatient clinic for psychotherapy and of the associated postgraduate training center. In research, Prof. Hoyer focusses on mechanisms and new settings in the psychotherapy of anxiety disorders and depression.
Prof. Susanne Knappe is an associate professor of mental health of children and adolescents at the Technische Universität Dresden (TUD). She was also the head of the Outpatient Center for Children and Adolescents of the TUD and co-director of the Center for Early Recognition and Preventive Intervention Studies (CEPRIS).
We would like to acknowledge the survey participants and Anthony Jorm, Marco Sarchiapone, Heidrun Bründel, and Gil Zalsman for their valuable input and critical comments. We thank Melissa Claire Daugelat for language editing.
References
(2004). An outcome evaluation of the SOS suicide prevention program. American Journal of Public Health, 94(3), 446–451. 10.2105/AJPH.94.3.446
(2002). A global perspective in the epidemiology of suicide. Suicidologi, 7(2), 6–8. 10.5617/suicidologi.2330
(2018). Suicide among youth: Epidemiology, (potential) etiology, and treatment. Journal of Child Psychology and Psychiatry, 59(4), 460–482. 10.1111/jcpp.12831
(2007). Suicide prevention: An analysis and replication of a curriculum-based high school program. Social Work, 52(1), 41–49. 10.1093/sw/52.1.41
(2014). Horizon 2020 priorities in clinical mental health research: Results of a consensus-based ROAMER expert survey. International Journal of Environmental Research and Public Health, 11(10), 915–939. 10.3390/ijerph111010915
(2003). Youth suicide risk and preventive interventions: A review of the past 10 years. Journal of the American Academy of Child & Adolescent Psychiatry, 42(4), 386–405. 10.1097/01.CHI.0000046821.95464.CF
(2014). Delphi-Befragungen: Ein Arbeitsbuch [Delphi surveys: A workbook] (3rd ed.). Springer VS.
(2020). Teen mental health first aid as a school-based intervention for improving peer support of adolescents at risk of suicide: Outcomes from a cluster randomised crossover trial. Australian & New Zealand Journal of Psychiatry, 54(4), 382–392. 10.1177/0004867419885450
(2018). Reflections on causes of suicidal behaviour. Epidemiology and Psychiatric Sciences, 28(5), 469–472. 10.1017/S2045796018000562
(2019). Effectiveness of suicide prevention interventions: A systematic review and meta-analysis. General Hospital Psychiatry, 63, 127–140. 10.1016/j.genhosppsych.2019.04.011
(1995). Qualitative research: Consensus methods for medical and health services research. BMJ, 311(7001), 376–380. 10.1136/bmj.311.7001.376
(2015). Using the Delphi expert consensus method in mental health research. Australian & New Zealand Journal of Psychiatry, 49(10), 887–897. 10.1177/0004867415600891
(2013). A systematic review of school-based suicide prevention programs. Depression and Anxiety, 30(10), 1030–1045. 10.1002/da.22114
(2019). Examining the unanticipated adverse consequences of youth suicide prevention strategies: A literature review with recommendations for prevention programs. Suicide and Life-Threatening Behavior, 49(4), 952–965. 10.1111/sltb.12492
(2017). School- and community-based youth suicide prevention interventions: Hot idea, hot air, or sham? The Canadian Journal of Psychiatry, 62(6), 381–387. 10.1177/0706743716659245
(2016). SoSci Survey (Version 2.6.00). [Computer software]. https://www.soscisurvey.de
(2011). Crucial elements in suicide prevention strategies. Progress in Neuro-Psychopharmacology and Biological Psychiatry, 35(4), 848–853. 10.1016/j.pnpbp.2010.11.038
(2020). Suicide prevention programs. Crisis, 41(Suppl 1), 99–124. 10.1027/0227-5910/a000671
(2005). Young people’s help-seeking for mental health problems. Australian e-Journal for the Advancement of Mental health, 4(3), 218–251. 10.5172/jamh.4.3.218
(2013). A systematic review of school-based interventions aimed at preventing, treating, and responding to suicide-related behavior in young people. Crisis, 34(3), 164–182. 10.1027/0227-5910/a000168
(2016). Ten recommendations for effective school-based, adolescent, suicide prevention programs. School Mental Health, 8(4), 413–424. 10.1007/s12310-016-9189-9
(2014). Suizidalität [Suicidality]. Hogrefe Verlag.
(2015). Delphi methodology in health research: How to do it? European Journal of Integrative Medicine, 7(4), 423–428. 10.1016/j.eujim.2015.07.002
(2012). Consensus measurement in Delphi studies: Review and implications for future quality assurance. Technological Forecasting and Social Change, 79(8), 1525–1536. 10.1016/j.techfore.2012.04.013
(2012). Suicide prevention for youth – A mental health awareness program: Lessons learned from the Saving and Empowering Young Lives in Europe (SEYLE) intervention study. BMC Public Health, 12(1), 776. https://10.1186/1471-2458-12-776
(2015). School-based suicide prevention programmes: The SEYLE cluster-randomised, controlled trial. The Lancet, 385(9977), 1536–1544. 10.1016/S0140-6736(14)61213-7
(2019). Fact sheet: Suicide. http://www.who.int/mediacentre/factsheets/fs398/en/
