You are here

Article Text

Article menu

Download PDFPDF

Statistics in practice
Vitruvian plot: a visualisation tool for multiple outcomes in network meta-analysis
  1. Edoardo Giuseppe Ostinelli1,2,3,
  2. Orestis Efthimiou1,4,5,
  3. Huseyin Naci6,
  4. Toshi A Furukawa7,
  5. Stefan Leucht8,
  6. Georgia Salanti5,
  7. Laurence Wainwright2,
  8. Caroline Zangani1,2,3,
  9. Franco De Crescenzo1,2,
  10. Katharine Smith1,2,3,
  11. Katherine Stevens2,
  12. Qiang Liu1,2,
  13. Andrea Cipriani1,2,3
  1. 1 Department of Psychiatry, University of Oxford, Oxford, UK
  2. 2 Oxford Precision Psychiatry Lab, NIHR Oxford Health Biomedical Research Centre, Oxford, UK
  3. 3 Oxford Health NHS Foundation Trust, Warneford Hospital, Oxford, UK
  4. 4 Institute of Primary Health Care (BIHAM), University of Bern, Bern, Switzerland
  5. 5 Institute of Social and Preventive Medicine, University of Bern, Bern, Switzerland
  6. 6 Department of Health Policy, London School of Economics and Political Science, London, UK
  7. 7 Department of Health Promotion and Human Behavior, School of Public Health, Kyoto, Japan
  8. 8 Department of Psychiatry and Psychotherapy, Technical University of Munich, Munchen, Germany
  1. Correspondence to Professor Andrea Cipriani, Department of Psychiatry, University of Oxford, Oxford, UK; andrea.cipriani{at}psych.ox.ac.uk

Abstract

Objective A network meta-analysis (NMA) usually assesses multiple outcomes across several treatment comparisons. The Vitruvian plot aims to facilitate communication of multiple outcomes from NMAs to patients and clinicians.

Methods We developed this tool following the recommendations on the communication of benefit–risk information from the available literature. We collected and implemented feedback from researchers, statisticians, methodologists, clinicians and people with lived experience of physical and mental health issues.

Results We present the Vitruvian plot, which graphically presents absolute estimates and relative performance of competing interventions against a common comparator for several outcomes of interest. We use two alternative colour schemes to highlight either the strength of statistical evidence or the confidence in the evidence. Confidence in the evidence is evaluated across six domains (within-study bias, reporting bias, indirectness, imprecision, heterogeneity and incoherence) using the Confidence in Network Meta-Analysis (CINeMA) system.

Conclusions The Vitruvian plot allows reporting of multiple outcomes from NMAs, with colourings appropriate to inform credibility of the presented evidence.

  • PSYCHIATRY
http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/ebmental-2022-300457

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction

    The impact of scientific findings on real-world clinical practice is determined not only by their intrinsic value but also by how effectively they can be communicated to clinicians and patients.1 This has been the case for more than 150 years. In 1854, nurse and statistician Florence Nightingale arrived in Turkey with a group of 37 nurses to attend the wounded of the Crimean War. While there, she recorded more casualties from epidemic diseases, malnutrition, poor sanitation and other modifiable factors than from battlefield wounds.e1 Her initial attempt to communicate how a large proportion of these deaths was avoidable used complex statistics. Despite her efforts, her findings were viewed with scepticism and disregarded.e1 Realising that the main pitfall was the format used to communicate her findings, she developed the Diagram of the Causes of Mortality in the Army of the East, now known also as Nightingale’s rose diagram or coxcomb.2 ,e2 The success of this communication strategy promoted the implementation of preventative public health measures and ultimately contributed to the reduction of death rates in hospitals.e1

    Available literature on behavioural decision research and communication on benefit–risk information indicates that the format in which findings are presented may significantly affect the comprehension and behaviour of end users.3–7 ,e3, e4 Well-designed patient decision aids—either visualisation tools or conversation aids—have been shown to materially improve patients’ understanding of the effects of outcomes and risks compared with control interventions.3, e4 First, bars and pictographs have been identified as the best strategies to visually communicate complex statistics and numerical results from multiple comparative alternatives.6 , e3 Second, a common timeframe and a consistent reference class (denominator) and format should be employed when comparing the chance of occurrence of two or more independent events, favouring absolute risks, either percentages or frequencies, over relative ones.6 Third, 6textual and graphical communication should acknowledge the average numeracy and literacy skills of the target audience.5 6 Fourth, the implications of communicating the inherent uncertainty in the results should also be considered.6 7 Finally, available information should be accompanied by narrative statements to inform both gain and loss frames (ie, people experiencing and not experiencing the event of interest)6 7

    Network meta-analysis (NMA) is a useful tool for summarising available evidence about the effects of three or more interventions for the same condition.8 Typically multiple outcomes are analysed separately, aiming to capture the comparative profile among treatment alternatives. The combination of multiple interventions and outcomes further increases the complexity of presenting benefit–risk findings from NMAs. Pooled estimates concerning an outcome are usually presented using forest plots or league tables.e5 These methods, however, are only fit to simultaneously show results regarding a small number of outcomes, limiting the overall interpretation of findings.9 10 Alternative methods to visualise multiple outcomes from NMAs have been proposed,11–14 ,e6-e10 with only limited consistency with good practice guidelines for communicating benefit–risk information.6 15–18 For instance, some tools prioritised the textual information over the graphical communication, with the latter limited to the use of numbers in coloured tables.12 ,e6-e8 A visual representation would allow patients and clinicians to quickly appraise and compare the available interventions across the considered outcome. On the other hand, other tools employed alternative visualisation display (ie, rank heat plot, bubble plot, spie chart, and scatter plot) to communicate risk information from advanced metrics.11 13 ,e9, e10

    The development of an effective and user-friendly communication tool would be a critical step forward in the implementation of an evidence-based approach in real-world shared decision-making processes. We aimed to create a tool to facilitate communication of findings from multiple outcomes in NMA to patients and clinicians. We developed this tool following recommendations from the available literature and collecting feedback from researchers, statisticians, methodologists, clinicians and people with lived experiences of physical and mental health issues. We named it a ‘Vitruvian plot’ after Leonardo da Vinci’s Vitruvian Man, which simultaneously represents multiple human proportions. This article illustrates two alternative visualisation strategies to communicate results from multiple NMAs and the evaluation of confidence in these findings. The data sets, the R code and related instructions to further customise the script are freely available on GitHub (https://github.com/EGOstinelli/Vitruvian-plot). In the online supplemental material, we provide two competing visualisation methods using the same data sets.

    Supplemental material

    Methods

    How to build the plot

    We developed the Vitruvian plot in R as a radial bar plot, which is a bar plot shifted from a cartesian coordinate system (where coordinates are defined as distances from two fixed perpendicular axes) to a polar one (where coordinates are defined as the distance from a polar point and an angle from a polar axis). Each Vitruvian plot corresponds to a single intervention and shows the NMA results for multiple outcomes.

    For binary outcomes, the impact of each intervention is shown on an absolute event rate scale, which is easier to understand for patients and clinicians.6 In our examples, we used the ORs from the outcome-specific NMAs and a fixed event rate for placebo (control event rate, CER) to estimate the event rate of each intervention19:

    Embedded Image

    For continuous outcomes, we can follow a similar procedure if the NMA is performed on the natural scale, that is, using mean difference. In this case, we would simply use the mean difference for each intervention versus placebo estimated from the NMA, and a fixed mean outcome for placebo. If, however, the NMA is performed using the standardised mean difference (SMD), these can be converted in ORs as follow, for instance using the Hasselblad and Hedges’ method:20 , e11

    Embedded Image

    When estimating CER from continuous data, the number of events in the population of patients allocated to the common comparator can be imputed using the baseline and endpoint continuous values and after setting a meaningful threshold (eg, 50% improvement from baseline).21 In all cases, the absolute estimate for the common comparator can either be derived from the data used in the NMA or supplied from an external source (ie, ‘real-world’ study).

    In the following analyses, we used a mock data set to analyse five outcomes with corresponding NMAs for demonstration purposes. Figure 1 shows a plot where the overall profile of each active intervention and placebo across five outcomes is depicted as wedges and expressed as absolute estimates: response, remission, all-cause drop-out (acceptability), nausea and headache. Absolute estimates of both the intervention of interest and the common comparator are reported as grey squares and light blue circles, respectively, to contextualise the absolute estimates. The Vitruvian plots can then be merged in a synoptic chart to simultaneously visualise and navigate among all the available interventions.

    Figure 1
    Figure 1

    Anatomy of the Vitruvian plot. The key elements of the Vitruvian plot and related terminology are hereby illustrated.

    We applied the recommendations on the communication of benefit–risk information from the available literature.3–7 ,e3, e4 Each radial bar represents the magnitude of effect for an outcome and also displayed as a percentage, using both textual and graphical communication strategies. Uncertainty over the relative performance against the common comparator is reported as the strength of statistical evidence (figure 2) or as the confidence in the evidence (figure 3). Finally, each visualisation is accompanied by a supportive statement to facilitate the interpretation of the Vitruvian plots.

    Figure 2
    Figure 2

    Synoptic chart showing strength of statistical evidence. The profiles of placebo and treatments 1 and 2 are compared across five domains: clinical response, clinical remission, acceptability (all-cause drop-out), nausea, and headache. A dark green colour indicates there is strong statistical evidence that the corresponding intervention performs better than placebo. A dark red colour indicates strong evidence that the intervention performs worse than placebo. Light green/red colours indicate weaker evidence of a beneficial/detrimental effect. Colours closer to yellow indicate an increasing lack of evidence on whether the intervention performs better or worse than placebo. The light blue colour identifies placebo as the common comparator. When presenting the Vitruvian plots to patients, we strongly suggest including both probabilities of events and non-events as well as the time frame. A suggested reporting statement for the ‘remission’ outcome and treatment 1 is: ‘Of 100 people with your clinical diagnosis, 15 may remit (1 more than with placebo) and 85 may not remit with treatment 2 over 8 weeks’.

    Figure 3
    Figure 3

    Synoptic chart showing confidence in the evidence. The profiles of placebo and treatments 1 and 2 are compared across five domains: clinical response, clinical remission, acceptability (all-cause drop-out), nausea and headache. Darker hues of purple indicate increasing confidence in the evidence for the comparison of the corresponding treatment vs placebo (categorised in four levels, that is, very low, low, moderate, high quality). The light blue colour identifies placebo as the common comparator. When presenting the Vitruvian plots to patients, we strongly suggest including both probabilities of events and non-events as well as the time frame. A suggested reporting statement for the ‘nausea’ outcome and treatment 1 is: ‘Of 100 people with your clinical diagnosis, 14 may experience nausea (2 more than with placebo) and 86 may not experience nausea with treatment 2 over 8 weeks’.

    How to colour the plot

    We have implemented two alternative and fully customisable colour schemes: the strength of statistical evidence, considering uncertainty as done by Seo and colleagues,12 and the confidence in the evidence, using the Confidence in Network Meta-Analysis (CINeMA) ratings.22 The colour scheme can be easily personalised (eg, to a colour blind accessible scheme such as the Viridis or Okabe-Ito palettes or a custom one) by changing the hexadecimal colour values in the code (https://github.com/EGOstinelli/Vitruvian-plot).

    Strength of statistical evidence

    This approach indicates the relative performance of the intervention and the common comparator, taking into account uncertainty.12 Z-scores are calculated from estimated effect sizes and SEs from the NMAs, coherently with the direction of effect. In our example, these values are reported as colours: from green (the intervention is better than the comparator) to yellow (unclear whether the intervention performs better or worse than the comparator) and red (the intervention is worse than the comparator). The common comparator is coloured in light blue. Given that p values are more widely used to communicate findings, these are calculated from Z-scores and reported in the legend of the plot.

    Confidence in the evidence

    Approaches such as Grading of Recommendations, Assessment, Development and Evaluations (GRADE) and CINeMA rate how confident we are in the evidence supporting a certain finding.22 ,e12 Specifically, CINeMA evaluates the confidence in the results from NMAs across six domains: within-study bias, reporting bias, indirectness, imprecision, heterogeneity and incoherence. These domains are then considered jointly to express a summarising judgement (very low, low, moderate, high confidence), an approach widely used to communicate and summarise key findings.23 In our example, darker hues of purple indicate an increasing level of confidence in the evidence. The common comparator is coloured in light blue.

    Involvement of members of the public and clinicians

    We actively involved people with lived experiences of physical or mental illnesses, and clinicians in the conceptualisation of the Vitruvian plot to ensure that the final output was optimised for these users. As the tool was rapidly evolving, we intentionally collected their feedback throughout the development process via ad hoc multiple individual interviews. We evaluated the intelligibility of the Vitruvian plot with guided (ie, explanation provided by a member of the team) and unguided (ie, use of written captions only) interpretation tasks using fictional clinical scenarios. For instance, we asked users to comment and compare the fictional interventions depicted in the Vitruvian plots. This allowed us to highlight a number of issues with the early versions of the tool, and to use their feedback to guide the overall development process. Finally, we used the collected inputs to develop the suggested supporting statements.

    Results

    Visualisation of the strength of statistical evidence

    Figure 2 shows clinical profiles of placebo and treatments 1 and 2 in terms of response, remission, all-cause drop-out (acceptability) and two adverse events, namely, nausea and headache. In this example, treatment 1 shows an overall higher probability of achieving response (35%, compared with 13% for placebo) and remission (15%, compared with 14% for placebo). The probability of dropping out due to any cause is lower compared with placebo (acceptability: 16% and 18%, respectively). Treatment 1 is associated with a higher probability of having headache and nausea (13% and 14%, respectively) compared with placebo (11% and 12%), with strong statistical evidence for response and nausea outcomes. Treatment 2 shows a promising overall pattern, but the probability to achieve response is only 20%. The statistical strength of the evidence is strong for all the outcomes except for nausea.

    Visualisation of the confidence in the evidence

    Figure 3 shows an alternative visualisation strategy for the same interventions and outcomes. In this example, it is evident how the confidence in the evidence for the comparison between treatment 2 and placebo is overall high, with the only exception of remission (low, 16%). On the other hand, ratings of the five outcomes are more heterogeneous for treatment 1, with moderate confidence for remission and nausea (15% and 14%, respectively), low confidence for response and acceptability (35% and 15%, respectively) and very low confidence for headache (13%).

    Discussion

    In this article, we have illustrated a novel way of presenting information for NMAs in a transparent, accessible and useful manner that can inform and support the decision-making process between patients and clinicians. This is instrumental to take full advantage of the available evidence .6 7 For instance, in our first example (figure 2), treatment 1 may be considered a strong candidate due to its response profile. Nonetheless, after considering the confidence in evidence (figure 3), treatment 2 might be a good candidate due to its high confidence ratings.

    One of the strengths of the Vitruvian plot is the active involvement of people with lived experience of physical or mental diseases, and clinicians. This approach, combined with the involvement of multidisciplinary experts and recommendations from available literature, allowed us to overcome some limitations of the existing modalities to present results from NMAs. For instance, although the Vitruvian plot shares visual similarities with the spie chart with equally weighted outcomes (see section 3 of online supplemental file), the latter did not include a measure of confidence in the evidence when presenting the magnitude of effect, something that Daly and colleagues expressed interested in as a possible future development.11 The spie chart implemented ranking metrics, such as the surface under the cumulative ranking curve (SUCRA) values, as metrics of choice,11 similarly to the radar graph developed by Seide and colleagues to visually compare multiple drugs over an outcome.14 The feedback we received from patients and clinicians was instrumental to develop the Vitruvian plot. For instance, we implemented the level of confidence in the evidence and favoured absolute estimates over relative ones or ranking metrics. The use of absolute metrics is in line with currently available recommendations for the comparison of the chance of occurrence of two or more independent events.6 7

    Absolute estimates have been used in the Kilim plot, together with the colour scheme we adopted in our first example (ie, strength of statistical evidence).12 Our approach differs from the Kilim plot in how we presented the metric of choice, combining textual information with corresponding visual elements. The interpretation of the magnitude of effect is conveyed by both textual and graphical formats, an optimal communication strategy when targeting populations with varying levels of numeracy and literacy skills.4–6 Although visual aids are not free from bias, such as an increase or reduction of risk-avoidant behaviour, well-developed visual formats have been proven effective to communicate health risks. For instance, Garcia-Retamero & Cokley found that adding visuals to written messages was more effective than written health information alone in improving accuracy in judgements and promoting changes in health behaviour related to sexually transmitted diseases in a high-risk population.24

    Communication and understanding of uncertainty are ‘arguably essential for informed decision- making’.6 This is in line with the increasing support to avoid dichotomising findings according to the statistical significance (ie, statistically significant or not), especially for NMAs.25 26 Nonetheless, how to effectively implement this in the shared decision-making framework is a challenge yet to be solved. Only a small number of studies have evaluated the potential impact of communicating uncertainty in healthcare, suggesting that it might be associated with pessimistic risk perceptions and avoidance of reaching a decision (ie, ‘ambiguity aversion’).6 27 28 ,e13-e17 Communication of uncertainty may exert an effect on how people understand it, feel about it, and ultimately trust the information.29 A careful assessment of the impact on these domains is essential not only to prevent detrimental effects but also to reinforce and enhance behavioural changes and confidence with the tool. For instance, reporting uncertainty may be perceived as an example of transparent communication in research, thus increasing trustworthiness.30 Findings from a recent study on the impact of communicating uncertainty suggest that people adequately recognised and perceived uncertainty, with only a limited impact on the perceived reliability of numbers.30

    The current version of the Vitruvian plot has some limitations. First, outcomes are not differentiated in terms of weight. Daly and colleagues explored non-equal weighting of outcomes.11 In their example, they used the regression coefficients from a multivariate regression model to tailor areas and ranks for each treatment to sex assigned at birth and age group. While this approach could be implemented in the current version of the Vitruvian plot, we aim at developing an interactive version in which outcomes can be optionally weighted according to the individual preferences of users. Second, the current version of the Vitruvian plot works best with a limited number of concurrent independent outcomes. Implementation of highly correlated outcomes is discouraged, as these will impact the overall profile of the intervention. Moreover, the number of outcomes presented should be carefully considered and balanced to present a comprehensive and informative synthesis of the available evidence in supporting practical decision-making. Third, continuous outcomes require conversion into dichotomous effect sizes when SMDs are used in the analysis. Finally, it is yet unclear whether this and competing methods of visualisation for multiple outcomes NMA provide a real practice benefit. The performance of these visualisation strategies in improving comprehension and better engaging users in decision-making should be carefully assessed.

    As the interplay between patients, clinicians and evidence continues to rapidly evolve, the development of accessible formats to communicate critical information is of increasing importance. We hope that the implementation of the Vitruvian plots in real-world practice will support patients and clinicians in evidence-based shared decision-making. Future steps would be to develop an interactive version of the Vitruvian plot and to properly assess this new communication strategy to explore the relative performance and preferences of different components of the Vitruvian plots, such as format (ie, static vs interactive) and specific approaches (eg, statistical strength vs confidence in the evidence), across different fields of medicine and healthcare.

    References e1–e17 are provided in online supplemental file.

    Ethics statements

    Patient consent for publication

    Ethics approval

    Not applicable.

    Acknowledgments

    We would like to acknowledge Mary Jane Attenburrow, Philip Cowen, Guy Goodwin, Orla MacDonald and Anneka Tomlinson for their interim feedback on the Vitruvian plot.

    References

    1. ↵
      2. Chevance A ,
      3. Ravaud P ,
      4. Tomlinson A , et al
      . Identifying outcomes for depression that matter to patients, informal caregivers, and health-care professionals: qualitative content analysis of a large international online survey. Lancet Psychiatry 2020;7:692–702.doi:10.1016/S2215-0366(20)30191-7 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32711710
      OpenUrlPubMed
    2. ↵
      2. Nightingale F
      . Notes on matters affecting the health, efficiency, and hospital administration of the British army. founded chiefly on the experience of the late war. presented by Request to the Secretary of state for war. London, United Kingdom: Harrison and Sons, 1858.
    3. ↵
      2. Stacey D ,
      3. Légaré F ,
      4. Lewis KB
      . Patient decision AIDS to engage adults in treatment or screening decisions. JAMA 2017;318:657–8.doi:10.1001/jama.2017.10289 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28810006
      OpenUrlPubMed
    4. ↵
      2. Feldman-Stewart D ,
      3. Kocovski N ,
      4. McConnell BA , et al
      . Perception of quantitative information for treatment decisions. Med Decis Making 2000;20:228–38.doi:10.1177/0272989X0002000208 pmid:http://www.ncbi.nlm.nih.gov/pubmed/10772360
      OpenUrlCrossRefPubMedWeb of Science
    5. ↵
      2. McCaffery KJ ,
      3. Dixon A ,
      4. Hayen A , et al
      . The influence of graphic display format on the interpretations of quantitative risk information among adults with lower education and literacy: a randomized experimental study. Med Decis Making 2012;32:532–44.doi:10.1177/0272989X11424926 pmid:http://www.ncbi.nlm.nih.gov/pubmed/22074912
      OpenUrlCrossRefPubMedWeb of Science
    6. ↵
      2. Trevena LJ ,
      3. Zikmund-Fisher BJ ,
      4. Edwards A , et al
      . Presenting quantitative information about decision outcomes: a risk communication primer for patient decision aid developers. BMC Med Inform Decis Mak 2013;13 Suppl 2:S7.doi:10.1186/1472-6947-13-S2-S7 pmid:http://www.ncbi.nlm.nih.gov/pubmed/24625237
      OpenUrlPubMed
    7. ↵
      2. Armstrong KA ,
      3. Metlay JP
      . Annals Clinical Decision Making: Communicating Risk and Engaging Patients in Shared Decision Making. Ann Intern Med 2020;172:688–92.doi:10.7326/M19-3495 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32311739
      OpenUrlPubMed
    8. ↵
      2. Leucht S ,
      3. Chaimani A ,
      4. Cipriani AS , et al
      . Network meta-analyses should be the highest level of evidence in treatment guidelines. Eur Arch Psychiatry Clin Neurosci 2016;266:477–80.doi:10.1007/s00406-016-0715-4 pmid:http://www.ncbi.nlm.nih.gov/pubmed/27435721
      OpenUrlCrossRefPubMed
    9. ↵
      2. Chaimani A ,
      3. Salanti G ,
      4. Leucht S , et al
      . Common pitfalls and mistakes in the set-up, analysis and interpretation of results in network meta-analysis: what clinicians should look for in a published article. Evid Based Ment Health 2017;20:88–94.doi:10.1136/eb-2017-102753 pmid:http://www.ncbi.nlm.nih.gov/pubmed/28739577
      OpenUrlAbstract/FREE Full Text
    10. ↵
      2. Kossmeier M ,
      3. Tran US ,
      4. Voracek M
      . Charting the landscape of graphical displays for meta-analysis and systematic reviews: a comprehensive review, taxonomy, and feature analysis. BMC Med Res Methodol 2020;20:26.doi:10.1186/s12874-020-0911-9 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32028897
      OpenUrlCrossRefPubMed
    11. ↵
      2. Daly CH ,
      3. Mbuagbaw L ,
      4. Thabane L , et al
      . Spie charts for quantifying treatment effectiveness and safety in multiple outcome network meta-analysis: a proof-of-concept study. BMC Med Res Methodol 2020;20:266.doi:10.1186/s12874-020-01128-2 pmid:http://www.ncbi.nlm.nih.gov/pubmed/33115431
      OpenUrlCrossRefPubMed
    12. ↵
      2. Seo M ,
      3. Furukawa TA ,
      4. Veroniki AA , et al
      . The Kilim plot: a tool for visualizing network meta-analysis results for multiple outcomes. Res Synth Methods 2021;12:86–95.doi:10.1002/jrsm.1428 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32524754
      OpenUrlPubMed
    13. ↵
      2. Veroniki AA ,
      3. Straus SE ,
      4. Fyraridis A , et al
      . The rank-heat plot is a novel way to present the results from a network meta-analysis including multiple outcomes. J Clin Epidemiol 2016;76:193–9.doi:10.1016/j.jclinepi.2016.02.016 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26939929
      OpenUrlCrossRefPubMed
    14. ↵
      2. Seide SE ,
      3. Jensen K ,
      4. Kieser M
      . Utilizing radar graphs in the visualization of simulation and estimation results in network meta-analysis. Res Synth Methods 2021;12:96–105.doi:10.1002/jrsm.1412 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32367691
      OpenUrlPubMed
    15. ↵
      2. Hughes D ,
      3. Waddingham E ,
      4. Mt-Isa S , et al
      . Recommendations for Benefit-risk assessment methodologies and visual representations. Pharmacoepidemiol Drug Saf 2016;25:251–62.doi:10.1002/pds.3958 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26800458
      OpenUrlPubMed
    16. ↵
      2. Trevena LJ ,
      3. Bonner C ,
      4. Okan Y , et al
      . Current challenges when using numbers in patient decision AIDS: advanced concepts. Med Decis Making 2021;41:834–47.doi:10.1177/0272989X21996342 pmid:http://www.ncbi.nlm.nih.gov/pubmed/33660535
      OpenUrlPubMed
    17. ↵
      2. Hallgreen CE ,
      3. Mt-Isa S ,
      4. Lieftucht A , et al
      . Literature review of visual representation of the results of benefit-risk assessments of medicinal products. Pharmacoepidemiol Drug Saf 2016;25:238–50.doi:10.1002/pds.3880 pmid:http://www.ncbi.nlm.nih.gov/pubmed/26521865
      OpenUrlPubMed
    18. ↵
      2. Spiegelhalter D
      . Risk and uncertainty communication. Annu Rev Stat Appl 2017;4:31–60.doi:10.1146/annurev-statistics-010814-020148
      OpenUrl
    19. ↵
      2. Leucht S ,
      3. Siafis S ,
      4. Engel RR , et al
      . How efficacious are antipsychotic drugs for schizophrenia? an interpretation based on 13 effect size indices. Schizophr Bull 2022;48:27–36.doi:10.1093/schbul/sbab094 pmid:http://www.ncbi.nlm.nih.gov/pubmed/34405881
      OpenUrlPubMed
    20. ↵
      2. Chinn S
      . A simple method for converting an odds ratio to effect size for use in meta-analysis. Stat Med 2000;19:3127–31.doi:10.1002/1097-0258(20001130)19:22<3127::AID-SIM784>3.0.CO;2-M pmid:http://www.ncbi.nlm.nih.gov/pubmed/11113947
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      2. Furukawa TA ,
      3. Cipriani A ,
      4. Barbui C , et al
      . Imputing response rates from means and standard deviations in meta-analyses. Int Clin Psychopharmacol 2005;20:49–52.doi:10.1097/00004850-200501000-00010 pmid:http://www.ncbi.nlm.nih.gov/pubmed/15602117
      OpenUrlCrossRefPubMedWeb of Science
    22. ↵
      2. Nikolakopoulou A ,
      3. Higgins JPT ,
      4. Papakonstantinou T , et al
      . Cinema: an approach for assessing confidence in the results of a network meta-analysis. PLoS Med 2020;17:e1003082–19.doi:10.1371/journal.pmed.1003082
      OpenUrlCrossRefPubMed
    23. ↵
      2. Schünemann HJ ,
      3. Higgins JPT ,
      4. Vist GE
      . Completing ‘Summary of findings’ tables and grading the certainty of the evidence. In: Higgins JPT , Thomas J , Chandler J , eds. Cochrane Handbook for systematic reviews of interventions version 6.2. Cochrane, 2021.
    24. ↵
      2. Garcia-Retamero R ,
      3. Cokely ET
      . Effective communication of risks to young adults: using message framing and visual AIDS to increase condom use and STD screening. J Exp Psychol Appl 2011;17:270–87.doi:10.1037/a0023677 pmid:http://www.ncbi.nlm.nih.gov/pubmed/21942316
      OpenUrlCrossRefPubMed
    25. ↵
      2. Efthimiou O ,
      3. White IR
      . The dark side of the force: multiplicity issues in network meta-analysis and how to address them. Res Synth Methods 2020;11:105–22.doi:10.1002/jrsm.1377 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31476256
      OpenUrlPubMed
    26. ↵
      2. Amrhein V ,
      3. Greenland S ,
      4. McShane B
      . Scientists rise up against statistical significance. Nature 2019;567:305–7.doi:10.1038/d41586-019-00857-9 pmid:http://www.ncbi.nlm.nih.gov/pubmed/30894741
      OpenUrlCrossRefPubMed
    27. ↵
      2. Han PKJ ,
      3. Klein WMP ,
      4. Lehman T , et al
      . Communication of uncertainty regarding individualized cancer risk estimates: effects and influential factors. Med Decis Making 2011;31:354–66.doi:10.1177/0272989X10371830 pmid:http://www.ncbi.nlm.nih.gov/pubmed/20671211
      OpenUrlCrossRefPubMedWeb of Science
    28. ↵
      2. Muscatello DJ ,
      3. Searles A ,
      4. MacDonald R , et al
      . Communicating population health statistics through graphs: a randomised controlled trial of graph design interventions. BMC Med 2006;4:33.doi:10.1186/1741-7015-4-33 pmid:http://www.ncbi.nlm.nih.gov/pubmed/17178006
      OpenUrlCrossRefPubMed
    29. ↵
      2. van der Bles AM ,
      3. van der Linden S ,
      4. Freeman ALJ , et al
      . Communicating uncertainty about facts, numbers and science. R Soc Open Sci 2019;6:181870.doi:10.1098/rsos.181870 pmid:http://www.ncbi.nlm.nih.gov/pubmed/31218028
      OpenUrlCrossRefPubMed
    30. ↵
      2. van der Bles AM ,
      3. van der Linden S ,
      4. Freeman ALJ , et al
      . The effects of communicating uncertainty on public trust in facts and numbers. Proc Natl Acad Sci U S A 2020;117:7672–83.doi:10.1073/pnas.1913678117 pmid:http://www.ncbi.nlm.nih.gov/pubmed/32205438
      OpenUrlAbstract/FREE Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @EGOstinelli, @Toshi_FRKW, @And_Cipriani

    • Contributors EGO: conceptualisation, methodology, software, formal analysis, visualisation, writing—original draft, writing—review and editing. OE: methodology, formal analysis, writing—review and editing. HN, TAF, SL, GS: methodology, writing—review and editing. LW: writing—review and editing. CZ: investigation, visualisation, writing—review and editing. FDC: investigation, conceptualisation, writing—review and editing. KS, KSt, QL: investigation, writing—review and editing. AC: conceptualisation, methodology, supervision, writing—review and editing.

    • Funding This research was funded by the National Institute for Health Research (NIHR) Research Professorship to Professor AC (grant RP-2017–08-ST2-006). EGO and AC are supported by the National Institute for Health Research (NIHR) Research Professorship (grant RP-2017–08-ST2-006), by the Oxford Health Biomedical Research Centre (BRC-1215-20005), and by the National Institute for Health Research (NIHR) Applied Research Collaboration (ARC) Oxford and Thames Valley (grant NIHR200172). EGO, CZ, KS, and AC are supported by the National Institute for Health Research (NIHR) Oxford Cognitive Health Clinical Research Facility (award CRF-2016–10014). OE was supported by project grant number 180 083 from the Swiss National Science Foundation (SNSF). GS was supported by the Swiss National Science Foundation (grant/award number 179158).

    • Disclaimer The views expressed are those of the authors and not necessarily those of the United Kingdom National Health Service, the NIHR or the United Kingdom Department of Health.

    • Competing interests EGO reports personal fees from Angelini Pharma, outside the submitted work; and is a member of the International Early Career Researchers Advisory Board for the Evidence-Based Mental Health journal. HN reports grant support from The Health Foundation; research funding from the UK National Institute for Health Research; financial support from the Pharmaceutical Group of the European Union; paid advisory role at the BMJ—all outside of the submitted work. TAF is Deputy Editor for the Evidence-Based Mental Health journal. SL is Associate Editor for the Evidence-Based Mental Health journal. GS is Section Editor for the Evidence-Based Mental Health journal. FDC is a DPhil candidate at the University of Oxford, but he is also an employee of Boehringer Ingelheim International GmbH (from February 2022). AC reports personal fees from Italian Network for Paediatric Trials and CARIPLO Foundation; grants and personal fees from Angelini Pharma outside the submitted work; and is Editor for the Evidence-Based Mental Health journal.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.