You are here

Article Text

Article menu

Download PDFPDF

Digital mental health
Original research
Impact of COVID-19 on telepsychiatry at the service and individual patient level across two UK NHS mental health Trusts
  1. James SW Hong1,
  2. Rebecca Sheriff1,2,
  3. Katharine Smith1,2,
  4. Anneka Tomlinson1,2,
  5. Fathi Saad2,
  6. Tanya Smith2,
  7. Tomas Engelthaler3,
  8. Peter Phiri4,
  9. Catherine Henshall2,5,
  10. Roger Ede2,
  11. Mike Denis3,
  12. Pamina Mitter1,2,
  13. Armando D'Agostino6,
  14. Giancarlo Cerveri7,
  15. Simona Tomassi8,
  16. Shanaya Rathod4,
  17. Nick Broughton2,
  18. Karl Marlowe2,
  19. John Geddes1,2,
  20. Andrea Cipriani1,2
  1. 1Department of Psychiatry, University of Oxford, Oxford, UK
  2. 2Oxford Health NHS Foundation Trust, Oxford, Oxfordshire, UK
  3. 3Akrivia Health, Oxford Centre for Innovation, Oxford, UK
  4. 4Southern Health NHS Foundation Trust, Southampton, UK
  5. 5Faculty of Health and Life Sciences, Oxford Brookes University Faculty of Health and Life Sciences, Oxford, UK
  6. 6Department of Health Sciences, University of Milan, Milano, Lombardia, Italy
  7. 7Department of Psychiatry and Addiction, ASST Lodi, Lodi, Lombardia, Italy
  8. 8Psychiatric Unit 1, Azienda ULSS 9 Scaligera, Verona, Veneto, Italy
  1. Correspondence to Professor Andrea Cipriani, University of Oxford Department of Psychiatry, Oxford, UK; andrea.cipriani{at}psych.ox.ac.uk

Abstract

Background The effects of COVID-19 on the shift to remote consultations remain to be properly investigated.

Objective To quantify the extent, nature and clinical impact of the use of telepsychiatry during the COVID-19 pandemic and compare it with the data in the same period of the 2 years before the outbreak.

Methods We used deidentified electronic health records routinely collected from two UK mental health Foundation Trusts (Oxford Health (OHFT) and Southern Health (SHFT)) between January and September in 2018, 2019 and 2020. We considered three outcomes: (1) service activity, (2) in-person versus remote modalities of consultation and (3) clinical outcomes using Health of the Nation Outcome Scales (HoNOS) data. HoNOS data were collected from two cohorts of patients (cohort 1: patients with ≥1 HoNOS assessment each year in 2018, 2019 and 2020; cohort 2: patients with ≥1 HoNOS assessment each year in 2019 and 2020), and analysed in clusters using superclasses (namely, psychotic, non-psychotic and organic), which are used to assess overall healthcare complexity in the National Health Service. All statistical analyses were done in Python.

Findings Mental health service activity in 2020 increased in all scheduled community appointments (by 15.4% and 5.6% in OHFT and SHFT, respectively). Remote consultations registered a 3.5-fold to 6-fold increase from February to June 2020 (from 4685 to a peak of 26 245 appointments in OHFT and from 7117 to 24 987 appointments in SHFT), with post-lockdown monthly averages of 23 030 and 22 977 remote appointments/month in OHFT and SHFT, respectively. Video consultations comprised up to one-third of total telepsychiatric services per month from April to September 2020. For patients with dementia, non-attendance rates at in-person appointments were higher than remote appointments (17.2% vs 3.9%). The overall HoNOS cluster value increased only in the organic superclass (clusters 18–21, n=174; p<0.001) from 2019 to 2020, suggesting a specific impact of the COVID-19 pandemic on this population of patients.

Conclusions and clinical implications The rapid shift to remote service delivery has not reached some groups of patients who may require more tailored management with telepsychiatry.

  • adult psychiatry
  • delirium & cognitive disorders
  • eating disorders
  • depression & mood disorders

Data availability statement

Data may be obtained from a third party and are not publicly available. NHS data may be obtained from a third party and are not publicly available.

https://creativecommons.org/licenses/by/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution 4.0 Unported (CC BY 4.0) license, which permits others to copy, redistribute, remix, transform and build upon this work for any purpose, provided the original work is properly cited, a link to the licence is given, and indication of whether changes were made. See: https://creativecommons.org/licenses/by/4.0/.

https://doi.org/10.1136/ebmental-2021-300287

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Background

    Telepsychiatry is the delivery of psychiatric assessments or follow-up interviews from a distance using technologies such as telephone calls, audio and video digital platforms, and healthcare monitoring devices.1 After years of protracted efforts to implement digital transformation in the National Health Service (NHS),2 the COVID-19 pandemic, and its associated UK-wide lockdown, led to near-overnight adoption of telepsychiatric services in clinical care. This unavoidable transition has undoubtedly ensured continuity of mental healthcare for patients.3 However, the impact of this profound shift, in terms of clinical effectiveness and patient satisfaction, and of the pandemic on patients, clinicians and services remains to be quantified.4

    Rapid reporting of country-specific, high-quality evidence is mandatory to inform relevant stakeholders at a time of unprecedented transformations in healthcare delivery. The wide-ranging impact of telepsychiatry on mental health service use and outcomes of care can be analysed efficiently with the UK’s electronic health records (EHR) infrastructure. The Clinical Record Interactive Search (CRIS) provides an automated platform on which to access secure, deidentified, real-world EHR data and gain clinically meaningful insights into changes at the service and individual patient levels.5 Previous studies have assessed the impact of the pandemic on broad population-level mental health outcomes,6 but few studies have quantified service-level and patient-level changes with a focus on telepsychiatry. Stewart and colleagues used CRIS to quantify pandemic-related changes in adult community mental health and home treatment teams.7 This study was, however, limited to the analysis of data from one Trust only and between 2019 and 2020, making it difficult to differentiate the impact of the COVID-19 pandemic from general trends in the population.

    To overcome these shortcomings and increase clinical validity and generalisability of findings, we conducted a study using EHR data from two UK NHS Trusts over a 3-year evaluation period. Our aim was to assess the impact of the COVID-19 pandemic on mental health services by quantifying the extent, nature and patient-level impact of the shift to telepsychiatry.

    Methods

    In May 2020, we contacted two NHS Trusts across England, Oxford Health NHS Foundation Trust (OHFT) and Southern Health NHS Foundation Trust (SHFT), which agreed to participate in the study. OHFT provides specialist mental health, physical health and social care services, covering a population of 1.9 million people across Oxfordshire, Buckinghamshire, Swindon, Wiltshire, Bath and North East Somerset. SHFT provides specialist mental health services, learning disability services, social care and integrated community healthcare services covering 1.8 million people in Hampshire, excluding Portsmouth City, with a mixture of urban city, suburbia and rural communities. The two Trusts operated different local EHR systems: CareNotes in OHFT and Servelec Open Rio in SHFT.

    Using Structured Query Language (SQL), deidentified mental healthcare data were extracted from local EHR systems by local Data Science Teams (FS, TS, PP, TE) for all patients accessing mental health services at OHFT and SHFT between 1 January and 30 September in 2018, 2019 and 2020 (see online supplemental appendix for data plan, variables and definitions, and services/teams). The same 9-month period across three different years was selected to capture the key moments of the COVID-19 pandemic before and after the first national lockdown on 23 March 2020, and to facilitate high-level serial comparisons across equivalent time frames. The period of analysis was limited to this 9-month period as the focus of the study was on the acute effects of a rapid, system-wide service transformation (ie, 3 months before and 6 months after the national lockdown). The full study protocol is available in the online supplemental appendix. NHS Trust approval was provided by each participating site. Data handling was compliant with NHS Information Governance regulations which include the Data Protection Act. A Patient and Public Involvement representative (RE) was involved in discussions during the planning, analysis and manuscript writing phases of the project.

    HoNOS data

    For the analysis of clinical outcomes, we used the HoNOS data (as they are routinely collected in the NHS) and identified two cohorts of patients at OHFT. SHFT could not contribute to this analysis, as the Trust does not readily store cluster-level HoNOS scores in an anonymised format (see below for a description of clusters). It was therefore not feasible to replicate the HoNOS analysis across both sites. HoNOS is a widely used and clinician-reported mental health outcome instrument comprised of 12 items/subscales, which cover symptom severity, functioning, social and environmental measures.8 The HoNOS score is then combined with additional risk-based scales to group patients into ‘clusters’ using the Mental Health Clustering Tool.9 The 21 clusters are divided into three superclasses (psychosis, non-psychosis and organic), which are a measure of combined symptomatic-functional-social-environmental severity.9 Within each superclass, the cluster value increases along an ordinal scale, reflecting increasing healthcare complexity with correspondingly higher costs for the NHS.10

    Cohort 1 included patients with at least one HoNOS assessment each year during the index period (ie, 2018, 2019 and 2020); by contrast, cohort 2 included patients with at least one HoNOS assessment during the index period only in 2019 and 2020. The HoNOS scores were linked to the data from clinical visits. If the date of the HoNOS assessment did not match the date of the visit as registered in the system, we took the first attended diary appointment date within 5 days prior to the HoNOS date. If a patient had more than one HoNOS assessment in a given year, the most recent assessment was included in the analysis. A focus group with senior clinical staff at OHFT was carried out to agree on the cluster data analysis and the interpretation of the results. For each cohort, we identified patients who did and did not move to a different superclass in the years considered. Changes in HoNOS scores and clusters were analysed within the same superclass. The Friedman test was used to test for statistical significance in three or more dependent samples. Where appropriate, post hoc analysis for differences between pairs of years (eg, 2018–2019, 2019–2020) was conducted using the Wilcoxon signed-rank test with Bonferroni correction. Where there were only two dependent samples, the Wilcoxon signed-rank test was used. An alpha level of 0.001 was preferred in the interpretation of statistical test results as recommended and used in real-world observational studies of health services.11 12 Statistical analyses were performed in Python.

    Findings

    Impact on service delivery

    There were 204 504 registered mental health patients in OHFT and 166 702 in SHFT (table 1). Of note, Child and Adolescent Mental Health Service data were not available for SHFT. However, most service activity variables, including referral, discharge and inpatient measures, were common to the two Trusts (table 1; online supplemental appendix, Sections 2 and 5). From 2019 to 2020, there was a reduction in measures of turnover, such as referrals and discharges, in both SHFT and OHFT (table 1). In OHFT, the 2019-to-2020 change in these measures reversed the increasing trend in referral and discharge numbers observed from 2018 to 2019, while SHFT experienced yearly decrease in referrals and discharges from 2018 to 2020 (table 1). Similar yearly trends were found in measures of inpatient service turnover at OHFT, while yearly increases in these measures were observed at SHFT (online supplemental appendix, table 8A). Despite these differences, both Trusts consistently reported a large decrease in referrals per month from March to April 2020 (online supplemental appendix, table 6D and figure 6H) and a decrease in number of distinct inpatients in March 2020 (online supplemental appendix, table 8C and figure 8A). However, monthly referral activity returned to pre-lockdown levels by June/July 2020 (online supplemental appendix, table 6D and 6H).

    View this table:
    Table 1

    Aggregate service measures from 2018 to 2020

    The total number of scheduled appointments increased in 2020 (table 1). In OHFT, there was a 15.4% overall increase in scheduled appointments from 2019 to 2020, compared with a 3.9% increase from 2018 to 2019 (table 1). In SHFT, there was a 5.6% increase in scheduled appointments from 2019 to 2020, compared with a stable number of total appointments from 2018 to 2019 (table 1).

    Remote consultations

    The number of non-face-to-face (remote) appointments—comprising video and telephone consultations—increased in both Trusts during the pandemic, going from 4685 to a peak of 26 245 appointments in OHFT and from 7117 to 24 987 appointments in SHFT, in February to June 2020. The monthly averages from April to September 2020 were 23 030 and 22 977 remote appointments/month in OHFT and SHFT, respectively (figure 1). At OHFT video consultations contributed to an average of 30.5% of the total telepsychiatric services per month from April to September 2020, while, prior to the pandemic, they comprised less than 10% (9.1% in February 2020) (figure 2). SHFT registered a smaller proportion of video consultations out of the total number of telepsychiatry activities (monthly average 7.6% from April to September 2020), but it is worth noting that less than 0.2% of all remote consultations were done by video before March 2020 (figure 2). The baseline attendance rate in February 2020, regardless of modality, was lower in SHFT (78.2% for in-person and 87.1% for remote) than OHFT (84.4% and 92.4%, respectively) (figure 3; online supplemental appendix, table 6E). Month-by-month analysis showed that attendance rate to remote visits decreased between January and September 2020 in both Trusts, and at OHFT, it became lower than the attendance rate of in-person appointments from May 2020 onwards (figure 3; online supplemental appendix, table 6E). After the lockdown, no-show rates increased for remote appointments at OHFT (4.6% in February to 8.2% in September 2020) and decreased for in-person appointments at OHFT (from 6.1% to 4.8%) and SHFT (from 9.6% to 7.0%) (online supplemental appendix, figure 6J). No-show rates at remote appointments at SHFT remained largely stable pre-to-post lockdown, with a rate of 11.6% in both February and September 2020 (online supplemental appendix, figure 6J).

    Figure 1
    Figure 1

    Attended face-to-face (F2F) and non-face-to-face (Non-F2F) appointments.

    Figure 2
    Figure 2

    Attended appointments by telephone and video.

    Figure 3
    Figure 3

    Attendance rate of face-to-face (F2F) and non-face-to-face (Non-F2F) appointments in 2020.

    The analysis of diagnostic group-level data (based on International Statistical Classification of Diseases and Related Health Problems, version 10 codes, or treatment team) showed that the ratio of remote to in-person appointments increased from 2019 to 2020 in all diagnostic groups (online supplemental appendix, figure 6C,G). The greatest change was for Eating Disorders (from 0.12 to 1.51 and 0.10 to 2.53 at OHFT and SHFT, respectively; online supplemental appendix, table 6B,C), with other groups, such as mental retardation (2.61), developmental disorders (2.20), dementia (2.11) and conduct disorders (2.05) to follow (online supplemental appendix, table 6B). For patients with dementia at OHFT, non-attendance rates at in-person appointments were fourfold higher than remote appointments (17.2% vs 3.9%) (online supplemental appendix, Table 6F).

    Analysis of HoNOS data

    Only OHFT reported HoNOS data. Cohort 1 (2018, 2019, 2020) comprised 998 patients (online supplemental appendix, table 7A). Of them, 792 patients (79.4%) stayed within the same superclass: 284 (35.9%) non-psychotic, 174 (22.0%) organic and 332 (41.9%) psychotic (2 remaining patients were in the ‘variance cluster’, ie, in need of mental healthcare, but not classified into any superclass) (online supplemental appendix, table 7B). Analysing the change of clusters within the same superclass, only the organic superclass reported a statistically significant difference in the cluster value between 2018 and 2020 (p<0.001) (online supplemental appendix, table 7L). A post hoc pairwise test showed that the significant increase happened between 2019 and 2020, but not between 2018 and 2019 (online supplemental appendix, table 7N). The results of the cluster analysis were supported by the analysis of HoNOS total scores within the same superclass (online supplemental appendix, tables 7E, 7F, 7H).

    Cohort 2 (2019, 2020) included 2712 patients, of whom 2318 (85.5%) stayed within the same superclass from 2019 to 2020: 909 patients (39.2%) were non-psychotic, 687 (29.6%) organic, 720 (31.1%) psychotic and 2 (0.1%) in the variance cluster (online supplemental appendix, tables 7O,P). The analysis of clusters within the same superclass showed increased complexity in the organic superclass (p<0.001) (online supplemental appendix, table 7Z). In these patients, the HoNOS total score worsened significantly (p<0.001), while the non-psychotic and psychotic superclass showed an improvement in total HoNOS score (online supplemental appendix, Table 7U). The complete description of HoNOS results is available in the online supplemental appendix, Section 7.

    Discussion

    Our study of two large NHS mental health Trusts demonstrated a rapid shift to remote service delivery as a result of the COVID-19 pandemic and associated national lockdown. These findings are consistent with other published reports.7 13–15 However, this is the first study assessing the clinical impact on mental health services and patients. Taken together, our study suggests that remote service activity increased markedly, with considerable use of video consultations, but this did not automatically translate to more patients attending the scheduled appointments. Notably, our findings on non-attendance differ with the experience of mental healthcare providers in the USA who have reported reductions in tele-mental health no-show rates after rapid virtualisation of services.16 17 In contrast, no-show rates at remote consultations increased in OHFT and remained stable in SHFT after lockdown-related service changes. We could not directly compare associated transatlantic clinical outcomes due to a dearth of such analyses in the literature. However, our study suggests that the impact on service activity is health system, region and diagnosis specific. Associated changes in clinical outcomes may also partly reflect such differences, but this currently remains a hypothesis to be tested. Importantly, our analysis of clinical outcomes shows that they can vary according to the broad diagnostic ‘superclasses’ as defined in the UK mental healthcare system.

    In terms of clinical outcomes, during the pandemic, patients belonging to the organic superclass significantly increased in healthcare complexity in comparison with previous years. While ICD-10 diagnoses and HoNOS clusters are not exactly the same,9 there is up to 76% overlap between the two for inpatients with a diagnosis of dementia and organic disorders.18 HoNOS scores can be a significant resource to gain insights into mental illness and treatment effectiveness.19 These findings are consistent with reports of pandemic-related worsening of behavioural and psychological symptoms in people with dementia,20 and can have implications for patients, carers and mental health services at different levels. While the remote attendance rates of patient with dementia were high, the effectiveness and usefulness of these consultations have not been established. A recent review suggests that remote management of dementia could achieve a level of diagnostic accuracy and patient and caregiver satisfaction comparable to in-person consultations.21 However, factors such as reduced cognitive function, confusion and sensory impairments may reduce the quality of teleconsultations for these patients as they may find it difficult to engage with their healthcare clinician using a virtual format.20 Literacy, availability and familiarity with the internet may also be an issue. For instance, in the UK in 2019, only 47% of adults above 75 years of age were considered recent internet users, compared with virtually all (99%) adults aged 16–44 years.22 Patient difficulties with using technology for telepsychiatry have been reported in both the 55–64 and 65–74 year old age groups.17

    The eating disorders (ED) group had the greatest relative pre-to-post increase in the use of telepsychiatry. Evidence suggests that the psychological well-being of individuals with ED has been particularly impacted by the pandemic.23 One ED-focused service evaluation study showed unexpected benefits of teleconferencing, including increased participation of patients in therapeutic groups and enhanced interdisciplinary communication between staff, but stated that significant practical challenges remained in the implementation of telepsychiatry for people with ED.24

    Questions remain about the wider applicability of telepsychiatry across the whole spectrum of mental health disorders. For personality disorders, schizophrenia and substance misuse diagnostic groups, in our study the ratio of remote versus in-person appointments was less than 1, indicating greater difficulty of remotely managing patients characterised by a more disruptive behavioural component. Further work remains to be done in better understanding how telepsychiatry services could be improved, and for whom and in what situations different remote modalities are most appropriate. This is important, as our study suggests, that the rapid shift to remote service delivery has not reached some groups of patients who may require more tailored management with telepsychiatry. Specific groups of patients may require more intensive and/or nuanced management in the post-pandemic era of digital psychiatry. Hybrid models of care, combining digital psychiatry with face-to-face assessment and care coordinators to support patients in overcoming practical problems, may be an effective way of managing highly complex groups of patients with a more personalised and blended approach. The quality, effectiveness and uptake of telepsychiatry can be enhanced by training clinicians25 and patients.26 There is a need for evidence-based, targeted telepsychiatric training programmes to enhance clinical efficacy and health outcomes in patients with mental health disorders. Training will be especially important not only in terms of assessment of patients via video or telephone consultations but also in terms of remote monitoring of symptoms27 and delivery of treatment modules.28

    The strengths of our study include the size of the dataset, comparison across two large NHS mental health Trusts in England and the use of patient-level data to characterise the impact of the pandemic, using data analysed over 3 years to conduct serial comparisons. However, the HoNOS analysis was not replicated at SHFT as the Trust does not readily store cluster-level HoNOS data in an anonymised format; hence, the patient-level outcomes are limited to OHFT. Moreover, the months considered were limited to January–September, meaning some significant events, such as the second national lockdown (5 November 2020), were excluded. Additional studies should be conducted to examine the long-term patterns in telepsychiatry use and the impact on clinical outcomes beyond the pandemic-associated acute service transformation. Furthermore, the findings cannot immediately be generalised to populations or healthcare settings outside of the Trusts. Patients seen in secondary/specialist mental health settings have distinct, often more complex, needs compared with those who are treated in primary care mental health settings.29 Future studies should aim to quantify these differences, adjust for individual and regional socioeconomic factors, and consider Trust-specific differences in the structure of teams and delivery of mental health services.

    This work is primarily a descriptive study of service changes. Patient outcomes and various dimensions of the quality of telepsychiatric services, such as effectiveness, utility and acceptability, cannot be definitively correlated. While we have demonstrated that it is feasible to use HoNOS data to assess patient outcomes and generate clinically meaningful insights,8 30 the HoNOS has been shown to have moderate inter-rater reliability and limited validity in relation to patient-reported symptomatology.31

    Our work reveals challenges in conducting a study using EHR from different mental health Trusts, each one with their own data infrastructure. OHFT and SHFT had different local EHR systems with some, although relatively minor, variations in definitions of variables, and the data infrastructure. A shared data plan was developed in the protocol and aimed at ensuring overall consistency in extracting variables of interest, their descriptions and optimising workflow. The collaboration required frequent, often separate discussions between individual data teams to ensure that analyses represented the same underlying variables, produced comparable visualisations and made any amendments to reflect theoretical iterations of the project. As EHR studies may generate important data-driven insights during the COVID-19 pandemic and beyond, more efficient and integrated data infrastructures, such as the development of a federated database, are likely to play an increasingly important role in the future of mental health research and service improvement processes.32

    Data availability statement

    Data may be obtained from a third party and are not publicly available. NHS data may be obtained from a third party and are not publicly available.

    Ethics statements

    Patient consent for publication

    Acknowledgments

    AC and KS are supported by the National Institute for Health Research (NIHR) Oxford Cognitive Health Clinical Research Facility. AC is also supported by an NIHR Research Professorship (grant RP-2017-08-ST2-006), by the NIHR Oxford and Thames Valley Applied Research Collaboration and by the NIHR Oxford Health Biomedical Research Centre (grant BRC-1215-20005). CH is also supported by the NIHR Senior Nurse and Midwife Research Leader Programme. The views expressed are those of the authors and not necessarily those of the UK National Health Service, the NIHR or the UK Department of Health. We would like to acknowledge the work and support of the Oxford Research Informatics Team, Adam Pill and Suzanne Fisher, CRIS Academic Support and Information Analysts. We would also like to thank Lisa Atkin and Matthew Taylor for taking part in focus groups to guide the analysis and interpretation of clinical outcomes. We finally thank the anonymous peer reviewers for providing critical commentary which improved the analysis and the quality of the overall manuscript.

    References

    1. ↵
      1. Melcher J,
      2. Hays R,
      3. Torous J
      . Digital phenotyping for mental health of college students: a clinical review. Evid Based Ment Health 2020;23:161–6.doi:10.1136/ebmental-2020-300180pmid:http://www.ncbi.nlm.nih.gov/pubmed/32998937
      OpenUrlAbstract/FREE Full Text
    2. ↵
      1. Blandford A,
      2. Wesson J,
      3. Amalberti R, et al
      . Opportunities and challenges for telehealth within, and beyond, a pandemic. Lancet Glob Health 2020;8:e1364–5.doi:10.1016/S2214-109X(20)30362-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/32791119
      OpenUrlPubMed
    3. ↵
      1. Sasangohar F,
      2. Bradshaw MR,
      3. Carlson MM, et al
      . Adapting an outpatient psychiatric clinic to telehealth during the COVID-19 pandemic: a practice perspective. J Med Internet Res 2020;22:e22523.doi:10.2196/22523pmid:http://www.ncbi.nlm.nih.gov/pubmed/32936768
      OpenUrlPubMed
    4. ↵
      1. Moreno C,
      2. Wykes T,
      3. Galderisi S, et al
      . How mental health care should change as a consequence of the COVID-19 pandemic. Lancet Psychiatry 2020;7:813–24.doi:10.1016/S2215-0366(20)30307-2pmid:http://www.ncbi.nlm.nih.gov/pubmed/32682460
      OpenUrlPubMed
    5. ↵
      1. Perera G,
      2. Broadbent M,
      3. Callard F, et al
      . Cohort profile of the South London and Maudsley NHS Foundation trust biomedical research centre (SLAM BRC) case register: current status and recent enhancement of an electronic mental health Record-derived data resource. BMJ Open 2016;6:e008721.doi:10.1136/bmjopen-2015-008721pmid:http://www.ncbi.nlm.nih.gov/pubmed/26932138
      OpenUrlAbstract/FREE Full Text
    6. ↵
      1. Pierce M,
      2. Hope H,
      3. Ford T, et al
      . Mental health before and during the COVID-19 pandemic: a longitudinal probability sample survey of the UK population. Lancet Psychiatry 2020;7:883–92.doi:10.1016/S2215-0366(20)30308-4pmid:http://www.ncbi.nlm.nih.gov/pubmed/32707037
      OpenUrlCrossRefPubMed
    7. ↵
      1. Stewart R,
      2. Martin E,
      3. Broadbent M
      . Mental health service activity during COVID-19 lockdown: South London and Maudsley data on working age community and home treatment team services and mortality from February to mid-May 2020. medRxiv 2020:2020.06.13.20130419.
    8. ↵
      1. Laugharne R,
      2. Eaves S,
      3. Mascas A, et al
      . Desperately seeking outcomes: quantifying the effectiveness of community mental healthcare using health of the nation outcome scales. BJPsych Open 2018;4:91–4.doi:10.1192/bjo.2018.3pmid:http://www.ncbi.nlm.nih.gov/pubmed/29606183
      OpenUrlPubMed
    9. ↵
      1. Yeomans D
      . Clustering in mental health payment by results: a critical summary for the clinician. Advances in Psychiatric Treatment 2014;20:227–34.doi:10.1192/apt.bp.113.011320
      OpenUrlAbstract/FREE Full Text
    10. ↵
      1. Jacobs R,
      2. Chalkley M,
      3. Böhnke JR, et al
      . Measuring the activity of mental health services in England: variation in Categorising activity for payment purposes. Adm Policy Ment Health 2019;46:847–57.doi:10.1007/s10488-019-00958-7pmid:http://www.ncbi.nlm.nih.gov/pubmed/31352638
      OpenUrlCrossRefPubMed
    11. ↵
      1. Johnson VE
      . Revised standards for statistical evidence. Proc Natl Acad Sci U S A 2013;110:19313–7.doi:10.1073/pnas.1313476110pmid:http://www.ncbi.nlm.nih.gov/pubmed/24218581
      OpenUrlAbstract/FREE Full Text
    12. ↵
      1. Amin A,
      2. Keshishian A,
      3. Trocio J, et al
      . A real-world observational study of hospitalization and health care costs among nonvalvular atrial fibrillation patients prescribed oral anticoagulants in the U.S. Medicare population. J Manag Care Spec Pharm 2018;24:911–20.doi:10.18553/jmcp.2018.24.9.911pmid:http://www.ncbi.nlm.nih.gov/pubmed/30156450
      OpenUrlPubMed
    13. ↵
      1. Patel R,
      2. Irving J,
      3. Brinn A, et al
      . Impact of the COVID-19 pandemic on remote mental healthcare and prescribing in psychiatry. medRxiv 2020:2020.10.26.20219576.
    14. ↵
      1. Connolly SL,
      2. Stolzmann KL,
      3. Heyworth L, et al
      . Rapid increase in Telemental health within the Department of Veterans Affairs during the COVID-19 pandemic. Telemed J E Health 2021;27:454-458.doi:10.1089/tmj.2020.0233pmid:http://www.ncbi.nlm.nih.gov/pubmed/32926664
      OpenUrlPubMed
    15. ↵
      1. Chen JA,
      2. Chung W-J,
      3. Young SK, et al
      . COVID-19 and telepsychiatry: early outpatient experiences and implications for the future. Gen Hosp Psychiatry 2020;66:89–95.doi:10.1016/j.genhosppsych.2020.07.002pmid:http://www.ncbi.nlm.nih.gov/pubmed/32750604
      OpenUrlCrossRefPubMed
    16. ↵
      1. Mishkind MC,
      2. Shore JH,
      3. Bishop K, et al
      . Rapid conversion to Telemental health services in response to COVID-19: experiences of two outpatient mental health clinics. Telemed J E Health 2021;27:778–84.doi:10.1089/tmj.2020.0304pmid:http://www.ncbi.nlm.nih.gov/pubmed/33393857
      OpenUrlPubMed
    17. ↵
      1. Guinart D,
      2. Marcy P,
      3. Hauser M, et al
      . Mental health care providers' attitudes toward Telepsychiatry: a Systemwide, multisite survey during the COVID-19 pandemic. Psychiatr Serv 2021;72:704–7.doi:10.1176/appi.ps.202000441pmid:http://www.ncbi.nlm.nih.gov/pubmed/33593104
      OpenUrlPubMed
    18. ↵
      1. Trevithick L,
      2. Painter J,
      3. Keown P
      . Mental health clustering and diagnosis in psychiatric in-patients. BJPsych Bull 2015;39:119–23.doi:10.1192/pb.bp.114.047043pmid:http://www.ncbi.nlm.nih.gov/pubmed/26191449
      OpenUrlPubMed
    19. ↵
      1. Dean NJ,
      2. Arnaoutoglou N,
      3. Underwood BR
      . Effectiveness of treatment for 6813 patients with mental health conditions in Cambridgeshire: a cross-sectional study. BJPsych Open 2020;6:e30.doi:10.1192/bjo.2020.14pmid:http://www.ncbi.nlm.nih.gov/pubmed/32192545
      OpenUrlPubMed
    20. ↵
      1. Soares WB,
      2. Silvestre IT,
      3. Lima AMdeO, et al
      . The influence of telemedicine care on the management of behavioral and psychological symptoms in dementia (BPSD) risk factors induced or exacerbated during the COVID-19 pandemic. Front Psychiatry 2020;11:577629.doi:10.3389/fpsyt.2020.577629pmid:http://www.ncbi.nlm.nih.gov/pubmed/33101090
      OpenUrlPubMed
    21. ↵
      1. Hatcher-Martin JM,
      2. Adams JL,
      3. Anderson ER, et al
      . Telemedicine in neurology: telemedicine work group of the American Academy of Neurology update. Neurology 2020;94:30–8.doi:10.1212/WNL.0000000000008708pmid:http://www.ncbi.nlm.nih.gov/pubmed/31801829
      OpenUrlPubMed
    22. ↵
      1. Office for National Statistics
      . Internet users, UK, 2019. Available: https://www.ons.gov.uk/businessindustryandtrade/itandinternetindustry/bulletins/internetusers/2019 [Accessed 28 Mar 2021].
    23. ↵
      1. Branley-Bell D,
      2. Talbot CV
      . Exploring the impact of the COVID-19 pandemic and UK lockdown on individuals with experience of eating disorders. J Eat Disord 2020;8:44.doi:10.1186/s40337-020-00319-ypmid:http://www.ncbi.nlm.nih.gov/pubmed/32874585
      OpenUrlPubMed
    24. ↵
      1. Datta N,
      2. Derenne J,
      3. Sanders M, et al
      . Telehealth transition in a comprehensive care unit for eating disorders: Challenges and long‐term benefits. International Journal of Eating Disorders 2020;53:1774–9.doi:10.1002/eat.23348
      OpenUrl
    25. ↵
      1. Torous J,
      2. Wykes T
      . Opportunities from the coronavirus disease 2019 pandemic for transforming psychiatric care with telehealth. JAMA Psychiatry 2020;77:1205.doi:10.1001/jamapsychiatry.2020.1640pmid:http://www.ncbi.nlm.nih.gov/pubmed/32391857
      OpenUrlPubMed
    26. ↵
      1. Hoffman L,
      2. Wisniewski H,
      3. Hays R, et al
      . Digital opportunities for outcomes in recovery services (doors): a pragmatic Hands-On group approach toward increasing digital health and smartphone competencies, autonomy, relatedness, and alliance for those with serious mental illness. J Psychiatr Pract 2020;26:80–8.doi:10.1097/PRA.0000000000000450pmid:http://www.ncbi.nlm.nih.gov/pubmed/32134881
      OpenUrlPubMed
    27. ↵
      1. Goodday SM,
      2. Atkinson L,
      3. Goodwin G, et al
      . The true colours remote symptom monitoring system: a decade of evolution. J Med Internet Res 2020;22:e15188.doi:10.2196/15188pmid:http://www.ncbi.nlm.nih.gov/pubmed/31939746
      OpenUrlPubMed
    28. ↵
      1. Gratzer D,
      2. Torous J,
      3. Lam RW, et al
      . Our digital moment: innovations and opportunities in digital mental health care. Can J Psychiatry 2021;66:5–8.doi:10.1177/0706743720937833pmid:http://www.ncbi.nlm.nih.gov/pubmed/32603188
      OpenUrlPubMed
    29. ↵
      1. Twomey C,
      2. Prina AM,
      3. Baldwin DS, et al
      . Utility of the health of the nation outcome scales (HoNOS) in predicting mental health service costs for patients with common mental health problems: historical cohort study. PLoS One 2016;11:e0167103.doi:10.1371/journal.pone.0167103pmid:http://www.ncbi.nlm.nih.gov/pubmed/27902745
      OpenUrlCrossRefPubMed
    30. ↵
      1. Crawford MJ,
      2. Zoha M,
      3. Macdonald AJD, et al
      . Improving the quality of mental health services using patient outcome data: making the most of HoNOS. BJPsych Bull 2017;41:172–6.doi:10.1192/pb.bp.116.054346pmid:http://www.ncbi.nlm.nih.gov/pubmed/28584656
      OpenUrlPubMed
    31. ↵
      1. Brooks R
      . The reliability and validity of the health of the nation outcome scales: validation in relation to patient derived measures. Aust N Z J Psychiatry 2000;34:504–11.doi:10.1080/j.1440-1614.2000.00755.xpmid:http://www.ncbi.nlm.nih.gov/pubmed/10881976
      OpenUrlCrossRefPubMed
    32. ↵
      1. Mandl KD,
      2. Kohane IS
      . Federalist principles for healthcare data networks. Nat Biotechnol 2015;33:360–3.doi:10.1038/nbt.3180pmid:http://www.ncbi.nlm.nih.gov/pubmed/25850061
      OpenUrlCrossRefPubMed

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Twitter @jsw_hong, @TomasPsychology, @And_Cipriani

    • Contributors JSWH contributed to data analysis, interpretation, manuscript drafting and revision. KS contributed to the conception and planning of the project, the analysis plan and interpretation of the data. TS developed the data extraction plan, contribution to the analysis and interpretation of the data. RS wrote the protocol, major contribution to the conception and planning of the project, contributed to the analysis plan and interpretation of the data. FS participated in the data extraction for OHFT, contribution to data extraction plan, performed the data analysis for OHFT, documented the HoNOS analysis methodology and interpretation of the data. PP and SR contributed to the planning at SHFT, data extraction plan, analysis and review the manuscript. TE contributed to the analysis of the Southern Health dataset, write the methods section and reviewed the manuscript. MD contributed to the design of Telemachus programme, engagement with NHS organisations and manuscript review. AC designed the study, secured the funding, contributed to the analysis and interpretation of the data, drafted and revised the manuscript. All other coauthors contributed to the interpretation of the data and critically reviewed the manuscript. All authors approved the final draft for publication. AC is responsible for the overall content as guarantor.

    • Funding This study was funded by Angelini Pharma (HQR02150) and by the NIHR Oxford Health Biomedical Research Centre (BRC-1215-20005), and it is part of a larger project including service evaluation, focus groups and digital surveys on telepsychiatry interventions in the UK and Italy (TELEMACHUS: TELEpsychiatry to Manage clinical Assessments during COVID-19: cHallenges and opportUnities for the NHS).

    • Competing interests AC has received research and consultancy fees from INCiPiT (Italian Network for Paediatric Trials), CARIPLO Foundation and Angelini Pharma, outside the submitted work. AT has received research and consultancy fees from INCiPiT (Italian Network for Paediatric Trials) and Angelini Pharma, outside the submitted work. PP reports grants from NovoNordisk, other from Queen Mary University of London, other from John Wiley & Sons, outside the submitted work. SR reports educational support from Janssen, Otsuka and Lundbeck, outside the submitted work.

    • Patient and public involvement statement This study was designed and co-produced with the direct involment of people with lived experience of mental health disorders.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.