Skip to main content
  • Study protocol
  • Open access
  • Published:

Evaluation of smartphone APP-based case-management services among antiretroviral treatment-naïve HIV-positive men who have sex with men: a randomized controlled trial protocol

Abstract

Background

Men who have sex with men (MSM) are disproportionally affected by HIV in China. ‘Treatment as Prevention’ is a promising strategy for HIV prevention but requires adequate adherence. Mobile health (mHealth) may be an acceptable and feasible approach for service delivery, but there is little evidence supporting mHealth intervention for improving antiretroviral treatment adherence among HIV-infected MSM in low- and middle-income countries, including China. This study will aim to develop a smartphone application-based case-management service and compare its efficacy to standard care with regards to adherence, CD4, HIV viral load and psychosocial outcomes among MSM patients in Guangzhou, China.

Methods

A non-blinded 1:1 parallel-group randomised controlled trial will be conducted in Guangzhou Eighth People’s Hospital, with 300 MSM enrolled in each arm. Eligible MSM who are newly initiating ART will be randomly assigned to an intervention group (standard-of-care case management plus mHealth intervention) or a control group (standard-of-care case management). The development of the mHealth intervention will be based on the information–motivation–behavioural skills theory of ART adherence, and comprise four components: educational articles, one-to-one online communication with case managers, support-service information and hospital-visit reminders. Outcome measures will be collected at baseline and at months 1, 3, 6, and 12. The primary outcomes will be ART adherence and CD4 count at month 6. Secondary outcomes include HIV RNA, sexual behaviours, mental health status, illness perceptions, and quality of life. χ2 test and t-test will be used for between-group comparisons. Intervention effects will be evaluated using General estimating equation performed by SAS 9.0, on the principle of intention-to-treat. Structural equation modelling will be used to test potential mechanisms of intervention effect.

Discussion

This study is the first to explore the efficacy of mHealth intervention in the case management services targeted at HIV-infected MSM in low-and middle-income countries. Once proven effective, the innovative mHealth service could be integrated into the routine case management of PLWH. as well as be tailored to the patient management service for other chronic conditions.

Trial registration

ClinicalTrial.gov: NCT03860116; Registered on 1 March 2019.

Peer Review reports

Background

Men who have sex with men (MSM) have been identified as one of the key populations in the HIV epidemic. The prevalence of HIV among MSM in China has been rapidly and continuously increasing, especially in metropolitan cities such as Beijing, Guangzhou, and Chengdu [1,2,3]. The pooled prevalence of HIV infection among MSM soared from 1.4% in 2001 to 9.0% in 2013 [4], and transmissions between MSM accounted for 25.5% of the new HIV infections in 2017 [5].

‘Treatment as Prevention’ (TasP) is a promising strategy for HIV prevention and control. In 2016, the Chinese Centre for Disease Control and Prevention (CDC) revised the national guidelines for antiretroviral treatment (ART) and suggested immediate treatment for all people living with HIV/AIDS (PLWHA) [6]. However, the long-term success of such efforts to scale-up coverage of ART for PLWHA may be negatively affected by deficits in adherence to the spectrum of HIV care. ART requires adequate adherence (usually defined as intake of at least 95% of ART doses [7, 8]) to yield satisfactory clinical outcomes, e.g., suppressing HIV viral replication [9], improving quality of life [10], and preventing transmission [11]. However, a recent meta-analysis indicated that only 77.6% (95% confidence interval [CI]: 71.6–83.1) of Chinese PLWHA on ART maintained adequate adherence [12]. Barriers to ART adherence include patient factors (e.g., education, lack of self-efficacy, mental illness), medication factors (e.g., side effects, pill burden, and food requirement), and system of care (e.g., linkage of care, negative experiences of healthcare system) [13,14,15]. Moreover, MSM can experience a fear of disclosure in public and healthcare settings and the stigma associated with their sexual orientation [16]. Thus, the ART adherence of this group warrants more attention.

Case management has been regarded as a routine practice to address the complex medical barriers to and needs for ART adherence [17, 18]. It incorporates a wide range of medical and supportive services and is routinely used in many countries. It has been shown to be effective in increasing CD4 counts [19], and in improving care engagement, ART adherence and quality of life [20, 21], as well as in reducing transmission-risk behaviours among PLWHA [22]. Concurrently, the development of information technology has led to a paradigm of mobile health (mHealth), which is characterized by the communication and delivery of health services via text messages, smartphone applications (APPs), websites, and social media. Consequently, mHealth has become a highly accessible and adaptive approach to medical practice, especially in low- and middle-income countries [23]. Currently, smartphones are used by 68% of the population in China [24], with up to 97.5% of MSM having a smartphone [25]. Adherence-supporting interventions incorporating mHealth have shown good acceptability and feasibility in HIV-positive MSM [26], and demonstrated efficacy in improving PLWHA’s linkage to care, retention in treatment and adherence to ART [27, 28]. However, to the authors’ knowledge, mHealth interventions to improve the adherence of HIV-positive MSM to ART and potential mechanisms of how an intervention works have not been fully explored in China, and many mHealth interventions have mainly relied on text messages [29]. Thus, there is a need for a rigorously designed and comprehensive mHealth intervention tailored to Chinese HIV-positive MSM.

In addition, it has been suggested in reviews that theory-based behavioural interventions are more likely to be effective than non-theory-based interventions [30]. The Information-Motivation-Behavioural Skills (IMB) model of ART adherence, for example, is widely used to understand the dynamics of ART adherence and to intervene with patients to promote better adherence [31]. This model has been effectively tested across diverse populations worldwide [32,33,34,35] and has often been used to guide the development of interventions to promote HIV-related healthy behaviour, such as ART adherence, retention and safe sex [36,37,38,39]. According to the IMB model, information on HIV and ART interacts with an individual’s motivation to take ART medication, thereby influencing the development of behavioural skills related to adherence; in other words, well-informed and motivated patients with adequate skills for enacting adherence-related behaviours will exhibit better adherence [40]. Further, psychological functioning (e.g., depression) and access to care are moderating factors in the associations between predictors and adherence outcomes [31] (Fig. 1).

Fig. 1
figure 1

Theoretical framework

The objective of this study is to design and evaluate the efficacy of an APP-based case-management intervention among MSM who are newly initiating ART, using the IMB model as the theoretical framework. We have trained nurses to deliver the intervention, and here we describe the protocol of what will be an ongoing open-label randomised controlled trial. The SPIRIT guidelines were adhered to for reporting this manuscript.

Methods

Study aim and design

This is a single-centre, parallel-group, open-label randomised controlled trial. We have designed a comprehensive smartphone APP-based case-management service model for MSM newly initiating ART, and will evaluate its efficacy compared to standard-of-care (SOC) case-management services in improving ART medication adherence, HIV clinical indicators, and psychosocial outcomes. The collaborative technology company Trusted Doctor Inc. has developed a smartphone APP (‘Trusted Doctor’) that is linked to WeChat. WeChat is the most popular social media platform in China and has more than 1.1 billion active users according to Tencent Inc.’s 2019 annual report; 93% of the residents in the major cities of China use WeChat every day [41].

This study will be based on the combination of ‘Trusted Doctor’ and WeChat by establishing a link, called WeChat official account, between the two smartphone APPs. Certified health care providers (HCPs; i.e., doctors, nurses, or case managers) can use the ‘Trusted Doctor’ APP to have direct and instantaneous online interactions with patients. And patients can use the WeChat platform to access their health care providers, as well as receive services including article alerts, follow-up questionnaires, appointment reminders, free phone calls, etc.

Study setting

The study is being conducted in Guangzhou, the capital city of Guangdong Province in the southeast of China. HIV prevalence among MSM in Guangzhou increased from 3.9% in 2009 to 11.4% in 2013 [42]. By the end of 2018, 10,453 HIV cases had been reported in Guangzhou and 9161 PLWHA (87.6%) were undergoing ART (data extracted from China’s National Information System of HIV/AIDS prevention and treatment). Participant recruitment and implementation of this study is in cooperation with Guangzhou Eighth People’s hospital, the first authoritative hospital for ART provision in Guangdong. The hospital has provided > 18,000 PLWHA from all around the country with ART and follow-up supportive services and has been exploring the use of case-management services for PLWHA since 2018, in the form of educational sessions and periodic physical visits. Supportive counselling is also provided, if necessary.

Participants

Eligibility criteria

The participant inclusion criteria are as follows: 1) male, aged ≥18; 2) HIV-positive; 3) ART-naïve, and planning to initiate ART on the day of recruitment; 4) self-reported to be HIV-infected through homosexual transmission; 5) having access to the Internet on a smartphone; 6) having a WeChat account and using it in daily communication; 7) willing to provide written informed consent. Participants who have been hospitalized due to severe opportunistic infections are excluded.

Recruitment, enrolment and randomization

Potential participants are being recruited from the collaborative hospital that provides ART for PLWHA. Four nurse case managers with previous experience of case management are responsible for participant recruitment and intervention delivery. Before the commencement of this study, all nurse managers received two four-hour training sessions on recruitment, data collection, intervention specification, and tutoring about the APP. After nurse case managers confirm their eligibility and obtaining written informed consent, participants are invited to complete baseline assessments and then randomly assigned to intervention (SOC + mHealth) or control groups (SOC) in a 1:1 allocation ratio. Block randomization (block length = 4) is adopted and random numbers are generated with SAS 9.4.

Written records of group-assignments are sealed in individual opaque envelopes marked with study identification numbers. The intervention group then receive brief training in use of the WeChat program official account from the investigators and are reminded to not forward the WeChat platform to others, to avoid potential confounding effects. Both the control and intervention group are required to ‘friend’ the research team’s WeChat account for data collection and to provide user feedback. Participants’ enrolment information including name, research ID (RID), medical ID, phone number, and their case managers, are documented. The research ID is composed of a skewed date (month plus one) and the sequence of the intraday enrolment to ensure confidentiality. All participants are encouraged to refrain from seeking other health interventions during the active study period. Standard operating procedure (SOP) and program manuals have been developed to guide the study.

Intervention

Control group

The control group receives SOC service at the hospital, which commences with a 20-min ART education session for MSM patients newly initiating ART. Follow-up hospital visits are then arranged at one, two, three and 6 months after ART initiation. During each hospital visit, patients undergo physical assessment, receive prescription refills, meet with case managers and schedule the date of their next appointment. Case managers track patients’ medication adherence, sexual behaviours, mental health, sleep quality and weight changes to confirm that the current treatment regimen is suitable. Supportive counselling will be provided to patients, if necessary. If the patients display adequate adherence in the initial three-month period, the refill interval will be extended to 3 months, and case managers will then schedule a fourth meeting at 6 months. After that, patients with stable medication-taking and hospital-visiting behaviour will be referred to the non-governmental organization (NGO) Red Ribbon for scheduling of prescription refills.

Intervention development

The intervention group receive SOC service and additional mHealth interventions. The design of the mHealth intervention is ground in and informed by the IMB model, which is the most widely used conceptual model for ART adherence. The core content of effective interventions must address the underlying determinants of ART adherence, which, according to the IMB model, include access to adherence-related information, motivation, and behavioural skills.

Elicitation research was conducted to clarify the information, motivation, and behavioural factors that underlie adherence or nonadherence in the targeted group. This involved an extensive review of the literature, lengthy conversations with experts in the field, and elicitation work with multiple stakeholders, including ten HIV-infected MSM with different durations of ART, four nurse case managers, three software engineers, and four NGO staff. These inputs were used to tailor the intervention design, delivery, and content to the needs and characteristics of HIV/AIDS patient, and identified four major intervention components: exposure to educational articles, online communication, supportive service information and hospital-visit reminders (Table 1).

Table 1 Innovative case management service model

Delivery of educational articles

According to the IMB model, if a patient possesses the requisite adherence information, they are more likely to enact specific adherence behavioural skills and exhibit behavioural change. Thus, during the 6 months of the intervention period, a series of educational articles containing ART-adherence information and adherence skills are delivered automatically and freely to the participants via the WeChat official account.

The content of the educational articles was developed based on a literature review, individual interviews, and group discussions. For instance, qualitative interviews of ART-experienced MSM were conducted to discuss the health concerns and information they needed most during their initial 6 months of ART. According to the needs expressed by interviewees, different topics of educational articles were derived, and approximately 1400 educational articles were subsequently collected from social media accounts. Six trained medical undergraduates worked in pairs to read and screen these articles for comprehensibility and timeliness. The final selection of 209 articles was authorized by the authors and then edited to add illustrations and remove sensitive words (e.g. HIV/AIDS) from the titles to avoid inadvertent disclosure of HIV status. Then, the modified articles were examined by physicians to ensure the information was valid and timely. The articles spanned 13 topics related to aspects such as adherence, side effects, and mental health. Detailed information on the contents of the articles is shown in Table 2.

Table 2 Educational article topics

To improve the accessibility and effectiveness of the intervention, the time and frequency for supplying information to participants was considered. In terms of timing, it was decided to send the educational material to participants at 10 p.m. to optimize subjects’ assimilation of the information. This timing was for three reasons: 1) a previous study had suggested that the rate of reading articles published on WeChat accounts peaked at 10.00 p.m. [43]; 2) individual MSM patients had suggested in interviews that educational articles should be delivered at night to avoid inadvertent sero-status disclosure to colleagues in the daytime or to friends during off hours. 3) case managers also recommended 10:00 p.m. for article-delivery, as this would be close to most patients’ daily medication-taking schedule, i.e., half an hour before patients’ bedtime.

The frequency of articles delivered are different over time to avoid inciting negative behaviours among the participants [44]. Thus, in the first 2 weeks of month one, educational materials are sent to the participants daily, and then for the second 2 weeks of month one, they are sent three times per week. After this, two articles are delivered per week in the following 2 weeks, and then once per week for the rest of the intervention period. Currently, 132 participants have been assigned to the intervention group and a total of 44 articles have been sent to this arm, with an average of 112 reading times for each article. Participants can also retrieve previous published articles at any time, with instructions on retrieval reiterated in monthly reminders.

Individual online communication

One-to-one online communication can be initiated by participants or nurses to provide social and instrumental support during the process of ART. Participants can send messages to the nurse case managers if they have any question regarding ART, and nurses are requested to promptly respond to the inquiries, i.e., within 2 days. In addition, automatic messages are sent to each participant asking about their medication-taking behaviours and mental health status at day 7 and day 14 after ART initiation, which were suggested by case managers as being important time points for identification of non-adherent behaviour and to improve adherence.

The purpose of these nurse–participant interactions is to evaluate participants’ progress and help identify barriers to progress, and nurses are required to provide tailored feedback and personalized information to an enquiring patient. If a patient expresses negative emotions during the individual online communication, the case managers will, if necessary, provide emotional comfort and counselling. Considering the feasibility of intervention, the numbers of messages that patients can send are limited to a certain level to balance the workload of case managers and the needs of patients.

Hospital-visit reminders

A message-reminder module for hospital visits was designed to run autonomously, to facilitate patients’ medication-taking and retention in the ART program. Four follow-up hospital visits are scheduled at months 1, 2, 3 and 6 after ART initiation. Case managers and patients discuss and agree on an exact date for each hospital visit at patients’ convenience. The reminder messages are sent 7 days prior to each scheduled hospital visit. The wording of the reminders is intentionally vague to avoid unwanted disclosure of HIV infection.

Supportive service information

The IMB model suggests that psychological functioning (e.g., depression) and access to care might moderate the associations between predictors and adherence outcomes [31]. Thus, in this study we provide easily accessible linkage and referrals to supportive services, including psychological counselling, sexually transmitted diseases (STD) treatment, detoxification, LGBT support (from an NGO), amongst others. Interviews with staff of psychological institutes, NGOs, and detoxification service centres revealed major variations in services provided by different organizations in terms of cost and approach, and the complexity that would arise if those services were all to be provided in our program.

Therefore, information regarding the location, opening hours, and potential charges of these supportive service institutions are collated and sent to patients monthly via WeChat, as a module of interventions delivered in the present study. Patients can also easily require services from relevant organizations using the WeChat platform. The flow chart is shown in Fig. 2.

Fig. 2
figure 2

Flow chart of the trial

Pilot study

A 1-month pilot study was conducted among 20 participants to assess the feasibility and acceptance of intervention, and detect potential problems in wording or length of the questionnaire, thus providing information that might help to optimize the design of the intervention. Thus, some questionnaire wording was revised after the pilot study, and the limit on the number of messages that could be sent by patients was adjusted according to patients’ feedback. During the initial pilot phase, the nurse case managers were observed and supervised in the whole process of recruitment and implementation, and were advised if they displayed any inappropriate expressions or behaviours, e.g., exaggerating the benefits of the intervention or recommending other health information resources to participants. An additional training session were delivered to case managers to strengthen the SOP after the pilot study.

Patient and public involvement (PPI)

PPI representatives worked with us to develop and refine the design of the intervention. However, it was difficult to involve patients in other areas of the study due to data protection restrictions. Patients were not consulted to develop patient-relevant outcomes or interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy.

Data collection

Baseline assessment is initially being conducted at the enrolment procedure on a tablet, under the instruction of investigators. Data on demographic characteristics (age, marital status, education, monthly income, employment, and eHealth literacy), HIV status (transmission route, duration after diagnosis, sexual orientation, sero-status disclosure, STDs), psycho-social conditions (mental health, quality of life, illness representation, perceived stigma, social support, IMB constructs) and risky behaviours (sexual and drug use behaviours) are collected.

Follow-up assessments will be scheduled at months 1, 3, 6 and 12 (6-month post-intervention). The content of follow-up assessments at each time-point is described in Table 3, and covers variables related to ART adherence, mental health, illness representation, sexual risk behaviour, IMB constructs and quality of life. The links to the online questionnaire will be sent to all participants automatically via WeChat, and they can fill in the questionnaire on any electronic device, such as smartphones. Questionnaires are checked weekly and reminder messages are sent to participants who have not completed the questionnaires in the required time. If the reminders have been sent twice and ignored twice, a phone call will be made to participants. Participants can receive an online cash transfer of 10 Chinese Yuan (CNY), 20 CNY and 20 CNY (50 CNY in total, which equals approximately 7 US dollars) if they finish the follow-up questionnaires in each of months 3, 6 and 12.

Table 3 Schedule of enrolment, interventions, and assessments

Measures

Primary outcome

The primary outcome of this study is self-reported ART medication adherence and CD4 count at month six. As literature studies have shown that self-reported adherence is usually positively skewed, mostly due to social desirability bias and recall bias [45], we adopt a composite scale that has high sensitivity [46]. This scale consists of three adherence items measuring medication-taking adherence in the preceding 30 days: 1) a ‘days missed’ item, in which participants are asked the number of days they have missed an ART dose; 2) a ‘rating’ item, in which participants are asked to rate their pill-taking performance; 3) a ‘frequency’ item, in which participants are asked how frequently they took their pills according to instructions. Cronbach’s alpha was 0.86 in the previously reported study using this scale [47]. Moreover, item responses for the three adherence items were linearly transformed to a 0–100 scale and the average scores of all three items are used. A summary of the individual adherence items was calculated as the mean of the three individual items. In this study, we consider anyone scoring below the maximum score on any item as non-adherent (i.e., a combined scale score of < 100). CD4 counts are examined in each physical test and measured by flow cytometer. The months one and three assessments allow for trend evaluation and the month 12 assessments will allow for evaluation of persistent effect of intervention, but these are not the primary outcomes.

Secondary outcomes

HIV-1 RNA viral load at month 12 was measured by polymerase chain reaction (PCR).

Mental health status includes anxiety and depression, which are measured by the seven-item Generalized Anxiety Disorder-7 (GAD-7) instrument [48] and nine-item Patient Health Questionnaire-9 (PHQ-9) [49], respectively. Both instruments comprise four-point Likert scales with the responses based on occurrence of symptoms during the last week. In each scale, the sum of all items will be calculated and categorized into different levels of mental health disorder: normal (0–4), mild (5–9), moderate (10–14), and moderate to severe (≥15) [50, 51].

Quality of life is measured by the 31-item HIV adaptation of the World Health Organization Quality of Life scale (WHOQOL-HIV bref) [52], which has shown good reliability and validity among Chinese PLWHA [53]. The scale consists of six domains: physical health, level of independence, psychological health, spirituality, social relations and environmental health. For each domain, a score ranging from 4 to 20 will be calculated, with higher scores denoting better quality of life.

Illness representation is measured by the Brief Illness Perception Questionnaire (B-IPQ), which has been used among PLWHA to evaluate their cognitive and emotional response towards HIV [54]. The B-IPQ comprises the following eight dimensions: consequences, timeline, personal control, treatment control, identity, coherence, concern, and emotional response. Each dimension has only one question, with responses rated on a scale from 0 to 10. A higher score indicates a greater perception of a threat from HIV [55].

ART-related perception is measured by the Life Windows Information-Motivation- Behavioural Skills ART Adherence Questionnaire (LW-IMB-AAD) [56]. The instrument consists of 33 items rated on a 5-point Likert scale, with nine measuring information, ten measuring motivation, and 14 measuring behavioural skills. Summative scores are computed for each aspect, with higher scores reflecting better information, motivation and behavioural skills about adherence [57].

Sexual risk behaviour before and after diagnosis is measured separately using self-constructed items, including numbers of male/female and regular/casual partners, and frequency of condom use. Example items include, ‘How many male partners have you had sex with after diagnosis?’, ‘How many of them are regular (or casual) partners?’ and ‘How often have you used condoms with your regular male partners during the last 3 months?’

Process evaluation

In addition, process evaluation indicators are also measured. The main components in process evaluation are contamination (e.g., ‘Have you subscribed to other information resources that provide HIV- or ART-related information?’ and ‘Have you ever consulted the research team’s WeChat account?’) and the usage of mHealth intervention (e.g., ‘Have you ever initiated online communication with case managers/read delivered articles/searched for articles of interest within the WeChat platform?’, ‘What topics have you consulted your case managers about?’, and ‘How frequently do you use this WeChat platform?).

Participants will also be asked to provide an overall comment about the extent to which the platform helps in their daily life. Qualitative interviews will be conducted with the four nurse case managers to investigate the feasibility and acceptability of design and implementation of the intervention. Objective process evaluation indicators will be retrieved from the intervention APP, including the number of times each educational article has been read, the number of enrolled patients for each case managers, the numbers of days when case managers had conversation with patients, and the total numbers of messages sent by case managers. In addition, with the consent of participants, the frequency of key words emerging in the conservation will be counted, to identify the top priority of patients during the initial ART phase.

Sample size estimation

We will calculate the size of effect that the sample size in our study can detect with 80% power (two-tailed alpha 0.05) with regard to the primary outcome. Based on previous studies conducted among Chinese MSM that newly initiate ART, we estimate that the adherent rate at month six in the control group will be 89% [14]. The target sample size of 600 (300 in each arm) will be powered to detect a smallest between-group difference of 9% in adherent rates, allowing for a 20% attrition rate at month six.

Data storage and management

All paper documents, including informed consents, service contracts and notebooks of enrolment information will be stored in a locker in the principal investigator’s office at Sun Yat-sen University. Digital data will be retrieved after the study period, stored on a password-protected computer, and deleted permanently from the ‘Trusted Doctor’ server. To preserve confidentiality, all direct personal identifiers will be removed and replaced with a RID in the data files, which will also be protected with a password. Only persons who are part of the research team will have access to the data. Nurses’ APP accounts and investigators’ WeChat accounts are password-protected for exclusive use in this study. No personal information will be documented on the investigator’s WeChat contact; each participant who friends an investigator will be given their RID as a WeChat alias.

Statistical analysis

Baseline comparison

The equivalence between the baseline characteristics of the two arms will be analysed by calculating the standard difference (SDiff). A value > 0.1 will reveal imbalance.

Effectiveness of intervention

The primary outcomes will be the prevalence of ART adherence and CD4 counts at month six. For the between-group comparisons, χ2 test will be used for binary outcomes (e.g., ART adherence, mental health disorders, sexual risk behaviours) and t-test for continuous outcomes (e.g., CD4 count, quality of life, illness representation, IMB constructs, and HIV viral load). Within-subject analyses will also be conducted, comparing the baseline response to follow-up session responses.

Intervention effects will be performed on the principle of intention-to-treat (ITT), using the data collected from all randomized participants in the analysis. Missing endpoints will be imputed by Multiple-Imputation approach. The General estimating equation (GEE) will be used for repeated measures analyses using SAS 9.0, which allows for the inclusion of all subjects in the analysis regardless of their number of visits. As GEEs require data to be randomly missing, the analysis will be repeated using a subset of participants with complete data. A sensitivity analysis will be conducted to determine the impact of dropout and to evaluate additional longer timeframes (month 12).

Intervention mechanism analysis

It has been hypothesized that ART adherence information, motivations, and skills will be improved by the intervention, and will function as mediators of the intervention effect on improving ART adherence and psychosocial health outcomes. Structural equation modelling (SEM) will be adopted to test the applicability of the hypotheses using AMOS 17.0.

Discussion

To our knowledge, this study is the first to explore the efficacy of mHealth intervention in the case management services targeted at HIV-infected MSM in low-and middle-income countries that incorporate timely online communication between case managers and patients, comprehensive educational articles, supportive service information and hospital-visit reminders.

As there is an unprecedented number of patients on ART after the country-wide scale-up of this therapy-provision, the Chinese CDC issued the ‘Exploration of HIV case management service model’ initiative, requiring case managers to deliver exclusive follow-up services for PLWHA in designated hospitals [58]. Research on the case management of HIV patients is still at an early stage in China, but some evidence already suggests that case management leads to better adherence to treatment and quality of life, and to less risky sexual behaviour [59, 60]. However, Chinese case managers continue to be burdened with exceedingly heavy workloads and restrained by limited policy and financial support [61, 62].

In this scenario, APP-based mHealth interventions have the potential to improve service delivery and positively affect patient outcomes [23]. Smartphones are more portable than computers, and together with their accessibility and intensive daily use by many people, they can enable point-to-point individualised interventions and help reduce some societal and structural barriers faced by stigmatised populations [28]. This study protocol provides a clear and thorough description of an APP-based mHealth intervention and will be used to investigate the effectiveness of mHealth intervention and underlying mechanisms. Findings in this study will provide new insights into HIV-infected MSM patients’ daily struggles of ART adherence and will also suggest how for mHealth services may be delivered in a cost-effective manner at a clinic and national level. There are several strengths of this study. First, the intervention is theory-driven and based on an adherence-targeting framework, IMB theory, which is a widely-adopted theory in mHealth intervention development [26]. Second, extensive elicitation research was conducted and multi-dimensional intervention components were integrated accordingly to tailor health services. Third, this study covers a wide range of measurements, including adherence, the biological parameters of HIV disease-progress (i.e., CD4 count and HIV RNA), mental health outcomes, and quality of life. Fourth, the linkage of an intervention APP and the WeChat platform means that the patients can use this highly accepted platform for daily communication and receiving intervention services, rather than having to download a new ‘ART-treatment’ APP, which patients may be reluctant to do. Fifth, adequate training and supervision is provided to the nurses, including the development of SOP and programme manuals. Moreover, the user experience of participants and the feedback of case managers and the intervention group as well as objective process evaluation indicators will be recorded and analysed, to investigate the feasibility and applicability of the intervention in the future.

There are a few study limitations worth noting. Most importantly, the open study-design means that we cannot mask the intervention, thus participants will be aware that they are part of an ART adherence intervention, which might introduce bias. Although self-reported adherence to ART has notable defects, we have adopted a composite and sensitive measurement scale, using CD4 counts and HIV-RNA levels as objective indicators of efficacy. In addition, we will not be able to control external factors that may confound study results, although we will document some potential confounders (e.g., health information received from other sources or other health programmes occurring in the intervention period) to help with the interpretation of findings.

This trial is among the first efforts to innovatively develop and test the smartphone APP-based case management mHealth intervention in China. Once proven effective, the innovative mHealth service could be integrated into the routine case management of PLWH. This mHealth intervention could also tailored to the patient management service for other chronic conditions. The trial is currently ongoing and the intervention effects will be reported after the data collection.

Availability of data and materials

The datasets during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

AIDS:

Acquired immune deficiency syndrome

APP:

Application

ART:

Antiretroviral treatment

CD4:

Cluster of differentiation 4

CDC:

Centre for disease control and prevention

CNY:

Chinese Yuan

GAD:

Generalized anxiety disorder

GEE:

General estimating equation

HIV:

Human immunodeficiency virus

IMB:

Information-motivation-behavioural skills

IPQ:

Illness perception questionnaire

ITT:

Intention-to-treat

LGBT:

Lesbian gay bisexual transgender

LW-IMB-AAQ:

Life windows information-motivation- behavioural skills art adherence questionnaire

mHealth:

mobile health

MSM:

Men who have sex with men

NGO:

Non-governmental organization

PHQ:

Patient Health Questionnaire

PLWHA:

People living with HIV/AIDS

PPI:

Patient and public involvement

RID:

Research identification

SDiff:

Standard difference

SEM:

Structural equation modelling

SOC:

standard-of-care

SOP:

Standard operating procedure

SPIRIT:

Standard protocol items: recommendations for interventional trials

STD:

Sexually transmitted diseases

TasP:

Treatment as prevention

WHOQOL:

World Health Organization quality of life scale

References

  1. Feng Y, Wu Z, Detels R, Qin G, Liu L, Wang X, Wang J, Zhang L. HIV/STD prevalence among men who have sex with men in Chengdu, China and associated risk factors for HIV infection. J Acquir Immune Defic Syndr. 2010;53(Suppl 1):S74–80.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Sun YM, Sun WD, Lu HY, Xin RL, He SF, Zhang Q, Yue H, Fan XG, Ma XY. Surveillance for HIV infection in MSM selected through respondent driven sampling in Beijing, 2005-2012. Zhonghua Liu Xing Bing Xue Za Zhi. 2016;37(10):1383–91.

    CAS  PubMed  Google Scholar 

  3. Wong NS, Mao J, Cheng W, Tang W, Cohen MS, Tucker JD, Xu H. HIV linkage to care and retention in care rate among MSM in Guangzhou, China. AIDS Behav. 2018;22(3):701–10.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Huang Q, Li Q, Li Y, Zeng G, Cui X, Yan P, Lyu F, Luan R. Prevalence of HIV infection and syphilis, sexual behaviors and awareness of HIV/AIDS related knowledge among men who have sex with men in China: a meta-analysis of data collected from 2010 to 2013. Zhonghua Liu Xing Bing Xue Za Zhi. 2015;36(11):1297–304.

    PubMed  Google Scholar 

  5. NCC N. Update on the AIDS/STD epidemic in China in December 2017. Chin J AIDS STD. 2018;24:111.

    Google Scholar 

  6. Aids, Hepatitis C Professional Group SoIDCMA, Chinese Center for Disease C, Prevention. Chinese guidelines for diagnosis and treatment of HIV/AIDS (2018). Zhonghua Nei Ke Za Zhi. 2018;57(12):867–84.

    Google Scholar 

  7. Wesevich A, Mtande T, Saidi F, Cromwell E, Tweya H, Hosseinipour MC, Hoffman I, Miller WC, Rosenberg NE. Role of male partner involvement in ART retention and adherence in Malawi's option B+ program. AIDS Care. 2017;29(11):1417–25.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Fatch R, Emenyonu NI, Muyindike W, Kekibiina A, Woolf-King S, Hahn JA. Alcohol interactive toxicity beliefs and ART non-adherence among HIV-infected current drinkers in Mbarara, Uganda. AIDS Behav. 2017;21(7):1812–24.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Wolff MJ, Giganti MJ, Cortes CP, Cahn P, Grinsztejn B, Pape JW, Padgett D, Sierra-Madero J, Gotuzzo E, Duda SN, et al. A decade of HAART in Latin America: long term outcomes among the first wave of HIV patients to receive combination therapy. PLoS One. 2017;12(6):e0179769.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  10. Thomas R, Burger R, Harper A, Kanema S, Mwenge L, Vanqa N, Bell-Mandla N, Smith PC, Floyd S, Bock P, et al. Differences in health-related quality of life between HIV-positive and HIV-negative people in Zambia and South Africa: a cross-sectional baseline survey of the HPTN 071 (PopART) trial. Lancet Glob Health. 2017;5(11):e1133–41.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, Hakim JG, Kumwenda J, Grinsztejn B, Pilotto JH, et al. Prevention of HIV-1 infection with early antiretroviral therapy. N Engl J Med. 2011;365(6):493–505.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Huan Z, Fuzhi W, Lu L, Min Z, Xingzhi C, Shiyang J. Comparisons of adherence to antiretroviral therapy in a high-risk population in China: a systematic review and meta-analysis. PLoS One. 2016;11(1):e0146659.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  13. Mills EJ, Nachega JB, Bangsberg DR, Singh S, Rachlis B, Wu P, Wilson K, Buchan I, Gill CJ, Cooper C. Adherence to HAART: a systematic review of developed and developing nation patient-reported barriers and facilitators. PLoS Med. 2006;3(11):e438.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Tao J, Qian HZ, Kipp AM, Ruan Y, Shepherd BE, Amico KR, Shao Y, Lu H, Vermund SH. Effects of depression and anxiety on antiretroviral therapy adherence among newly diagnosed HIV-infected Chinese MSM. AIDS. 2017;31(3):401–6.

    CAS  PubMed  Google Scholar 

  15. Chesney MA. Factors affecting adherence to antiretroviral therapy. Clin Infect Dis. 2000;30(Suppl 2):S171–6.

    Article  PubMed  Google Scholar 

  16. Yang JP, Simoni JM, Dorsey S, Lin Z, Sun M, Bao M, Lu H. Reducing distress and promoting resilience: a preliminary trial of a CBT skills intervention among recently HIV-diagnosed MSM in China. AIDS Care. 2018;30(sup5):S39–48.

    Article  PubMed  Google Scholar 

  17. Grube B, Chernesky RH. HIV/AIDS case management tasks and activities: the results of a functional analysis study. Soc Work Health Care. 2001;32(3):41–63.

    Article  CAS  PubMed  Google Scholar 

  18. Wilson MG, Husbands W, Makoroka L, Rueda S, Greenspan NR, Eady A, Dolan LA, Kennedy R, Cattaneo J, Rourke S. Counselling, case management and health promotion for people living with HIV/AIDS: an overview of systematic reviews. AIDS Behav. 2013;17(5):1612–25.

    Article  PubMed  Google Scholar 

  19. Sarfo B, Vanderpuye NA, Addison A, Nyasulu P. HIV case management support service is associated with improved CD4 counts of patients receiving Care at the Antiretroviral Clinic of Pantang hospital, Ghana. AIDS Res Treat. 2017;2017:4697473.

    PubMed  PubMed Central  Google Scholar 

  20. Husbands W, Browne G, Caswell J, Buck K, Braybrook D, Roberts J, Gafni A, Taylor A. Case management community care for people living with HIV/AIDS (PLHAs). AIDS Care. 2007;19(8):1065–72.

    Article  CAS  PubMed  Google Scholar 

  21. Brennan-Ing M, Seidel L, Rodgers L, Ernst J, Wirth D, Tietz D, Morretti A, Karpiak SE. The impact of comprehensive case management on HIV client outcomes. PLoS One. 2016;11(2):e0148865.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  22. Gasiorowicz M, Llanas MR, DiFranceisco W, Benotsch EG, Brondino MJ, Catz SL, Hoxie NJ, Reiser WJ, Vergeront JM. Reductions in transmission risk behaviors in HIV-positive clients receiving prevention case management services: findings from a community demonstration project. AIDS Educ Prev. 2005;17(1 Suppl A):40–52.

    Article  PubMed  Google Scholar 

  23. Hamine S, Gerth-Guyette E, Faulx D, Green BB, Ginsburg AS. Impact of mHealth chronic disease management on treatment adherence and patient outcomes: a systematic review. J Med Internet Res. 2015;17(2):e52.

    Article  PubMed  PubMed Central  Google Scholar 

  24. mHealth: New horizons for health through mobile technologies [https://www.who.int/goe/publications/goe_mhealth_web.pdf]. Accessed 21 Nov 2019.

  25. Nehl EJ, He N, Wang X, Lin L, Wong FY, Yu F. Feasibility and willingness of using e-technologies for HIV prevention and research targeting Chinese MSM. AIDS Care. 2013;25(7):926–30.

    Article  PubMed  Google Scholar 

  26. Muessig KE, LeGrand S, Horvath KJ, Bauermeister JA, Hightow-Weidman LB. Recent mobile health interventions to support medication adherence among HIV-positive MSM. Curr Opin HIV AIDS. 2017;12(5):432–41.

    Article  PubMed  PubMed Central  Google Scholar 

  27. Catalani C, Philbrick W, Fraser H, Mechael P, Israelski DM. mHealth for HIV Treatment & Prevention: a systematic review of the literature. Open AIDS J. 2013;7:17–41.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Daher J, Vijh R, Linthwaite B, Dave S, Kim J, Dheda K, Peter T, Pai NP. Do digital innovations for HIV and sexually transmitted infections work? Results from a systematic review (1996-2017). BMJ Open. 2017;7(11):e017604.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Guo Y, Xu Z, Qiao J, Hong YA, Zhang H, Zeng C, Cai W, Li L, Liu C. Development and feasibility testing of an mHealth (text message and WeChat) intervention to improve the medication adherence and quality of life of people living with HIV in China: pilot randomized controlled trial. JMIR Mhealth Uhealth. 2018;6(9):e10274.

    Article  PubMed  PubMed Central  Google Scholar 

  30. Michie SJM, Francis J, Hardeman W, Eccle M. From theory to intervention: mapping theoretically derived behaviorual determinants to behaviour change techniques. Appl Psychol. 2008;57:660–80.

    Article  Google Scholar 

  31. Fisher JD, Fisher WA, Amico KR, Harman JJ. An information-motivation-behavioral skills model of adherence to antiretroviral therapy. Health Psychol. 2006;25(4):462–73.

    Article  PubMed  Google Scholar 

  32. Starks TJ, Millar BM, Lassiter JM, Parsons JT. Preintervention profiles of information, motivational, and behavioral self-efficacy for methamphetamine use and HIV medication adherence among gay and bisexual men. AIDS Patient Care STDs. 2017;31(2):78–86.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Hawkins A, Evangeli M, Sturgeon K, Le Prevost M, Judd A, Committee AS. Episodic medication adherence in adolescents and young adults with perinatally acquired HIV: a within-participants approach. AIDS Care. 2016;28(Suppl 1):68–75.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Zarani F, Besharat MA, Sarami G, Sadeghian S. An information-motivation-behavioral skills (IMB) model-based intervention for CABG patients. Int J Behav Med. 2012;19(4):543–9.

    Article  PubMed  Google Scholar 

  35. Wells AA, Shon EJ, McGowan K, James A. Perspectives of low-income African-American women non-adherent to mammography screening: the importance of information, behavioral skills, and motivation. J Cancer Educ. 2017;32(2):328–34.

    Article  PubMed  Google Scholar 

  36. Konkle-Parker DJ, Amico KR, McKinney VE. Effects of an intervention addressing information, motivation, and behavioral skills on HIV care adherence in a southern clinic cohort. AIDS Care. 2014;26(6):674–83.

    Article  PubMed  Google Scholar 

  37. Christopoulos KA, Riley ED, Tulsky J, Carrico AW, Moskowitz JT, Wilson L, Coffin LS, Falahati V, Akerley J, Hilton JF. A text messaging intervention to improve retention in care and virologic suppression in a U.S. urban safety-net HIV clinic: study protocol for the Connect4Care (C4C) randomized controlled trial. BMC Infect Dis. 2014;14:718.

    Article  PubMed  PubMed Central  Google Scholar 

  38. Smith LR, Amico KR, Fisher JD, Cunningham CO. 60 minutes for health: examining the feasibility and acceptability of a low-resource behavioral intervention designed to promote retention in HIV care. AIDS Care. 2018;30(2):255–65.

    Article  PubMed  Google Scholar 

  39. Erguera XA, Johnson MO, Neilands TB, Ruel T, Berrean B, Thomas S, Saberi P. WYZ: a pilot study protocol for designing and developing a mobile health application for engagement in HIV care and medication adherence in youth and young adults living with HIV. BMJ Open. 2019;9(5):e030473.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Fisher JD, Amico KR, Fisher WA, Harman JJ. The information-motivation-behavioral skills model of antiretroviral adherence and its applications. Curr HIV/AIDS Rep. 2008;5(4):193–203.

    Article  PubMed  Google Scholar 

  41. Zhang X, Wen D, Liang J, Lei J. How the public uses social media wechat to obtain health information in China: a survey study. BMC Med Inform Decis Mak. 2017;17(Suppl 2):66.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Cheng W, Xu H, Zhong F, Pan S, Tucker JD, Weir S, Zhao J, Tang W. Can HIV service data be used for surveillance purposes?: a case study in Guangzhou, China. BMC Public Health. 2018;18(1):1268.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Fang JLW. An empirical study on the factors influencing the popularity of WeChat public account information dissemination. J Intell. 2016;35(02):157–62.

    Google Scholar 

  44. Ji HS, Cai DN. Analysis of factors affecting information transmission of WeChat Public Number. J Jimei University. 2016;19(4):107–15.

    Google Scholar 

  45. Berg KM, Arnsten JH. Practical and conceptual challenges in measuring antiretroviral adherence. J Acquir Immune Defic Syndr. 2006;43(Suppl 1):S79–87.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Phillips T, Brittain K, Mellins CA, Zerbe A, Remien RH, Abrams EJ, Myer L, Wilson IB. A self-reported adherence measure to screen for elevated HIV viral load in pregnant and postpartum women on antiretroviral therapy. AIDS Behav. 2017;21(2):450–61.

    Article  PubMed  PubMed Central  Google Scholar 

  47. Wilson IB, Fowler FJ Jr, Cosenza CA, Michaud J, Bentkover J, Rana A, Kogelman L, Rogers WH. Cognitive and field testing of a new set of medication adherence self-report items for HIV care. AIDS Behav. 2014;18(12):2349–58.

    Article  PubMed  Google Scholar 

  48. Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–7.

    Article  PubMed  Google Scholar 

  49. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. 2001;16(9):606–13.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Marc LG, Henderson WR, Desrosiers A, Testa MA, Jean SE, Akom EE. Reliability and validity of the Haitian creole PHQ-9. J Gen Intern Med. 2014;29(12):1679–86.

    Article  PubMed  PubMed Central  Google Scholar 

  51. Whittle HJ, Sheira LA, Wolfe WR, Frongillo EA, Palar K, Merenstein D, Wilson TE, Adedimeji A, Weber KM, Adimora AA, et al. Food insecurity is associated with anxiety, stress, and symptoms of posttraumatic stress disorder in a cohort of women with or at risk of HIV in the United States. J Nutr. 2019;149(8):1393–403.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Marcellin F, Preau M, Ravaux I, Dellamonica P, Spire B, Carrieri MP. Self-reported fatigue and depressive symptoms as main indicators of the quality of life (QOL) of patients living with HIV and hepatitis C: implications for clinical management and future research. HIV Clin Trials. 2007;8(5):320–7.

    Article  PubMed  Google Scholar 

  53. Hsiung PC, Fang CT, Wu CH, Sheng WH, Chen SC, Wang JD, Yao G. Validation of the WHOQOL-HIV BREF among HIV-infected patients in Taiwan. AIDS Care. 2011;23(8):1035–42.

    Article  PubMed  Google Scholar 

  54. Broadbent E, Petrie KJ, Main J, Weinman J. The brief illness perception questionnaire. J Psychosom Res. 2006;60(6):631–7.

    Article  PubMed  Google Scholar 

  55. Robbins GK, Cohn SE, Harrison LJ, Smeaton L, Moran L, Rusin D, Dehlinger M, Flynn T, Lammert S, Wu AW, et al. Characteristics associated with virologic failure in high-risk HIV-positive participants with prior failure: a post hoc analysis of ACTG 5251. HIV Clin Trials. 2016;17(4):165–72.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  56. Fisher JD, Amico KR, Fisher WA, Cornman DH, Shuper PA, Trayling C, Redding C, Barta W, Lemieux AF, Altice FL, et al. Computer-based intervention in HIV clinical care setting improves antiretroviral adherence: the LifeWindows project. AIDS Behav. 2011;15(8):1635–46.

    Article  PubMed  Google Scholar 

  57. Santillan Torres Torija C, Villagran Vazquez G, Robles Montijo SS, de Lourdes Eguiluz Romo L. The Information and Motivation and Behavioral Skills Model of ART Adherence among HIV-Positive Adults in Mexico. J Int Assoc Provid AIDS Care. 2015;14(4):335–42.

    Article  PubMed  Google Scholar 

  58. LU Guilan FC, Kong F, Bai C. The effects of case management on the compliance behaviors of MSM with HIV/AIDS. Adv Nurs Pract. 2017;17(1):133–6.

    Google Scholar 

  59. Yang Li WJ, Renfang NI, Xiulan W, Yanrong W, Weiping C. The measures and effects evaluation of case management in HIV-infected women during perinatal period. Adv Nurs Pract. 2016;16(3):419–21.

    Google Scholar 

  60. Wen Yan-yu LL, Xian-yu Y, Jing-qiu F, Shan-shan Z. Influence of case intervention on behaviors of homosexual behavior patients with HIV infection. Chin J Nosocomiol. 2018;28(22):3410–3.

    Google Scholar 

  61. Tian Bo LJ, Kun-li W, Rong H, Chong-xi L, Jing-song B. Exploration of case management mode in antiretroviral therapy. J Dermatol Venereol. 2018;40(2):171–3.

    Google Scholar 

  62. Kong F. WH-q: Research progress for nursing interventions for HIV/AIDS patients. Chinese Gen Pract Nurs. 2018;16(1):21–4.

    Google Scholar 

Download references

Acknowledgments

The authors would like to thank all the field investigators, case managers and patients that participate in this study.

Funding

This work was funded by the National Natural Science Foundation of China (Grant #71774178); Science and Technology Planning Project of Guangdong (Grant #2017A020212006); the Guangzhou Science and Technology Project (Grant #201607010368); and National Major Science and Technology Projects of China: Infectious Disease Prevention and Control (Grant # 2018ZX10715004).

Author information

Authors and Affiliations

Authors

Contributions

FX wrote the original draft; SR made major revisions. LC and ZH recruited participants and conducted survey; JTFL, HC, LJ, LL and GJ conceptualized the study; LL supervised the investigation; HY and GJ acquired funding. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Jing Gu.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the Ethics Committees of Sun Yat-sen University (Approval number: #003 in 2017 Committee of School of Public Health, Sun Yat-sen University). Written informed consent was obtained from the participants before the baseline assessment.

Consent for publication

Not applicable.

Competing interests

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fan, X., She, R., Liu, C. et al. Evaluation of smartphone APP-based case-management services among antiretroviral treatment-naïve HIV-positive men who have sex with men: a randomized controlled trial protocol. BMC Public Health 20, 85 (2020). https://doi.org/10.1186/s12889-020-8171-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-020-8171-5

Keywords