Translation, Adaptation, and Psychometric Testing of the Almere Technology Acceptance Questionnaire (ATAQ) Among Older Adults in China

Introduction

The care needs of elderly people are currently not met owing to the shortage of care resources and rapid increase in the aging population worldwide. Social Assistant Robots (SARs) play an important role in promoting intelligent assisted aging and alleviating aging problems.¹ SARs can sense, process the sensory information, and perform activities that benefit the elderly and people with disabilities.² This helps the elderly people to live an independent life at their homes. In this context, there is growing interest in investigating attitudes toward robots and their acceptance by older adults.³ The application and benefits of SARs can only be achieved if people accept, embrace, and use SARs. A previous study showed that up to one-third of all assistive technologies are abandoned within one year of use.⁴ This poor acceptance of social assistant robots will limit application of robots in the elderly care sector. Therefore, acceptance of SARs is a significant challenge faced by designers and health workers. This calls for investigations into strategies to improve acceptance of SARs to accelerate robot design and application.

Several studies have been conducted in the last decade to explore technology acceptance of SARs, focusing on the use of self-report questionnaires to measure target user acceptance of robot technology.^5–7 For example, the Godspeed Questionnaire is mainly used to evaluate the perceptions of anthropomorphism, animacy, likability, intelligence, and safety of robots.⁸ The Negative Attitudes Toward Robots Scale (NARS) was developed to determine general attitudes of users toward robots before conducting any robot interaction studies with these individuals.⁹ However, these survey tools mainly comprise single or few risk factors. The Godspeed questionnaire mainly focuses on anthropomorphic degrees whereas NARS focuses on negative attitudes towards robots. Consequently, it is often difficult to comprehensively explore factors that may influence technology acceptance using these measurement tools.

The Almere Technology Acceptance Questionnaire (ATAQ) was developed for evaluation of acceptance and attitudes of elderly people towards SARs.¹⁰ The tool is based on the Unified Theory of Acceptance and Use of Technology (UTAUT) model.¹¹ The questionnaire comprises influencing factors, including social influence, attitudes, perceived usefulness, perceived ease of use, perceived entertainment, trust, perceived adaptability, anxiety, social presence, and perceived sociality. ATAQ has several advantages over other methods. It was designed to for assessment of the perceptions of SARs by users during human-robot interaction, especially among elderly people. In addition, the ATAQ uses a simple quantitative measure (a Likert-type questionnaire with 41 items is used). In addition, the questionnaire has good reliability and validity. ATAQ has been widely validated using a sample of older adults in many countries, including Canada,⁷ Poland,¹² and Switzerland.¹³ The widespread use of the questionnaire further indicates its validity.

Studies have not fully explored SARs in elderly care in China. In addition, only few studies have evaluated technology acceptance and measurement tools among the elderly. ATAQ is a questionnaire based on classical theory and practice. It was designed specifically for the elderly population, and is used for evaluating technology acceptance of SARs. However, ATAQ is only available in English language and studies have not explored detailed translation procedures. Therefore, the current study sought to translate the English version of ATAQ to Mandarin. Further, Context Relevance Evaluation was conducted to explore the cultural relevance of the scale and test the reliability of the Mandarin version among the elderly population in China.

Methods

Study Design

The study comprised 2 phases as follows: (1) context relevance evaluation, instrument translation, and cultural adaption from English to Mandarin Chinese; (2) a quantitative cross-sectional survey.

Context Relevance Evaluation and Instrument Translation

Context Relevance Evaluation

Context Relevance Index (CRI) is designed to evaluate differences between the context in which an instrument was originally developed and the target context in terms of behavior, preference, religion, clinical regimen, and the healthcare system. The parameters are rated using a four-point Likert scale where 1 represents “totally irrelevant” and 4 represents “totally relevant”.¹⁴ A panel comprising 6 specialists assessed the contextual relevance of ATAQ in April 2021. The six experts had experience ranging from 10 to 36 years and have excellent English language proficiency. Out of the 6 experts, 4 experts had experience in geriatric and community studies, 1 in elderly services research, and 1 in robotics management. The first author presented to the experts each aspect of the ATAQ in English through a PowerPoint presentation. All experts were then asked to give feedback using the CRI.

Translation of the Tool

The English version of the tool was translated into Mandarin Chinese based on Brislin’s (1986) adapted translation model through the following three steps.¹⁵

Step 1: Forward translation. Two Chinese-native translators, proficient in Mandarin Chinese and English, completed the forward translation of the instrument evaluation independently. One of the translators had a Ph.D. degree in nursing with CET-6 qualification, and the other had a Master’s degree in English with 3 years of experience in English to Chinese translation. The two translated ATAQ (ATAQ-Forward Translation A and ATAQ-Forward Translation B) were compared and synthesized to obtain a single forward-translated ATAQ (Chinese version) denoted the 1st C-ATAQ.

Step 2: Back translation. The 1st C-ATAQ was back translated to English by two bilingual researchers working independently. The researchers were fluent in Mandarin Chinese and English and were fully blinded to the original version. Translator 1 involved in the back translation was a master’s student, whereas translator 2 was a doctoral student. Back Translation C and Back Translation D were obtained after this translation, respectively. The back-translated versions were combined into one (B-ATAQ) and compared with the authors’ English version and the versions from the four translators. The B-ATAQ was reviewed by the original author to verify its equivalence to the original version. Further, the 2nd C-ATAQ was obtained.

Step 3: Evaluation of translation equivalence. Specialists who had the theoretical knowledge and practical experience on SARS were invited to modify the items of the 2nd C-ATAQ. Validity of the questionnaire content, including its suitability for Chinese culture and precision of the translation, was evaluated. The items of the 2nd ATAQ were scored separately using the Translation Validity Index (TVI). TVI was adapted from the Content Validity Index based on a four-point Likert scale where 1 represented “totally different” and 4 represented “equivalent”.¹⁶

Cultural discrepancies were resolved through discussion between the specialists. For example, entry 3 “I find the robot scary” and Entries 4 “I find the robot intimidating” only varied slightly from each other. Therefore, the experts recommended that entry 4 should be retained, and entry 3 removed. The experts recommended that the formulation in entries 8–10 “during the next few days”, was not in line with the national context. The formulation was amended to read “in the future for a period of time”. In addition, entry 10 “I need to use a robot”, was added in the “Intention to use” dimension, based on the expert opinion. The 3rd C-ATAQ was obtained from these changes.

Pre-analysis of the questionnaire was conducted in June 2021 in a community health service center in Changsha city of Chinas. A total of 30 elderly participants were recruited. Modifications were made, to clarify ambiguous or unclear questions and vague terms. For example, in entry 1 “If I should use the robot, I would be afraid to make mistakes with it”, older adults did not to understand the phrase “make mistakes”. This entry was changed to “make operation error”. The investigator’s team discussed items, and a final version was obtained. The instrument translation procedure was completed successfully.

Testing of Psychometric Properties

A cross-sectional survey design was conducted to test the psychometric properties of the ATAQ from June to September 2021. Terms analysis, content validity, construct validity, internal consistency, and test-retest reliability were explored in this phase.

Setting and Sampling

Convenience sampling was employed to recruit interviewees. A total of 317 elderly in 2 community health service centers in Changsha city were selected from July to September 2021 to participate in the study. The inclusion criteria were as follows: older adults (aged ≥60 years); subjects living in urban communities (A resident in the community for >1 year); persons that accurately understood the content of the questionnaires. Participants with a diagnosis of dementia or any cognitive impairment and sensory impairment symptoms in electronic health records in the community health service center were excluded.

Data Collection

Personal information questionnaire and the Chinese version of ATAQ were used to collect participant data. A staff or family doctor contacted the patients in the community health service center through phone and arranged an appointment with the interviewing team. A door-to-door survey was conducted by uniformly trained and qualified investigators. The investigators explained the purpose of the study to all patients and written consent was obtained prior to participation. Those who required assistance to fill the survey form were helped on the spot.

A total of 328 questionnaires were distributed, among which 328 were returned. Questionnaires with unanswered or wrongly filled answers were defined as invalid questionnaires and were excluded. After exclusion of invalid questionnaires, 317 valid questionnaires were selected with a recovery rate of 96.6%. The test-retest reliability was administered to 30 randomized participants 2 weeks after the survey.

Measures

The Personal Information Questionnaire included parameters such as age, gender, education level and occupation of individual respondents, marital status, type of dwelling, use of mobility devices, living situation (current), as well as frequency of children’s visits.

Acceptance and attitudes of older people towards SARs were evaluated using the Chinese version of Almere Technology Acceptance Questionnaire. This is a 30-item self-reported questionnaire. Items were scored using a five-point Likert scale where “1” represented “never”, and “5” indicated “always.” Higher scores indicated higher levels of the respective dimension. Studies report that the original ATAQ has excellent internal consistency and construct validity (Cronbach’s alpha ranged from 0.707–0.948, GFI=0.96, c2/df=1.99).⁸

Statistical Analysis

Statistical analyses were performed using Excel 2010, SPSS 24.0, and AMOS 26.0. Quantitative variables were presented as means and standard deviations (SD). Qualitative data were presented as frequencies and percentages.

Item Analysis: Mean and standard deviation (SD) of every item in the questionnaire were presented. Critical Ratios (CR) were used to explore the differences between high and low scores in each item.

Reliability: Reliability of the questionnaire was assessed by estimating internal consistency (the degree of homogeneity of the 2 items forming the scale) and by evaluating test-retest reliability (the degree of stability of the score when there were no changes in health status).¹⁷ Internal consistency was evaluated using the coefficient Cronbach’s alpha, with an alpha of 0.70 indicating acceptable reliability and 0.80 or higher indicating good reliability.¹⁸ Test-retest reliability of the scale was estimated using intraclass correlation coefficient (ICC, ICC> 0.85 was acceptable).¹⁹

Validity Evaluation: Validity evaluates the accuracy of an instrument.²⁰ The content validity index for items (I-CVI) and the average content validity index for scale (S-CVI/Ave) were calculated. Each question used a 4-point Likert-type scoring method, ranging from 1 (inconsistent) to 4 (consistent). Content validity of the scale was considered good if I-CVI was ≥0.78 and S-CVI/AVE ≥0.9.¹⁶

Factor Analysis: The factor structure of ATAQ was evaluated through Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA). A random number algorithm was used to divide the sample into two groups. EFA was carried out on the first group (n=105) and CFA (n=212) was carried out on the second group. Kaiser-Meyer-Olkin (KMO) was calculated before EFA to ensure that the sample size was adequate. KMO values between 0 and 0.49 showed unacceptable sample size, 0.5–0.7 indicated moderate; 0.7–0.8 are good, 0.8–0.9 indicated great sample size, and > 0.9 indicated excellent sample size.²¹ EFA was performed through principal component analysis and orthogonal rotation using the Varimax method (factor loading >0.40, eigenvalue >1.0, cumulative contribution rate >40%.²² CFA was conducted using maximum likelihood procedure to evaluate whether the ATAQ model’s fit for the data. The model’s fit was assessed using the chi-square/degree of freedom ratio (CMIN/DF), goodness of fit index (GFI), Root Mean Square Error of Approximation (RMSEA), Root Mean-square Residual (RMR), Tucker-Lewis Index (TLI), Comparative Fit Index (CFI), and Incremental Fit Index (IFI). The model was considered to be a good fit if the (x2/df) <3, RMSEA ≤ 0.08, RMR <0.05, GFI >0.90, CFI>0.90, IFI>0.90, and TLI>0.90.²³

Results

Context Relevance and Translation Equivalence Evaluation

All items used for the context relevance evaluation were rated “3 = relevant” or “4 = totally relevant” by all experts.

A good consensus was obtained on translation equivalence. The findings showed that 100% items were rated 3 or 4 by all experts.

Sociodemographic Characteristics

A total of 317 elderly residents were included in the study. The participants’ mean age was 70.3±7.5, with 55.5% of participants being women. More than half of the elderly patients had a junior high school diploma or lower (59.3%), and the vast majority were married (80.1%). The participant live mainly in couple (42.6%) or with children (46.7%). For most patients, their children visited them on nearly every day (57.10%). Half of the elderly have a pension of between 1000 and 4000 RMB. Concerning occupational status, most were workers (31.20%), and most of them were conversant with mobile service devices (73.80%). The vast majority of the elderly residents have a good health status. The detailed results are shown in Table 1.

Table 1 Sociodemographic Characteristics of Participants (N=317)

Item Analysis

The ATAQ scores were sorted from low to high. The items with scored above 27% were assigned to the high score group whereas items with scores below 27% were assigned to the low score group.

Critical ratios for the three items of the anxiety dimension (1, 2, 3) were −1.188, −1.189, −1.615, respectively. The critical ratio for item 32 of the social presence was 2.767. Entry 32 was removed because it was less discriminating. Difference in critical ratios among the remaining 35 entries was statistically significant (CR>3.000, P<0.001), thus they were used in the subsequent analysis.

Pearson correlation analysis was used to evaluate the correlation between each item and the total score. The correlation coefficients ranged from 0.374 to 0.763. Item 19 (Perceived Ease of Use), and item 36 (Facilitating Conditions) were removed because they had correlation coefficients less than 0.4 (r=0.377, r=0.374; p>0.05). The correlation coefficients of entries ranged from 0.403 to 0.763 after exclusion of two items (P<0.001).

Validity Content Validity Test

Six experts evaluated the content validity of the Chinese version of the ATAQ. Item 29, 30 (Social Influence), and item 37 (Facilitating Conditions) were removed from the tool because the I-CVI values of these items were less than 0.78. The final questionnaire comprised 30 items, with I-CVI ranging from 0.83 to 1.00, and the S-CVI was 0.92.

Factor Analysis

Internal structure of the 30-item ATAQ was evaluated using the EFA. The results showed that the KMO value was 0.852. Further analysis was performed using the Bartlett spherical test (χ²=2177.589, P <0.001), and the findings indicated that the data were suitable for exploratory factor analysis. A total of 9 factors were extracted through exploratory factor analysis and the cumulative variance contribution rate was 77.175%. The factor load of each item in the corresponding dimension was above 0.4.

The final version of the questionnaire comprised 30 questions. The common factors were named with reference to the original questionnaire. Factors 1 to 9 were Attitude, Intention to use, Social Presence, Perceived Sociability, Perceived Enjoyment, Perceived ease of Use, Perceived Adaptivity, Perceived Usefulness and Trust. Factors in each item were slightly different from the ones in the original questionnaire. Notably, item 13 belonged to the dimension of perceived entertainment in the original questionnaire, whereas consultation between the experts and the research group showed that the meaning expressed by item 13 was closer to that of factor 5. Item 20 belonged to the dimension of perceived social ability, whereas the meaning expressed was similar to that of factor 4. Therefore, they were classified as factor 5 and factor 4, respectively. EFA results are presented in Figure 1 and Table 2.

Table 2 Factor Loading of Rotated Factor Analysis of the ATAQ

Figure 1 Rotated factor analysis of the ATAQ, as a basis for dividing factors. (In the scree plot, the point where the slope of the curve is clearly leveling off at component 10 indicates the 9 factors that should be generated by the screen test in the 30 items, N =105).

Further, the 9-factor model solutions were evaluated using subsample 2 comprising 212 patients through CFA. Structural equation modeling (SEM) function in AMOS26.0 software was used for testing the model. The models showed adequate model fit (χ2/df=2.006, RMSEA=0.069, RMR=0.059, GFI=0.816, IFI=0.913, TLI=0.896, CFI=0.912) (Figure 2). The indexes of the nine-factor model indicated that the model had an acceptable fit.

Figure 2 Chinese version of the ATAQ structural models (N=212).

Reliability

Internal Consistency Reliability: The overall internal consistency coefficient (Cronbach’s α) of the ATAQ was 0.945, and the Cronbach’s α of each subscale ranged between 0.664 and 0.891, indicating the model was reliable.

Test-retest reliability: The test-retest reliability coefficient of the questionnaire was 0.980, with the domains ranged from 0.918 to 0.986, and results indicate satisfactory reliability and stability.

Discussion

A 30-item ATAQ was established in the present study. The findings showed that the ATAQ was fairly consistent, reliable, and valid. Cross-cultural validity was ensured through a five-step translation and back-translation process. This is the first study to explore reliability and validity of ATAQ among community-dwelling elderly residents in China. The findings indicated that the Chinese version of ATAQ was a reliable and valid measure for evaluating acceptance of SARs among elderly people.

CRI was used to explore the cultural relevance of the questionnaire and appropriateness in the Chinese cultural context. All the items in the ATAQ version developed in the present study were relevant to the Chinese context of elderly people in China. The results indicate that this CRI of cross-cultural adaptation reduces bias during translation, and provides a basis for further clinical translation of this tool.

The original ATAQ comprises nine dimensions and 30 items. Three dimensions (anxiety, facilitation, and social impact) present in the original version of the questionnaire were absent in the version developed in the present study. This is mainly because the SARs was not familiar to elderly people in China. These results are consistent with findings from previous studies. A study by Yunhua et al²⁴ reported that current living environment of elderly people limits extensive use of SARs. Therefore, studies have not fully explored anxiety in elderly people. In addition, there are few facilities that use robots and social networks for the elderly population do not widely recommend use of robots, because companion robots are not commonly used in the country.²⁵

EFA and CFA were performed to explore the internal structure of ATAQ. This study reports for the first time the number of factors and cumulative variance contribution of the Chinese version of ATAQ. Nine factors were extracted through exploratory factor analysis, and the cumulative variance contribution rate was 77.175%. The results indicated that the new version had a different factor structure compared with the original English version, mainly attributed to removal of the three dimensions from the questionnaire. The final 9 residual factors were similar to the ones in the original questionnaire. CFA also showed that the factor structure of this scale was appropriate. The fit metrics of the model in the present study were better compared with those for the technology acceptance model reported by Chen et al.²⁶

The factors affecting the technological acceptance of SARs among Chinese elders did not significantly change. Therefore, slight adjustments were made. The study findings indicate that the structure of the 9-factor ATAQ was suitable for a realistic assessment of technological acceptance of SARs among elderly people. However, the tool needs further verification using large sample sizes.

Reliability of a measure is an important factor in the model. The results showed that the ATAQ had excellent reliability and high item-total correlations. The results of the current study showed that the Cronbach’s α coefficient of the Chinese ATAQ was 0.945, which was higher compared with value reported by Huang (α=0.907).²⁷ The Cronbach’s α coefficient of all dimensions was 0.664~0.891, slightly lower relative to value for the original English version (α=0.707~0.948).¹⁰ These differences can be attributed to the different studies being conducted in different countries. For example, a study by Louie reported that Cronbach’s α coefficient was 0.200~0.800),⁷ whereas another study reported that the coefficient was 0.622~0.966).²⁸ Differences in results in different studies conducted in different countries can be attributed to the differences in social and cultural structure. The findings of the present study are informative. Cronbach’s alpha analysis indicated good internal consistency of the model. The cross-culture translation may have contributed to the high internal consistency observed, which maintained the conceptual equivalence to the original ATAQ.

Notably, test-retest reliability was not explored for the original ATAQ. The present study is the first to evaluate the test-retest reliability of the ATAQ. The test-retest reliability of ATAQ was evaluated by ICC instead of Pearson correlation analysis, which is commonly used in similar research for testing reliability. The retest reliability value should be more than 0.7 for any developed measurement tool.²³ The retest correlation coefficient of the tool in the current study was 0.980, indicating that the reliability of the questionnaire met the requirements of psychological measurement tools. This value shows good stability of the tool.

In summary, a tool for assessing the technology Acceptance of SARs was developed in the present study. Although subtle differences were observed between the English and Chinese versions, the results show that the ATAQ can be culturally adapted for use in Chinese and other settings to evaluate the technology acceptance of SARs among elderly Chinese people. Moreover, the Chinese version of ATAQ had good reliability and validity. The questionnaire comprised objective, well-defined, and easy-to-understand questions, and the time to complete the questionnaire was about 5~10 min. In addition, ATAQ exhibits several attributes that make it a valid and easy-to-operate assessment tool. Furthermore, the Chinese version of ATAQ can be used to evaluate acceptance of SARs and provides a comprehensive measure of the factors that affect acceptance of SARs.²⁹ Further application of the assessment tool will increase understanding on technology acceptance in the era of artificial intelligence. Notably, use of different language versions allows comparing technology acceptance across different countries, thus providing insight on the cultural variability of the technology acceptance model.

The present study had some limitations. First, the psychometric testing of the ATAQ did not include tests of equivalence based on responses of bilingual people. The population included in the present study did not comprise enough bilingual people; therefore, the original English version of the instrument was not evaluated. Second, the sample size used in the current study was relatively small. In addition, participants only comprised community-dwelling elderly people living in Changsha City. Further studies with a larger sample size comprising participants recruited from multiple institutions should be conducted to validate these findings.

Conclusion

In this study, the ATAQ was translated from English to Chinese. Further, the questionnaire was adapted culturally, and its psychometric properties were verified. The results indicated that the ATAQ had good reliability and validity and can be used as a reliable measurement tool. In addition, the tool can be used to evaluate the technology acceptance of SARs among elderly people in China.