German Version of the mHealth App Usability Questionnaire in a Cohort of Patients With Cancer: Translation and Validation Study

Quifer-Rada P., Aguilar-Camprubí L., Gómez-Sebastià I., Padró-Arocas A., Mena-Tudela D.

Mobile applications (apps) are increasingly used during pregnancy, postpartum and lactation. To ensure the utility and usability of breastfeeding support apps, they need to be evaluated using a reliable scale specific to mHealth apps and breastfeeding users. To translate the original the mHealth App Usability Questionnaire (MAUQ) questionnaire into Spanish and to adapt it to breastfeeding support apps environment. The questionnaire was translated by one high English proficiency translator and was back translated. The items of the questionnaire were modified for Spanish readers and for breastfeeding users. The modified questionnaire was assessed for content validity with a panel of 5 experts and 12 users and the modified kappa statistic was used to determine the interrater agreement among the raters. The reliability of the questionnaire was assessed in a mobile application for breastfeeding support (LactApp) by 202 users. The structure of the questionnaire was validated using exploratory factor analysis. All items of the questionnaire were relevant, clear or comprehensible with content validity index values higher than 0.79. The modified kappa agreement for each item of the modified MAUQ (m-MAUQ) proved an excellent agreement (κ = 0.9–1.0). Factor analysis of the m-MAUQ showed four subscales. The internal consistency of the complete questionnaire was high (Cronbach α = 0.89). The Spanish and modified MAUQ demonstrated high reliability and validity and it might be used to evaluate the usability, utility and acceptability of mHealth apps aiming to support lactating women.

Weber F., Müller C., Bahns C., Kopkow C., Färber F., Gellert P., Otte I., Vollmar H.C., Brannath W., Diederich F., Kloep S., Rothgang H., Dieter V., Krauß I., Kloek C., et. al.

Abstract Introduction Hip and knee osteoarthritis are associated with functional limitations, pain and restrictions in quality of life and the ability to work. Furthermore, with growing prevalence, osteoarthritis is increasingly causing (in)direct costs. Guidelines recommend exercise therapy and education as primary treatment strategies. Available options for treatment based on physical activity promotion and lifestyle change are often insufficiently provided and used. In addition, the quality of current exercise programmes often does not meet the changing care needs of older people with comorbidities and exercise adherence is a challenge beyond personal physiotherapy. The main objective of this study is to investigate the short- and long-term (cost-)effectiveness of the SmArt-E programme in people with hip and/or knee osteoarthritis in terms of pain and physical functioning compared to usual care. Methods This study is designed as a multicentre randomized controlled trial with a target sample size of 330 patients. The intervention is based on the e-Exercise intervention from the Netherlands, consists of a training and education programme and is conducted as a blended care intervention over 12 months. We use an app to support independent training and the development of self-management skills. The primary and secondary hypotheses are that participants in the SmArt-E intervention will have less pain (numerical rating scale) and better physical functioning (Hip Disability and Osteoarthritis Outcome Score, Knee Injury and Osteoarthritis Outcome Score) compared to participants in the usual care group after 12 and 3 months. Other secondary outcomes are based on domains of the Osteoarthritis Research Society International (OARSI). The study will be accompanied by a process evaluation. Discussion After a positive evaluation, SmArt-E can be offered in usual care, flexibly addressing different care situations. The desired sustainability and the support of the participants’ behavioural change are initiated via the app through audio-visual contact with their physiotherapists. Furthermore, the app supports the repetition and consolidation of learned training and educational content. For people with osteoarthritis, the new form of care with proven effectiveness can lead to a reduction in underuse and misuse of care as well as contribute to a reduction in (in)direct costs. Trial registration German Clinical Trials Register, DRKS00028477. Registered on August 10, 2022.

Schillings C., Meissner D., Erb B., Schultchen D., Bendig E., Pollatos O.

BackgroundStress levels in the general population had already been increasing in recent years, and have subsequently been exacerbated by the global pandemic. One approach for innovative online-based interventions are “chatbots” – computer programs that can simulate a text-based interaction with human users via a conversational interface. Research on the efficacy of chatbot-based interventions in the context of mental health is sparse. The present study is designed to investigate the effects of a three-week chatbot-based intervention with the chatbot ELME, aiming to reduce stress and to improve various health-related parameters in a stressed sample.MethodsIn this multicenter, two-armed randomised controlled trial with a parallel design, a three-week chatbot-based intervention group including two daily interactive intervention sessions via smartphone (á 10–20 min.) is compared to a treatment-as-usual control group. A total of 130 adult participants with a medium to high stress levels will be recruited in Germany. Assessments will take place pre-intervention, post-intervention (after three weeks), and follow-up (after six weeks). The primary outcome is perceived stress. Secondary outcomes include self-reported interoceptive accuracy, mindfulness, anxiety, depression, personality, emotion regulation, psychological well-being, stress mindset, intervention credibility and expectancies, affinity for technology, and attitudes towards artificial intelligence. During the intervention, participants undergo ecological momentary assessments. Furthermore, satisfaction with the intervention, the usability of the chatbot, potential negative effects of the intervention, adherence, potential dropout reasons, and open feedback questions regarding the chatbot are assessed post-intervention.DiscussionTo the best of our knowledge, this is the first chatbot-based intervention addressing interoception, as well as in the context with the target variables stress and mindfulness. The design of the present study and the usability of the chatbot were successfully tested in a previous feasibility study. To counteract a low adherence of the chatbot-based intervention, a high guidance by the chatbot, short sessions, individual and flexible time points of the intervention units and the ecological momentary assessments, reminder messages, and the opportunity to postpone single units were implemented.Trial registrationThe trial is registered at the WHO International Clinical Trials Registry Platform via the German Clinical Trials Register (DRKS00027560; date of registration: 06 January 2022). This is protocol version No. 1. In case of important protocol modifications, trial registration will be updated.

Weichbroth P.

In the area of broadband wireless Internet, mobile applications have already replaced their desktop equivalents and are recognized as valuable tools for any size of businesses and for private use. With the emergence of millions of apps, the quality of their interaction with the user remains an open question for software vendors. While female and male requirements and preferences are not always similar, to the best of our knowledge, few studies have examined the impact of gender on mobile applications usability. Therefore, the goal of this study is to assess their usability from the perspective of female and male users, and to evaluate the differences between them. In our study, based on an experimental setup with a group of 40 users (16 females and 24 males), with regard to three usability attributes, namely efficiency, effectiveness and satisfaction, both qualitative and quantitative data were collected, respectively via pre- and post-testing questionnaires and during application testing sessions, combined with the think aloud protocol. To analyze the collected data, descriptive statistics were extracted from the video data and used to calculate the inferential statistics. With a significance level (alpha) of 5%, our findings show that between the groups of females and males, there were no statistically significant differences in the performance accuracy, average completion time, and perceived satisfaction, since all ${p}$ values are greater than the assumed alpha. Hence, one can conclude that no effect of gender was observed with regard to the usability of the Gmail application. Overall, the empirical results contribute to the ongoing research on mobile application usability by providing evidence-based insights that we believe may be valuable for both theory and practice.

Pape M., Färber T., Seiferth C., Roth T., Schroeder S., Wolstein J., Herpertz S., Steins-Loeber S.

Background Given the increase in the prevalence of overweight and obesity worldwide, the number of digital weight loss interventions has also risen. However, these interventions often lack theoretical background and data on long-term effectiveness. The consideration of individual and gender differences in weight-related psychological parameters might enhance the efficacy and sustainability of mobile-based weight loss interventions. Objective This paper presented an introduction to and the process evaluation of a 12-week gender-sensitive mobile health (mHealth) weight loss intervention (I-GENDO) combining computer-based and self-tailoring features. Methods Between August 2020 and August 2021, individuals with overweight (BMI 25.0-29.9 kg/m²), those with obesity class I (BMI 30.0-34.9 kg/m²), and those with obesity class II (BMI 35.0-39.9 kg/m²) were recruited to the I-GENDO project, a multicenter study in Germany. The mHealth intervention aimed at targeting individual psychological factors associated with the development and persistence of overweight and obesity (eg, emotional eating) using computer-based tailoring. Moreover, the intervention took a gender-sensitive approach by implementing self-tailoring of gender-targeted module versions. The computer-based assignment of the main modules, self-selection of gender-targeted module versions, and use patterns were evaluated while considering gender. Moreover, gender differences in the usability assessment were analyzed. Results Data from the intervention arm of the study were processed. A total of 116 individuals with overweight and obesity (77/116, 66.4% women; age mean 47.28, SD 11.66 years; BMI mean 33.58, SD 3.79 kg/m2) were included in the analyses. Overall, the compliance (90/109, 82.6%) and satisfaction with the app (mean 86% approval) were high and comparable with those of other mobile weight loss interventions. The usability of the intervention was rated with 71% (5.0/7.0 points) satisfaction. More women obtained the main module that focused on emotion regulation skills. Most men and women selected women-targeted versions of the main modules. Women used the app more frequently and longer than men. However, women and men did not differ in the progress of use patterns throughout the intervention. Conclusions We developed a tailored gender-sensitive mHealth weight loss intervention. The usability of and engagement with the intervention were satisfactory, and the overall satisfaction with the intervention was also high. Gender differences must be considered in the evaluation of the effectiveness and sustainability of the intervention.

Mähs M., Pithan J.S., Bergmann I., Gabrys L., Graf J., Hölzemann A., Van Laerhoven K., Otto-Hagemann S., Popescu M.L., Schwermann L., Wenz B., Pahmeier I., Teti A.

One relevant strategy to prevent the onset and progression of type 2 diabetes mellitus (T2DM) focuses on increasing physical activity. The use of activity trackers by patients could enable objective measurement of their regular physical activity in daily life and promote physical activity through the use of a tracker-based intervention. This trial aims to answer three research questions: (1) Is the use of activity trackers suitable for longitudinal assessment of physical activity in everyday life? (2) Does the use of a tracker-based intervention lead to sustainable improvements in the physical activity of healthy individuals and in people with T2DM? (3) Does the accompanying digital motivational intervention lead to sustainable improvements in physical activity for participants using the tracker-based device? The planned study is a randomized controlled trial focused on 1642 participants with and without T2DM for 9 months with regard to their physical activity behavior. Subjects allocated to an intervention group will wear an activity tracker. Half of the subjects in the intervention group will also receive an additional digital motivational intervention. Subjects allocated to the control group will not receive any intervention. The primary outcome is the amount of moderate and vigorous physical activity in minutes and the number of steps per week measured continuously with the activity tracker and assessed by questionnaires at four time points. Secondary endpoints are medical parameters measured at the same four time points. The collected data will be analyzed using inferential statistics and explorative data-mining techniques. The trial uses an interdisciplinary approach with a team including sports psychologists, sports scientists, health scientists, health care professionals, physicians, and computer scientists. It also involves the processing and analysis of large amounts of data collected with activity trackers. These factors represent particular strengths as well as challenges in the study. The trial is registered at the World Health Organization International Clinical Trials Registry Platform via the German Clinical Studies Trial Register (DRKS), DRKS00027064 . Registered on 11 November 2021.

Shan Y., Ji M., Xie W., Li R., Qian X., Zhang X., Hao T.

Background The usability of mobile health (mHealth) apps needs to be effectively evaluated before they are officially approved to be used to deliver health interventions. To this end, the mHealth App Usability Questionnaire (MAUQ) has been designed and proved valid and reliable in assessing the usability of mHealth apps. However, this English questionnaire needs to be translated into other languages, adapted, and validated before being utilized to evaluate the usability of mHealth apps. Objective This study aims to improve, further adapt, and validate the Chinese version of the MAUQ (C-MAUQ; interactive for patients) on Left-handed Doctor, one of the most popular “reaching out to patients” interactive mHealth apps with chatbot function in China, to test the reliability and cross-cultural adaptability of the questionnaire. Methods The MAUQ (interactive for patients) has been translated into Chinese and validated for its reliability on Good Doctor, one of the most influential “reaching out to patients” mHealth apps without chatbot function in China. After asking for the researchers’ approval to use this Chinese version, we adjusted and further adapted the C-MAUQ by checking it against the original English version and improving its comprehensibility, readability, idiomaticity, and cross-cultural adaptability. Following a trial survey completed by 50 respondents on wenjuanxing, the most popular online questionnaire platform in China, the improved version of the C-MAUQ (I-C-MAUQ) was finally used to evaluate the usability of Left-handed Doctor through an online questionnaire survey (answered by 322 participants) on wenjuanxing, to test its internal consistency, reliability, and validity. Results The I-C-MAUQ still retained the 21 items and 3 dimensions of the original MAUQ: 8 items for usability and satisfaction, 6 items for system information arrangement, and 7 items for efficiency. The translation problems in the C-MAUQ, including (1) redundancy, (2) incompleteness, (3) misuse of parts of speech, (4) choice of inappropriate words, (5) incomprehensibility, and (6) cultural difference–induced improper translation, were improved. As shown in the analysis of data obtained through the online survey, the I-C-MAUQ had a better internal consistency (ie, the correlation coefficient between the score of each item and the total score of the questionnaire determined within the range of 0.861-0.938; P<.01), reliability (Cronbach α=.988), and validity (Kaiser–Meyer–Olkin=0.973), compared with the C-MAUQ. It was effectively used to test the usability of Left-handed Doctor, eliciting over 80% of informants’ positive attitudes toward this mHealth app. Conclusions The I-C-MAUQ is highly reliable and valid for Left-handed Doctor, and suitable for testing the usability of interactive mHealth apps used by patients in China. This finding further confirms the cross-cultural validity, reliability, and adaptability of the MAUQ. We identified certain factors influencing the perceived usability of mHealth apps, including users’ age, gender, education, profession, and possibly previous experience with mHealth apps and the chatbot function of such apps. Most notably, we found a wider acceptance of this new technology among young Chinese female college students who were more engaged in the interaction with health care chatbots. The age-, gender-, and profession-induced preference for new digital health interventions in China aligns with the findings in other similar studies in America and Malaysia. This preference identifies areas for further research on the social, cultural, and gender adaptation of health technologies.

Alanzi T.M.

Purpose This study aims to investigate the users’ satisfaction levels about mHealth applications and their intentions to use them in future (in post-Covid-19 times) in Saudi Arabia. Methods A cross-sectional survey design was adopted in this study. The mHealth App Usability Questionnaire (MAUQ)22 was used in this study. An online version of the survey was created using Google Surveys, and a link for the survey was generated. As the objective of this study is to assess the users’ satisfaction levels about mHealth applications and their intentions to use them in future (in post-Covid-19 times) in Saudi Arabia, all individuals who used or using mHealth applications will be included in this study. At the end of the survey time period, 318 responses were received. Results Items including ’easy to learn using the app’ (mean rating = 3.9), ’easy to find information on the application’(mean rating = 3.9), ’interface of the app’ (mean rating = 3.8) were rated to be highly effective by the participants.). However, ability to recover from mistakes while using the applications (mean rating = 2.9), inconsistency in navigation (mean rating = 2.9), and lack of all necessary functions (mean rating = 2.3) were few issues identified. No statistically significant difference of opinions was observed in relation to all sub-scales of usability factor. Conclusion Although the satisfaction levels are good with respect to mHealth applications, the sudden rise in the mHealth users can be partly linked to the several psychological issues such as anxiety and depression among people and also because of restricted access during the pandemic.

Zhao S., Cao Y., Cao H., Liu K., Lv X., Zhang J., Li Y., Davidson P.M.

BackgroundMobile health (mHealth) apps have shown the advantages of improving medication compliance, saving time required for diagnosis and treatment, reducing medical expenses, etc. The World Health Organization (WHO) has recommended that mHealth apps should be evaluated prior to their implementation to ensure their accuracy in data analysis.ObjectiveThis study aimed to translate the patient version of the interactive mHealth app usability questionnaire (MAUQ) into Chinese, and to conduct cross-cultural adaptation and reliability and validity tests.MethodsThe Brislin’s translation model was used in this study. The cross-cultural adaptation was performed according to experts’ comments and the results of prediction test. The convenience sampling method was utilized to investigate 346 patients who used the “Good Doctor” (“Good Doctor” is the most popular mHealth app in China), and the reliability and validity of the questionnaire were evaluated as well.ResultsAfter translation and cross-cultural adaptation, there were a total of 21 items and 3 dimensions: usability and satisfaction (8 items), system information arrangement (6 items), and efficiency (7 items). The content validity index was determined to be 0.952, indicating that the 21 items used to evaluate the usability of the Chinese version of the MAUQ were well correlated. The Cronbach’s α coefficient of the total questionnaire was 0.912, which revealed that the questionnaire had a high internal consistency. The values of test-retest reliability and split-half reliability of the Chinese version of the MAUQ were 0.869 and 0.701, respectively, representing that the questionnaire had a good stability.ConclusionThe translated questionnaire has good reliability and validity in the context of Chinese culture, and it could be used as a usability testing tool for the patient version of interactive mHealth apps.

Hajesmaeel-Gohari S., Khordastan F., Fatehi F., Samzadeh H., Bahaadinbeigy K.

Various questionnaires are used for evaluating satisfaction, usability, acceptance, and quality outcomes of mobile health (mHealth) services. Using the best one to meet the needs of an mHealth study is a challenge for researchers. Therefore, this study aimed to review and determine the frequently used questionnaires for evaluating the mentioned outcomes of mHealth services. The PubMed database was searched for conducting this review in April 2021. Papers that used a referenced questionnaire to evaluate the satisfaction, usability, acceptance, or quality outcomes of mHealth were included. The first author’s name, year of publication, evaluation outcome, and evaluation questionnaire were extracted from relevant papers. Data were analyzed using descriptive statistics. In total, 247 papers were included in the study. Questionnaires were used for usability (40%), quality (34.5%), acceptance (8.5%), and satisfaction (4%) outcomes, respectively. System usability scale (36.5%), mobile application rating scale (35.5%), post study system usability questionnaire (6%), user mobile application rating scale (5%), technology acceptance model (4.5%), computer system usability questionnaire (2.5%), net promoter score (2%), health information technology usability evaluation scale (2%), the usefulness, satisfaction, and ease of use (1.5%), client satisfaction questionnaire (1.5%), unified theory of acceptance and use of technology (1.5%), questionnaire for user interaction satisfaction (1%), user experience questionnaire (1%), and after-scenario questionnaire (1%) were the most used questionnaires, respectively. Despite the existence of special questionnaires for evaluating several outcomes of mHealth, general questionnaires with fewer items and higher reliability have been used more frequently. Researchers should pay more attention to questionnaires with a goal-based design.

Meryk A., Kropshofer G., Hetzer B., Riedl D., Lehmann J., Rumpold G., Haid A., Holzner B., Crazzolara R.

Background Several stakeholders, including patients and health care providers, suggest symptom self-reporting measurements for a more patient-directed cancer control approach. However, services tailored to measure daily reporting and implementing it in clinical care are lacking. This study aimed to evaluate the feasibility and value of daily patient-reported outcome measures (PROMs) by children receiving chemotherapy for cancer. Methods Health status was recorded daily with a web-based child-friendly patient portal (ePROtect). Following aspects of feasibility and usability were assessed: (a) the completion rate and time, (b) user feedback on usability and satisfaction, and (c) the performed interventions if moderate to severe symptom deterioration was noted. Results Twelve children (median age: 7.2 years) were included. A total number of 891 daily reports were collected during the study period; the median percentage of ePROtect completion days was 85.3% (interquartile range [IQR] 64.2–100.0) and 55.9% (IQR 51.9–76.9) for inpatient and outpatient stay, respectively. Mean time to complete the questionnaire was 47.6 seconds. Severe symptoms were reported in 14.7% of measurement time points, which led to prompt health care interventions in 57 cases, including extension of supportive care (n = 37) and pre-emptive inpatient admissions (n = 5). Over 80% of the patients (10/12) and their proxies (16/18) provided feedback with high rating for satisfaction (>90%) and usefulness (>80%) of ePROtect. Conclusion Our study shows that daily symptom monitoring is feasible for all children with newly diagnosed cancer aged 5–18 years. Monitoring offers the opportunity to identify symptoms early and trigger appropriate clinical action.

Mustafa N., Safii N.S., Jaffar A., Sani N.S., Mohamad M.I., Abd Rahman A.H., Mohd Sidik S.

Background Mobile health (mHealth) apps play an important role in delivering education, providing advice on treatment, and monitoring patients’ health. Good usability of mHealth apps is essential to achieve the objectives of mHealth apps efficiently. To date, there are questionnaires available to assess the general system usability but not explicitly tailored to precisely assess the usability of mHealth apps. Hence, the mHealth App Usability Questionnaire (MAUQ) was developed with 4 versions according to the type of app (interactive or standalone) and according to the target user (patient or provider). Standalone MAUQ for patients comprises 3 subscales, which are ease of use, interface and satisfaction, and usefulness. Objective This study aimed to translate and validate the English version of MAUQ (standalone for patients) into a Malay version of MAUQ (M-MAUQ) for mHealth app research and usage in future in Malaysia. Methods Forward and backward translation and harmonization of M-MAUQ were conducted by Malay native speakers who also spoke English as their second language. The process began with a forward translation by 2 independent translators followed by harmonization to produce an initial translated version of M-MAUQ. Next, the forward translation was continued by another 2 translators who had never seen the original MAUQ. Lastly, harmonization was conducted among the committee members to resolve any ambiguity and inconsistency in the words and sentences of the items derived with the prefinal adapted questionnaire. Subsequently, content and face validations were performed with 10 experts and 10 target users, respectively. Modified kappa statistic was used to determine the interrater agreement among the raters. The reliability of the M-MAUQ was assessed by 51 healthy young adult mobile phone users. Participants needed to install the MyFitnessPal app and use it for 2 days for familiarization before completing the designated task and answer the M-MAUQ. The MyFitnessPal app was selected because it is one among the most popular installed mHealth apps globally available for iPhone and Android users and represents a standalone mHealth app. Results The content validity index for the relevancy and clarity of M-MAUQ were determined to be 0.983 and 0.944, respectively, which indicated good relevancy and clarity. The face validity index for understandability was 0.961, which indicated that users understood the M-MAUQ. The kappa statistic for every item in M-MAUQ indicated excellent agreement between the raters (κ ranging from 0.76 to 1.09). The Cronbach α for 18 items was .946, which also indicated good reliability in assessing the usability of the mHealth app. Conclusions The M-MAUQ fulfilled the validation criteria as it revealed good reliability and validity similar to the original version. M-MAUQ can be used to assess the usability of mHealth apps in Malay in the future.

Messner E., Terhorst Y., Barke A., Baumeister H., Stoyanov S., Hides L., Kavanagh D., Pryss R., Sander L., Probst T.

Background The number of mobile health apps (MHAs), which are developed to promote healthy behaviors, prevent disease onset, manage and cure diseases, or assist with rehabilitation measures, has exploded. App store star ratings and descriptions usually provide insufficient or even false information about app quality, although they are popular among end users. A rigorous systematic approach to establish and evaluate the quality of MHAs is urgently needed. The Mobile App Rating Scale (MARS) is an assessment tool that facilitates the objective and systematic evaluation of the quality of MHAs. However, a German MARS is currently not available. Objective The aim of this study was to translate and validate a German version of the MARS (MARS-G). Methods The original 19-item MARS was forward and backward translated twice, and the MARS-G was created. App description items were extended, and 104 MHAs were rated twice by eight independent bilingual researchers, using the MARS-G and MARS. The internal consistency, validity, and reliability of both scales were assessed. Mokken scale analysis was used to investigate the scalability of the overall scores. Results The retranslated scale showed excellent alignment with the original MARS. Additionally, the properties of the MARS-G were comparable to those of the original MARS. The internal consistency was good for all subscales (ie, omega ranged from 0.72 to 0.91). The correlation coefficients (r) between the dimensions of the MARS-G and MARS ranged from 0.93 to 0.98. The scalability of the MARS (H=0.50) and MARS-G (H=0.48) were good. Conclusions The MARS-G is a reliable and valid tool for experts and stakeholders to assess the quality of health apps in German-speaking populations. The overall score is a reliable quality indicator. However, further studies are needed to assess the factorial structure of the MARS and MARS-G.

Ibili E., Billinghurst M.

Zhou L., Bao J., Setiawan I.M., Saptono A., Parmanto B.

After a mobile health (mHealth) app is created, an important step is to evaluate the usability of the app before it is released to the public. There are multiple ways of conducting a usability study, one of which is collecting target users' feedback with a usability questionnaire. Different groups have used different questionnaires for mHealth app usability evaluation: The commonly used questionnaires are the System Usability Scale (SUS) and Post-Study System Usability Questionnaire (PSSUQ). However, the SUS and PSSUQ were not designed to evaluate the usability of mHealth apps. Self-written questionnaires are also commonly used for evaluation of mHealth app usability but they have not been validated.The goal of this project was to develop and validate a new mHealth app usability questionnaire.An mHealth app usability questionnaire (MAUQ) was designed by the research team based on a number of existing questionnaires used in previous mobile app usability studies, especially the well-validated questionnaires. MAUQ, SUS, and PSSUQ were then used to evaluate the usability of two mHealth apps: an interactive mHealth app and a standalone mHealth app. The reliability and validity of the new questionnaire were evaluated. The correlation coefficients among MAUQ, SUS, and PSSUQ were calculated.In this study, 128 study participants provided responses to the questionnaire statements. Psychometric analysis indicated that the MAUQ has three subscales and their internal consistency reliability is high. The relevant subscales correlated well with the subscales of the PSSUQ. The overall scale also strongly correlated with the PSSUQ and SUS. Four versions of the MAUQ were created in relation to the type of app (interactive or standalone) and target user of the app (patient or provider). A website has been created to make it convenient for mHealth app developers to use this new questionnaire in order to assess the usability of their mHealth apps.The newly created mHealth app usability questionnaire-MAUQ-has the reliability and validity required to assess mHealth app usability.

Gäumann S., Ziller C., Paulissen N., Behrendt F., Suica Z., Crüts B., Gammerschlag L., Parmar K., Gerth H.U., Bonati L.H., Schuster-Amft C.

IntroductionEffective rehabilitation is essential to prevent physical and cognitive decline, but many stroke patients face challenges to maintain rehabilitation efforts after hospital discharge. Telerehabilitation, delivered via digital platforms, represents a promising approach for intensive continuation of stroke rehabilitation after discharge. The Swiss tele-assisted rehabilitation and training program (START), delivered through the Blended Clinic mobile application, seeks to support patients to start during inpatient rehabilitation, continue during the transition to the home environment, continue until outpatient rehabilitation starts and beyond. The study aims to evaluate feasibility, safety and performance of the START program on the Blended Clinic platform during inpatient, transition, and outpatient rehabilitation with patients in the early and late subacute phase after a stroke. Furthermore, patients' functional status, mobility and activity level, and health-related quality of life are monitored.MethodsThis single-center feasibility trial with three measurement sessions will include 40 patients, who will be introduced to START during their inpatient rehabilitation. Patients will continue for 12 weeks post-discharge. For the feasibility assessment, process-, training- and mHealth-related parameter will be evaluated, which include recruitment rate, process-evaluation, safety, adherence, drop-out rate, stability and maintenance of the system, usability, quality, satisfaction, user and program experience, and perceived change. Secondary outcomes will focus on motor function, mobility, quality of life, activity level, heart rate, blood pressure, and performance-based measures.DiscussionThe study's strengths include its foundation in previous usability analyses, which informed refinements to the START program. The study's design is based on the ISO 14155 standard, ensuring high standards for medical device research and supporting the future certification of the START program on the Blended Clinic platform. Potential challenges include patient self-reporting via the mobile application and barriers related to technology use among older adults and older mobile devices. Additionally, the availability of coaching is limited to business hours, which may affect adherence. Despite these challenges, the study's findings will provide insights into the feasibility of mobile-based telerehabilitation and guide the design of a future randomized controlled trial. Clinical Trial RegistrationThe study is registered with the Swiss National Clinical Trial Portal (SNCTP000005943), EUDAMED (CIV-CH-24-05-046954), and clinicaltrils.gov (NCT06449612).

Åvik Persson H., Castor C., Andersson N., Hylén M.

Abstract Background The Swedish health care system is undergoing a transformation. eHealth technologies are increasingly being used. The System Usability Scale is a widely used tool, offering a standardized and reliable measure for assessing the usability of digital health solutions. However, despite the existence of several translations of the System Usability Scale into Swedish, none have undergone psychometric validation. This highlights the urgent need for a validated and standardized Swedish version of the System Usability Scale to ensure accurate and reliable usability evaluations. Objective The aim of the study was to translate and psychometrically evaluate a Swedish version of the System Usability Scale. Methods The study utilized a 2-phase design. The first phase translated the System Usability Scale into Swedish and the second phase tested the scale’s psychometric properties. A total of 62 participants generated a total of 82 measurements. Descriptive statistics were used to visualize participants’ characteristics. The psychometric evaluation consisted of data quality, scaling assumptions, and acceptability. Construct validity was evaluated by convergent validity, and reliability was evaluated by internal consistency. Results The Swedish version of the System Usability Scale demonstrated high conformity with the original version. The scale showed high internal consistency with a Cronbach α of .852 and corrected item-total correlations ranging from 0.454 to 0.731. The construct validity was supported by a significant positive correlation between the System Usability Scale and domain 5 of the eHealth Literacy Questionnaire (P=.001). Conclusions The Swedish version of the System Usability Scale demonstrated satisfactory psychometric properties. It can be recommended for use in a Swedish context. The positive correlation with domain 5 of the eHealth Literacy Questionnaire further supports the construct validity of the Swedish version of the System Usability Scale, affirming its suitability for evaluating digital health solutions. Additional tests of the Swedish version of the System Usability Scale, for example, in the evaluation of more complex eHealth technology, would further validate the scale.

Oudbier S.J., Smets E.M., Nieuwkerk P.T., Neal D.N., Nurmohamed A.S., Meij H.J., Dusseljee-Peute L.W.

Abstract Background The field of digital health solutions (DHS) has grown tremendously over the past years. DHS include tools for self-management, which support individuals to take charge of their own health. The usability of DHS, as experienced by patients, is pivotal to adoption. However, well-known questionnaires that evaluate usability and satisfaction use complex terminology derived from human-computer interaction and are therefore not well suited to assess experienced usability of patients using DHS in a home setting. Objective This study aimed to develop, validate, and assess an instrument that measures experienced usability and satisfaction of patients using DHS in a home setting. Methods The development of the “Experienced Usability and Satisfaction with Self-monitoring in the Home Setting” (GEMS) questionnaire followed several steps. Step I consisted of assessing the content validity, by conducting a literature review on current usability and satisfaction questionnaires, collecting statements and discussing these in an expert meeting, and translating each statement and adjusting it to the language level of the general population. This phase resulted in a draft version of the GEMS. Step II comprised assessing its face validity by pilot testing with Amsterdam University Medical Center’s patient panel. In step III, psychometric analysis was conducted and the GEMS was assessed for reliability. Results A total of 14 items were included for psychometric analysis and resulted in 4 reliable scales: convenience of use, perceived value, efficiency of use, and satisfaction. Conclusions Overall, the GEMS questionnaire demonstrated its reliability and validity in assessing experienced usability and satisfaction of DHS in a home setting. Further refinement of the instrument is necessary to confirm its applicability in other patient populations in order to promote the development of a steering mechanism that can be applied longitudinally throughout implementation, and can be used as a benchmarking instrument.

Ashkani N., Erfannia L., Rezaee R., Savareh B.A., Bashiri A.

Cancer remains a leading cause of death, underscoring the importance of reliable health apps. This study aims to design and validate a content quality evaluation tool for cancer mobile applications ensuring they provide standard services and features to users. This descriptive-analytical study was conducted in steps including identifying mobile application contents, designing and validating evaluation tool. Studies published until February 19, 2023, in PubMed, Web of Science, Scopus, and Scientific Information Database (SID) databases, as well as mobile applications in Myket, Bazaar, and Google Play, were searched. Data extraction and expert panel validation followed, leading to four expert sessions to design the tool. Nineteen oncologists, health information management and medical informatics specialties were included in the study as a research community. The tool’s face validity, content validity and reliability were assessed using CVR (Content Validity Ratio), CVI (Content Validity Index), and Cronbach’s alpha respectively. Out of 242 initial search results in databases and 125 mobile applications in app stores, 25 articles and 20 mobile applications included based on the search limits. The designed tools consist of 8 main themes (prevention, diagnosis, treatment, follow-up, education, communication, requests/order and other) with 43 question items. All items received scores above 0.79 in CVI and 0.42 in CVR, leading to their approval and the reliability of the tool was accepted with a score of 0.967. Evaluating cancer mobile applications can be very helpful in developing useful and supportive mobile apps, and assist health policymakers in identifying and recommending high-quality cancer-related apps. This will enable patients to choose the most suitable apps, enhancing the quality of cancer care.

Zimmermann J., Morf H., Scharf F., Knitza J., Moeller H., Muehlensiepen F., Nathrath M., Orlemann T., Voelker T., Deckers M.

Background The exponential growth of telehealth is revolutionizing health care delivery, but its evaluation has not matched the pace of its uptake. Various forms of assessment, from single-item to more extensive questionnaires, have been used to assess telehealth and digital therapeutics and their usability. The most frequently used questionnaire is the “Telehealth Usability Questionnaire” (TUQ). The use of the TUQ is limited by its restricted availability in languages other than English and its feasibility. Objective The aims of this study were to create a translated German TUQ version and to derive a short questionnaire for patients—“Telehealth Usability and Perceived Usefulness Short Questionnaire for patients” (TUUSQ). Methods As a first step, the original 21-item TUQ was forward and back-translated twice. In the second step, 13 TUQ items were selected for their suitability for the general evaluation of telehealth on the basis of expert opinion. These 13 items were surveyed between July 2022 and September 2023 in 4 studies with patients and family members of palliative care, as well as patients with chronic autoimmune diseases, evaluating 13 health care apps, including digital therapeutics and a telehealth system (n1=128, n2=220, n3=30, and n4=12). Psychometric exploratory factor analysis was conducted. Results The analysis revealed that a parsimonious factor structure with 2 factors (“perceived usefulness in health care” and “usability”) is sufficient to describe the patient’s perception. Consequently, the questionnaire could be shortened to 6 items without compromising its informativeness. Conclusions We provide a linguistically precise German version of the TUQ for assessing the usability and perceived usefulness of telehealth. Beyond that, we supply a highly feasible shortened version that is versatile for general use in telehealth, mobile health, and digital therapeutics, which distinguishes between the 2 factors “perceived usefulness in health care” and “usability” in patients. Trial Registration German Clinical Trials Register DRKS00030546; https://drks.de/search/de/trial/DRKS00030546

Anders C., Moorthy P., Svensson L., Müller J., Heinze O., Knaup P., Wallwiener M., Deutsch T.M., Le T., Weinert L.

Background Early identification of quality of life (QoL) loss and side effects is a key challenge in breast cancer therapy. Digital tools can be helpful components of therapeutic support. Enable, a smartphone app, was used in a multicenter, prospective randomized controlled trial in 3 breast cancer centers. The app simultaneously serves as a therapy companion (eg, by displaying appointments), a tool for documenting QoL (eg, by enabling data collection for QoL questionnaires), and documentation of patient-reported side effects. The need for digital tools is continually rising. However, evidence of the effects of long-term use of mobile health (mHealth) apps in aftercare for patients with breast cancer is limited. Therefore, evaluating the usability and understanding the user experience of this mHealth app could potentially contribute valuable insights in this field. Objective A usability study was conducted to explore how patients with breast cancer receiving neoadjuvant, adjuvant, or palliative outpatient treatment rated their engagement with the app , the user experience, and the benefits of using the app. Methods A mixed methods approach was chosen to combine subjective and objective measures, including an eye-tracking procedure, a standardized usability questionnaire (mHealth App Usability Questionnaire), and semistructured interviews. Participants were surveyed twice during the study period. Interviews were transcribed verbatim and analyzed using thematic analysis. Analysis of the eye-tracking data was carried out using the tracker-integrated software. Descriptive analysis was conducted for the quantitative data. Results The mHealth App Usability Questionnaire results (n=105) indicated good overall usability for 2 different time points (4 wk: mean 89.15, SD 9.65; 20 wk: mean 85.57, SD 12.88). The qualitative analysis of the eye-tracking recordings (n=10) and interviews (n=16) showed that users found the Enable app easy to use. The design of the app, information about therapies and side effects, and usefulness of the app as a therapy companion were rated positively. Additionally, participants contributed requests for additional app features and suggestions for improving the content and usability of the app. Relevant themes included optimization of the appointment feature, updating the app’s content regularly, and self-administration. In contrast to the app’s current passive method of operation, participants expressed a desire for more active engagement through messaging, alarms, or emails. Conclusions The results of this study demonstrate the good usability of the Enable app as well as the potential for further development. We concluded from patients’ feedback and requests that mHealth apps could benefit from giving patients a more active role (eg, being able to actively document side effects as they occur). Additionally, regular updates of app content could further contribute to encouraging continued use of mHealth apps. Our findings may also assist other researchers in tailoring their mHealth apps to the actual needs of patients undergoing breast cancer therapy.

Deshmukh A.M., Chalmeta R.

In recent years, user experience (UX) has gained importance in the field of interactive systems. To ensure its success, interactive systems must be evaluated. As most of the standardized evaluation tools are dedicated to graphical user interfaces (GUIs), the evaluation of voice-based interactive systems or voice user interfaces is still in its infancy. With the help of a well-established evaluation scale, the System Usability Scale (SUS), two prominent, widely accepted voice assistants were evaluated. The evaluation, with SUS, was conducted with 16 participants who performed a set of tasks on Amazon Alexa Echo Dot and Google Nest Mini. We compared the SUS score of Amazon Alexa Echo Dot and Google Nest Mini. Furthermore, we derived the confidence interval for both voice assistants. To enhance understanding for usability practitioners, we analyzed the Adjective Rating Score of both interfaces to comprehend the experience of an interface’s usability through words rather than numbers. Additionally, we validated the correlation between the SUS score and the Adjective Rating Score. Finally, a paired sample t-test was conducted to compare the SUS score of Amazon Alexa Echo Dot and Google Nest Mini. This resulted in a huge difference in scores. Hence, in this study, we corroborate the utility of the SUS in voice user interfaces and conclude by encouraging researchers to use SUS as a usability metric to evaluate voice user interfaces.

Šafran V., Lin S., Nateqi J., Martin A.G., Smrke U., Ariöz U., Plohl N., Rojc M., Bēma D., Chávez M., Horvat M., Mlakar I.

The importance and value of real-world data in healthcare cannot be overstated because it offers a valuable source of insights into patient experiences. Traditional patient-reported experience and outcomes measures (PREMs/PROMs) often fall short in addressing the complexities of these experiences due to subjectivity and their inability to precisely target the questions asked. In contrast, diary recordings offer a promising solution. They can provide a comprehensive picture of psychological well-being, encompassing both psychological and physiological symptoms. This study explores how using advanced digital technologies, i.e., automatic speech recognition and natural language processing, can efficiently capture patient insights in oncology settings. We introduce the MRAST framework, a simplified way to collect, structure, and understand patient data using questionnaires and diary recordings. The framework was validated in a prospective study with 81 colorectal and 85 breast cancer survivors, of whom 37 were male and 129 were female. Overall, the patients evaluated the solution as well made; they found it easy to use and integrate into their daily routine. The majority (75.3%) of the cancer survivors participating in the study were willing to engage in health monitoring activities using digital wearable devices daily for an extended period. Throughout the study, there was a noticeable increase in the number of participants who perceived the system as having excellent usability. Despite some negative feedback, 44.44% of patients still rated the app’s usability as above satisfactory (i.e., 7.9 on 1–10 scale) and the experience with diary recording as above satisfactory (i.e., 7.0 on 1–10 scale). Overall, these findings also underscore the significance of user testing and continuous improvement in enhancing the usability and user acceptance of solutions like the MRAST framework. Overall, the automated extraction of information from diaries represents a pivotal step toward a more patient-centered approach, where healthcare decisions are based on real-world experiences and tailored to individual needs. The potential usefulness of such data is enormous, as it enables better measurement of everyday experiences and opens new avenues for patient-centered care.