Proposal
Abstract
Background: Communication and comprehension of medical information are known barriers in health communication and equity, especially for non–English-speaking caregivers of children with special health care needs.
Objective: The objective of this proposal was to develop an interoperable and scalable medical translation app for non–English-speaking caregivers to facilitate the conversation between provider and caregiver/patient.
Methods: We employed user-centered and participatory design methods to understand the problems and develop a solution by engaging the stakeholder team (including caregivers, physicians, researchers, clinical informaticists, nurses, developers, nutritionists, pharmacists, and interpreters) and non–English-speaking caregiver participants.
Results: Considering the lack of interpreter service accessibility and advancement in translation technology, our team will develop and test an integrated, multimodal (voice-interactive and text-based) patient portal communication and translation app to enable non–English-speaking caregivers to communicate with providers using their preferred languages. For this initial prototype, we will focus on the Spanish language and Spanish-speaking families to test technical feasibility and evaluate usability.
Conclusions: Our proposal brings a unique perspective to medical translation and communication between caregiver and provider by (1) enabling voice entry and transcription in health care communications, (2) integrating with patient portals to facilitate caregiver and provider communications, and (3) adopting a translation verification model to improve accuracy of artificial intelligence–facilitated translations. Expected outcomes include improved health communications, literacy, and health equity. In addition, data points will be collected to improve autotranslation services in medical communications. We believe our proposed solution is affordable, interoperable, and scalable for health systems.
doi:10.2196/21038
Keywords
Introduction
Background
Children with special health care needs have a high number of hospitalizations and require more specialized services. Due to the complexity of their conditions, transitioning children with special health care needs from inpatient care to continuing therapy in the home setting is often challenging. Despite these challenges, a successful transition of care is critical to help families of children with special health care needs optimize their child’s outcome. Caregivers of children with special health care needs need to communicate with the clinical team frequently to report symptoms, request medication refills, receive care instructions, seek care advice, and discuss other health issues. Thus, ensuring that caregivers of children with special health care needs understand the discharge summaries and care plans, have timely access to communication channels to address their questions, and are given guidance to appropriate services is essential for a smooth transition of care and to achieve the best state of health for their children.
Communication and comprehension of medical information is a known barrier for patients and their caregivers, especially for non–English-speaking caregivers [
, ]. Non–English-speaking caregivers, by definition, are caregivers who have limited proficiency and understanding of the English language. Use of professional interpreters in medical communications with non–English-speaking caregivers and patients is a standard procedure. During discharge, instructions on medications and care plans are typically delivered through a professional interpreter—in person or through phone/video services [ ]. However, it is possible that non–English-speaking caregivers might not fully understand the information provided during transition of care and might not feel comfortable engaging in a conversation with the clinical team. In addition, it is not common for translated medication instructions and care plans to be provided in written format, especially for less common languages. Non–English-speaking caregivers have to rely on their own notes or memories to adhere to the care or medication guidance. There are also scenarios when public health and clinical guidelines have been disseminated broadly without support for non–English-speaking communities, which may lead to inequitable care [ , ], as evidenced by the current COVID-19 pandemic crisis [ ].Gap Analysis
Unfortunately, it can be difficult to access a certified medical interpreter whenever needed. In addition to accessibility and availability, there is the additional burden of cost to the health care system of providing interpretation services [
, ]. Another barrier to adequate interpretation for non–English-speaking caregivers is the limited engagement of interpreters at the point of care. Communication typically occurs at a single point in time and mostly focuses on specific current aspects of care without a comprehensive conversation (eg, treatment plans, assessment, diagnosis). Needing a professional interpreter creates a hurdle for patients and their caregivers outside clinical encounters, especially after hospital discharge, when non–English-speaking caregivers need information and guidance the most, including care navigation.Under Title VI of the 1964 Civil Rights Act, public agencies are obligated to provide competent language assistance to individuals with limited proficiency in the English language. Use of certified medical interpreters is ideal, but these individuals may be limited in their availability, particularly outside of business hours, in less-resourced settings, and for less common languages or dialects. Technology-based language translation was identified as a potential approach for facilitating and improving communication among patients and health care providers [
]. Web-based translation tools, such as Google Translate, were found to be useful in patient-provider interactions [ ]. However, these tools demonstrate significant quality issues in terms of discrepancy and inaccuracy for clinical communications [ - ]. Evidence suggests that significant improvements in commonly used web-based translation tools will be required before these tools can be used reliably for routine medical translation.In summary, there is a critical need to fill this gap of translation services for non–English-speaking caregivers of children with special health care needs who need it most during transitions of care and for timely communications with providers outside of clinical settings. We propose to develop an easy-to-use mobile translation and documentation app that could be integrated with patient portals to bridge the communication gap between providers and non–English-speaking caregivers. We hypothesize that our app may reduce the dependency on medical interpreters for certain communication needs, thereby reducing potential delays in communication between non–English-speaking caregivers and their child’s care team. In addition, such an app may increase the efficiency of two-way communication and documentation of non–English-speaking caregivers by enabling message archiving into both the app and the patient’s electronic health record (EHR). In this study, our aim is to understand the technical feasibility and usability of the app for health translation services during the transition of children with special health care needs to home care.
Prior Works
To our knowledge, there is no existing medical translation app with translation ability for Spanish users specializing in the care of children with special health care needs. Current translation services in apps (eg, Google Translate, iTranslate, and Speak & Translate) provide very limited support with generic medical term translations. All translation apps, other than Google Translate, are accessible only by paid subscribers. Currently, voice-to-text or voice-to-voice translation services for Spanish exist, but they are not integrated with medical systems [
]. Canopy Speak was found to perform well as a medical translator app [ ], but it was not capable of integration with EHRs at the time of the study. provides a list of medical diary apps available on the market as of March 2020. The majority of the apps are in the English language and few support other languages. None of the diary apps have any functionality of translation or integration to EHRs for communications.Methods
User-Centered Approach
The Digital Health Innovation Team and the Section of Complex Care at Nationwide Children’s Hospital (NCH) have collaborated to bring together multidisciplinary perspectives and expertise to develop digital health solutions in this area. We engaged with end users iteratively throughout the entire process of problem solving—from initial problem identification and formulation, ideation of possible solutions, integration with clinical workflow, testing of user acceptance, and choosing a deployment strategy to obtaining postimplementation user feedback.
During preliminary exploration and shadowing sessions in the clinical areas with providers, nurses, nutritionists, and pharmacists, multiple challenges of translation service availability and accessibility were identified. One example was when a mother of a patient called the clinic’s nursing line when translation services were not available. She left a message in her native language; by the time the nurse had a medical interpreter available to call the mother back, she was not available. This situation recurred multiple times during the observation period. There was consensus among the clinical team that only a few languages were common in their patient population but translation services for those languages may not be readily accessible. This leads to additional stress and burden for both families and clinical staff. In the case of urgent concerns, the lack of timely interpretation could have negative health consequences. We employed a user-centered design in first understanding the problems identified by our complex care team members, and then brainstorming together about potential solutions. Additionally, we interviewed complex care teams to better understand the clinical workflow and how potential solutions could fit into it.
After exploring the use of medical interpreters, the lack of interpreter service accessibility, and the lack of translation technology maturity, we propose a medical translation communication app, which we initially intend to support Spanish-speaking families. If successful, the app could be easily expanded to accommodate other languages. Since 2006, the staff at NCH has cared for more than 24,000 registered Spanish-speaking pediatric patients across 800,000 medical encounters and generated more than 2.8 million patient notes. We plan to interview non–English-speaking caregivers to understand their needs in medical communications, their challenges in using a traditional patient portal, and how a translation app could be seamlessly integrated into the existing channels of communication with their child’s care team. Our previous user-centered research has been successfully implemented [
], funded (eg, Challenges of the Health Resources and Service Administration’s Maternal and Child Health Bureau), and supported in complex care [ , ]. We will follow a similar strategy to engage end users and community members with our multidisciplinary stakeholder team.Research Team
We will build the app in-house by leveraging the expertise of our experienced developers and the information systems team at NCH. We assembled an interdisciplinary stakeholder team (including caregivers, physicians, researchers, clinical informaticists, nurses, developers, nutritionists, pharmacists, and interpreters) at our hospital to better understand and outline the existing problems and opportunities, and to create the potential solutions. Supporting this team are a number of information technology (IT) personnel who have experience building Amazon Web Services (AWS) solutions, EHR analysts, physician informaticists, other clinicians, and 150 full-time care coordinators.
Proposed Solution
We propose to develop an EHR-integrated, multimodal (voice-interactive and text-based) communication and translation app to enable non–English-speaking caregivers and providers to communicate with one another, each in their preferred language, during transition to home care (off site). Major user functionalities are grouped into 3 categories: communication, translation, and documentation.
Communication
Non–English-speaking caregivers will be able to enter medical messages via voice or text in Spanish. Entries will be transcribed and translated online using AWS. The provider will receive an English transcribed text (with audio attachment if voice entry is used) and will be able to respond using text or voice, which will be translated into Spanish through AWS and sent as transcribed text with an audio attachment (
).Providers will be able to initiate communication and access communication histories through the EHR with no change to their existing clinical workflow. The app will be integrated into the EHR through the patient portal, which allows providers to see patient messages in the EHR message inbox. Providers will have access to a web app to translate their messages. The translated texts will be sent over the patient portal.
provides an example of an app interaction between a Spanish non–English-speaking caregiver and a complex care nurse. As a nurse initiates the conversation over the EHR messaging service, it will be translated into Spanish using AWS and will appear in the app interface.Translation
Web-based translation services through AWS will be used to translate messages and information shared by caregivers and providers. However, to reduce the number of translation errors, we will add a verification process that requires users to manually review translated messages (
). When a caregiver enters a message in Spanish, the text will be translated into English and back into Spanish in real time. The caregiver will be able to review the Spanish text and make edits, such as rephrasing the sentence or changing words, to reduce errors. This process may continue iteratively until the caregiver is satisfied with the translation. Similarly, providers will be able to enter English text through the web app and to review translated texts.When necessary, on-site professional interpreters will be able to assist with message translations by correcting and verifying complex or crucial autotranslated messages to ensure validity. Verification will be shown on the app (
shows verified and unverified messages). Interpreter assistance will be helpful in translating discharge documents for the caregiver. Spanish non–English-speaking caregivers will also be able to use autotranslation in their communications through the app, and to correct or verify the content ( and ).Documentation
Nurses will be able to share discharge documentation and care plans through the app, which can be translated into Spanish through AWS. The app will log each communication entry and archive it on the cloud server and in the app, which non–English-speaking caregivers can review offline. Eventually, as outlined in
, the app will take on some of the workload of medical communication and translation. In the current practice, in each medical communication during home care transition, a professional interpreter is needed to communicate transition documentation and help with phone triaging. With the proposed model, translations will be maintained by the system (translating documents and messages), and professional interpreters will be asked to help when necessary for validation of complex communications. However, in-person communications at the hospital will still be maintained by professional interpreters.Infrastructure and Integration
To maximize access, minimize cost, and minimize disruption to the clinical or administrative workflow, we will design a scalable, extensible, and integrated solution, leveraging AWS as our backend environment for translation services, and integrate it with our EHR system through O-Auth 2.0 and Fast Healthcare Interoperability Resources (FHIR) standards. AWS has demonstrated promising developments in translation and natural language processing services, especially with medical transcription and understanding [
], with improving accuracy and expansion to additional languages. Using AWS allows us to develop a scalable infrastructure for the app. The elasticity of AWS will enable the app to be deployable to a larger user base without any code or infrastructure changes. Integration to the EHR through FHIR is also critical for a modern health application, although traditional web services may be leveraged for specific data elements to maintain compatibility with current EHR platforms. As health care provider organizations and EHR systems increasingly support these types of interoperable standards, the IT and administrative costs have been reduced to support the integration and deployment of a mobile app that interfaces with the EHR. NCH has already established guidelines and standard processes to address security and privacy issues. Some of the technical requirements are as follows: (1) infrastructure needs of AWS’ web server, DynamoDB database, and Lex/Polly services, which may affect the cost of service as more users adopt the app, (2) AWS translation application programming interfaces between Spanish and English [ ], and (3) an existing iPhone operating system (iOS) device (iOS 11+; support will be extended to Android devices in later phases).Evaluation
The usability and acceptance of the app will be measured using scientific methods. Our team has an extensive track record in acceptability and usability testing [
, , ]. We will be employing a user-centered, participatory design and testing methodology, and widely adopted usability and acceptance scales, such as the system usability scale (SUS) and the technology acceptance model (TAM) [ , ]. The app will be tested by including non–English-speaking caregivers and the stakeholder team in the initial design, performing prototype and real-world testing, and eventually gathering their feedback through semi-structured interviews or surveys. Technical feasibility will be evaluated through internal assessment of the extent of interoperability with EHR and scalability with AWS.The user-centered, participatory design protocol involves all stakeholders who will be invited to participate in design sessions. The initial meeting will aim to identify the problem space, current practice, needs, and expectations. An interactive design session is planned to capture stakeholder thoughts using words, color codes, and drawing boards. These projective tools are aimed to collect rich information and feedback from the participants [
]. Follow-up meetings will be held to communicate the initial design of the app prototype and to get feedback. The last session will be a testing session during which stakeholders and non–English-speaking caregiver participants will test the prototype.Non–English-speaking caregiver participants (n=20) will be recruited from caregivers of NCH patients via email, phone call, text message, or face-to-face communications. Eligibility criteria are (1) limited English-language proficiency, (2) Spanish as the primary language, and (3) having a child receiving care at the complex care clinic at NCH. Non–English-speaking caregiver participants will be offered a gift card of up to US $30 for their participation. Their feedback will be collected through an electronic survey or interview that is guided by usability and acceptance questions and guidelines informed by the SUS and the TAM [
, ]. Given the limitations of face-to-face meetings during the COVID-19 pandemic, our backup plan is to complete all meetings and testing online. Our plan is to use virtual drawing boards and interaction materials to communicate needs and expectations and to receive feedback. In addition, we may run the app on a webpage for users to interact through browsers on their computers or mobile phones.Overall, the expected participatory design and development timeline is 6 months. Throughout the study, we will periodically communicate with all stakeholders via email, SMS text messaging, or phone call to update them about development progress and get feedback when necessary. This study is exempt from institutional review board review.
Results
Thematic analysis will be used to identify, assess, and analyze patterns in the data [
]. We will record the audio and also keep meeting notes and observational notes during the sessions. The recorded audio will be transcribed and merged into a single document with observational notes that will be kept by at least one researcher. Thematic coding will be done by following Braun and Clarke’s thematic analysis guideline: (1) familiarizing ourselves with the data, (2) generating initial codes, (3) searching for themes, (4) reviewing and refining themes, (5) defining and naming themes, and (6) reporting the findings [ ]. The expected outcomes of our evaluation are qualitative input from all stakeholders and qualitative and quantitative feedback from non–English-speaking caregiver participants in response to usability and acceptability surveys. Eventually, our aim for the proposed app is to increase engagement and timely communication between caregivers and providers, which may lead to improved health outcomes [ ]. A tentative timeline is shared in .Month 1 | Month 2 | Month 3 | Month 4 | Month 5 | Month 6 | |
Kickoff meeting and planning | ✓ | |||||
Participatory design sessions | ✓ | ✓ | ✓ | |||
Development | ✓ | ✓ | ✓ | ✓ | ||
Maintenance | ✓ | ✓ | ||||
Testing | ✓ | ✓ | ||||
Reporting | ✓ |
Discussion
Principal Findings
The proposed solution should lead to better health outcomes in both the short term and the long term, especially for patients with chronic and/or complex conditions, by empowering and engaging patients and caregivers. By overcoming the language barriers between clinical teams and non–English-speaking patients or caregivers, the proposed solution will transform the way they can communicate and how they work together to achieve the best outcome for the patients. By making information more accessible and communication easier, non–English-speaking patients and caregivers will (1) be equipped with the correct information in their own language, (2) be able to more actively engage in clinical care decision making, (3) be more adherent to care plans, (4) carry out needed care activities outside of clinical settings, and (5) keep the clinical team informed.
The proposed solution could also lead to better health literacy. It helps caregivers to be more familiar and comfortable with medical terminology and health information, to develop a habit to document symptoms and issues, and to improve English-language proficiency. Communicating with clinical teams and revisiting these archived communication notes would help non–English-speaking caregivers to get familiar with communication language, as well as medical terms and concepts in care communications. Currently, in-person translation services may be lacking at providing transcription of communications, which may limit non–English-speaking caregivers to learning only during the engagement. The ability to access transcripts would facilitate gaining English-language proficiency. With the support of AWS medical vocabulary and increased accuracy in capturing medical terms, transcription would include accurate medical terms (such as diagnosis and medications), and would eventually help non–English-speaking caregivers to learn the correct spelling and pronunciation of medical terms and medication names (which has also been problematic in communications, especially during phone triaging). In the long term, this engagement would have a positive impact on the effectiveness of in-person communications with providers.
In addition to improving health outcomes and literacies for individual patients and caregivers, we believe this communication transformation could lead to more equitable care for non–English-speaking communities. These communities can be hard to reach for public health announcements and dissemination of general health information due to language and cultural barriers. They have a lower rate of patient portal sign-ups and may be unable to understand health-related information posted on health care organizations’ websites. This proposed solution will encourage their adoption of patient portals, allowing them to have meaningful communications with their health care organizations. In turn, health care organizations could leverage this communication channel to disseminate important health information. In future work, we aim to propose the use of the translation service over patient portals once we gather evidence on the feasibility and performance of the app.
The proposed solution could improve the accuracy of automated medical translation services in the long term. The app will be able to collect information about original and revised translations, corrected sentences and words, frequency of translations, and used and modified texts with timestamps. On the provider side, we will be able to label original translations by professional interpreters. The data will be used to train and test natural language processing and natural language understanding algorithms to improve the online translation services for Spanish and complex care communication. In future phases of this research, this algorithm will add a layer between AWS and the app—as a quality check and correction service maintained by the research team.
The proposed solution is planned to be scalable and affordable. To patients and caregivers, the estimated cost is anticipated to be relatively low compared with conventional methods of translation, only requiring a download of a free mobile app on existing personal mobile phones and learning some intuitive functions of the app. Families may have an added expense related to their data plans. This solution requires minimal administrative, financial, clinical, and technical investments, thus offering greater accessibility. With the flexible back-end solutions, it can also be used by other patients and hospitals and is not limited to children with special health care needs or pediatric care. However, children with special health care needs and other patients with chronic illnesses may be the most motivated to adopt this tool and may benefit from it the most.
Challenges and Limitations
One of the main limitations with online translation services is accuracy. As mentioned previously, online translation could yield unintended translations with dire consequences in a clinical setting. To avoid that, we will translate the content back to the sender to validate if the content is accurate.
demonstrates the process and shows a wireframe of the app user interface for Spanish text. It will also be applicable for English text from providers in the EHR dashboard. The translation and verification process outlined in could take a couple of iterations if the statements are complex. To overcome this limitation, we plan to use professional interpreters to review translations on the provider side. However, non–English-speaking caregivers and providers may need to simplify communication text to increase the accuracy of translations. We aim to improve translation services in the long term as we collect more data. We plan to keep records of translations for future efforts to develop and improve artificial intelligence algorithms in language understanding and app translations in complex care. The target population is assumed to have limited English-language proficiency and adequate literacy to engage with the app. To reduce any medicolegal risk, messages sent to providers will include a notice regarding the potential risk of translation errors and a suggestion to include an interpreter service when necessary. For non–English-speaking caregivers, users will be informed about verified and unverified messages and potential errors in translations.Further research is planned to be inclusive of low-literacy non–English-speaking caregivers through voice assistant–based engagement. The current proposal includes voice interaction in message entry only. Future implementations would include partner hospitals and organizations, as well as public and federal funding agencies to improve the solution, scale the deployment, and sustain funding. Eventually, we plan to make the app open source and to work with commercialization support offices to plan for long-term sustainability of the app.
Acknowledgments
We acknowledge Tyler Leiser and our advisory board members for their support and feedback. This project received a Phase 1 prize from the Agency for Healthcare Research and Quality’s Digital Solutions to Support Care Transitions Challenge.
Conflicts of Interest
None declared.
Consumer-level medical diary apps.
DOCX File , 22 KBReferences
- Diamond LC, Schenker Y, Curry L, Bradley EH, Fernandez A. Getting by: underuse of interpreters by resident physicians. J Gen Intern Med 2009 Feb;24(2):256-262 [FREE Full text] [CrossRef] [Medline]
- Villarruel AM, Portillo CJ, Kane P. Communicating with limited English proficiency persons: implications for nursing practice. Nurs Outlook 1999;47(6):262-270. [CrossRef] [Medline]
- Using interpreter Services: Victorian Government guidelines on policy and procedures. Department of Premier and Cabinet. Melbourne: Victorian Multicultural Commission; 2019. URL: https://www.multicultural.vic.gov.au/images/2017/UsingInterpretingServicesVictorianGovernmentGuidelines.pdf [accessed 2020-06-06]
- Cowley S, Houston A. A structured health needs assessment tool: acceptability and effectiveness for health visiting. Journal of advanced nursing 2003;43(1):92. [CrossRef]
- Rao DV, Warburton J, Bartlett H. Health and social needs of older Australians from culturally and linguistically diverse backgrounds: issues and implications. Australas J Ageing 2006 Dec;25(4):174-179. [CrossRef]
- Velasquez D, Uppal N, Perez N. Equitable Access To Health Information For Non-English Speakers Amidst The Novel Coronavirus Pandemic. Health affairs 2020. [CrossRef]
- Exploring Barriers and Facilitators to the Use of Qualified Interpreters in Health. The Victorian Foundation Foundation for Survivors of Torture. Melbourne: Foundation House; 2012. URL: https://refugeehealthnetwork.org.au/wp-content/uploads/Interpreters+in+health+discussion+paper+FINAL.pdf [accessed 2020-06-06]
- O'Hagan M. Massively open translation: unpacking the relationship between technology and translation in the 21st Century. International Journal of Communication 2016;10:929-946 [FREE Full text]
- Chang DTS, Thyer IA, Hayne D, Katz DJ. Using mobile technology to overcome language barriers in medicine. Ann R Coll Surg Engl 2014 Sep;96(6):e23-e25 [FREE Full text] [CrossRef] [Medline]
- Kaliyadan F, Gopinathan Pillai S. The use of Google language tools as an interpretation aid in cross-cultural doctor-patient interaction: a pilot study. Informatics in Primary Care 2010;18(2):141-143. [CrossRef]
- Patil S, Davies P. Use of Google Translate in medical communication: evaluation of accuracy. BMJ 2014;349(g7392). [CrossRef]
- Beh THK, Canty DJ. English and Mandarin translation using Google Translate software for pre-anaesthetic consultation. Anaesth Intensive Care 2015 Nov;43(6):792-793 [FREE Full text] [Medline]
- Panayiotou A, Gardner A, Williams S, Zucchi E, Mascitti-Meuter M, Goh AM, et al. Language Translation Apps in Health Care Settings: Expert Opinion. JMIR Mhealth Uhealth 2019 Apr 09;7(4):e11316 [FREE Full text] [CrossRef] [Medline]
- Khander A, Farag S, Chen KT. Identification and Evaluation of Medical Translator Mobile Applications Using an Adapted APPLICATIONS Scoring System. Telemedicine and e-Health 2018 Aug;24(8):594-603. [CrossRef]
- Sezgin E, Weiler M, Weiler A, Lin S. Proposing an Ecosystem of Digital Health Solutions for Teens With Chronic Conditions Transitioning to Self-Management and Independence: Exploratory Qualitative Study. J Med Internet Res 2018 Sep 06;20(9):e10285. [CrossRef]
- HRSA. 2019 HRSA MCHB Care coordination challenge winners. 2019. URL: https://mchbgrandchallenges.hrsa.gov/challenges/care-coordination-cshcn/winners [accessed 2020-06-06]
- Sezgin E, Noritz G, Elek A, Conkol K, Rust S, Bailey M, et al. Capturing At-Home Health and Care Information for Children With Medical Complexity Using Voice Interactive Technologies: Multi-Stakeholder Viewpoint. J Med Internet Res 2020 Feb 13;22(2):e14202. [CrossRef]
- Amazon Web Services. Amazon Transcribe Medical. URL: https://aws.amazon.com/transcribe/medical/ [accessed 2020-06-06]
- Amazon Web Services. Amazon Translate. URL: https://aws.amazon.com/translate/faqs/ [accessed 2020-06-06]
- DeForte S, Sezgin E, Huefner J, Lucius S, Luna J, Satyapriya A, et al. Usability of a Mobile App for Improving Literacy in Children With Hearing Impairment: Focus Group Study. JMIR Hum Factors 2020 May 28;7(2):e16310 [FREE Full text] [CrossRef] [Medline]
- Militello L, Sezgin E, Huang Y, Lin S. Delivering Perinatal Health Information via a Voice Interactive App: A Mixed Method Study. JMIR Preprints. 13/02/ 2020. [CrossRef]
- Lewis J, Sauro J. The factor structure of the system usability scale. 2009 Presented at: International conference on human centered design; 2009 Jul 19; Berlin, Heidelberg p. 94-103. [CrossRef]
- Davis FD. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 1989 Sep;13(3):319. [CrossRef]
- Sanders L. In: Stappers PJ, editor. Convivial design toolbox: generative research for the front end of design. Amsterdam: BIS; 2012.
- Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology 2006 Jan;3(2):77-101. [CrossRef]
Abbreviations
AWS: Amazon Web Services |
EHR: electronic health record |
FHIR: Fast Healthcare Interoperability Resources |
iOS: iPhone operating system |
IT: information technology |
NCH: Nationwide Children’s Hospital |
SUS: system usability scale |
TAM: technology acceptance model |
Edited by G Eysenbach; submitted 04.06.20; peer-reviewed by T Jadczyk, J Ayre; comments to author 29.06.20; revised version received 24.08.20; accepted 04.09.20; published 14.10.20
Copyright©Emre Sezgin, Garey Noritz, Jeffrey Hoffman, Yungui Huang. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 14.10.2020.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.