Background: Homecare settings across the United States provide care to more than 5 million patients every year. About one in five homecare patients are rehospitalized during the homecare episode, with up to two-thirds of these rehospitalizations occurring within the first 2 weeks of services. Timely allocation of homecare services might prevent a significant portion of these rehospitalizations. The first homecare nursing visit is one of the most critical steps of the homecare episode. This visit includes an assessment of the patient’s capacity for self-care, medication reconciliation, an examination of the home environment, and a discussion regarding whether a caregiver is present. Hence, appropriate timing of the first visit is crucial, especially for patients with urgent health care needs. However, nurses often have limited and inaccurate information about incoming patients, and patient priority decisions vary significantly between nurses. We developed an innovative decision support tool called Priority for the First Nursing Visit Tool (PREVENT) to assist nurses in prioritizing patients in need of immediate first homecare nursing visits.
Objective: This study aims to evaluate the effectiveness of the PREVENT tool on process and patient outcomes and to examine the reach, adoption, and implementation of PREVENT.
Methods: Employing a pre-post design, survival analysis, and logistic regression with propensity score matching analysis, we will test the following hypotheses: compared with not using the tool in the preintervention phase, when homecare clinicians use the PREVENT tool, high-risk patients in the intervention phase will (1) receive more timely first homecare visits and (2) have decreased incidence of rehospitalization and have decreased emergency department use within 60 days. Reach, adoption, and implementation will be assessed using mixed methods including homecare admission staff interviews, think-aloud observations, and analysis of staffing and other relevant data.
Results: The study research protocol was approved by the institutional review board in October 2019. PREVENT is currently being integrated into the electronic health records at the participating study sites. Data collection is planned to start in early 2021.
Conclusions: Mixed methods will enable us to gain an in-depth understanding of the complex socio-technological aspects of the hospital to homecare transition. The results have the potential to (1) influence the standardization and individualization of nurse decision making through the use of cutting-edge technology and (2) improve patient outcomes in the understudied homecare setting.
Trial Registration: ClinicalTrials.gov NCT04136951; https://clinicaltrials.gov/ct2/show/NCT04136951
International Registered Report Identifier (IRRID): PRR1-10.2196/20184
Each year, more than 5 million patients are admitted to the approximately 12,000 homecare agencies across the United States [- ]. Registered nurses (RNs) make several critical decisions before and during homecare admission, including identifying patients at risk for poor outcomes who might benefit from early interventions. However, there are no rigorously developed standards to assist in making these important decisions.
In this study, we focus on patients admitted to homecare from hospitals because their recent acute exacerbation makes them likely to be in urgent need of timely care, and they constitute up to 70% of all homecare recipients [, ]. Nationwide evidence shows that about 20% of homecare patients are rehospitalized during the up to 60-day homecare episode [ , ], with as many as 68% of these rehospitalizations occurring within the first 2 weeks of services [ , - ]. The Medicare regulations require a visit within 48 hours of referral to homecare, but for many patients, this may be too late. A growing body of evidence shows that a significant portion of rehospitalizations may be prevented by timely and appropriately targeted homecare services [ , , , ]. Patient prioritization at the time of transition to homecare services is a critical key to better patient outcomes.
The first homecare visit, usually conducted by an RN, is one of the most critical steps of the homecare episode [, ]. This start of care visit includes an assessment of the patient’s capacity for self-care, medication reconciliation, an examination of the home environment, and a discussion regarding whether a caregiver is present and able to help. A unique care plan is created based on this evaluation of the patient’s needs [ ]. Hence, appropriate timing of the first visit is crucial, especially for patients with urgent health care needs. However, recent research by our team showed that nurses have very limited and often inaccurate information about incoming patients due to lack of information exchanges between care settings and lack of standards about the necessary information needed for patient prioritization [ - ]. Often operating with limited information, resources, and time [ ], nurses must decide how to prioritize the patient’s first homecare visit.
Some studies have examined solutions for improving patient prioritization in a postacute care setting. For example, one study from Canada developed a method called Method for Assigning Priority Levels (MAPLe) to assist case managers in determining the relative priority that should be attached to patients when postacute care referrals are made . However, the MAPLe system is not specific to homecare, and it prioritizes patients irrespective of the care setting. Another study from the United States has developed a tool that helps prioritize patients discharged to skilled nursing facilities [ ]. None of the existing studies have developed prioritization tools specific to homecare.
Our team has developed an innovative clinical decision support system (CDSS) called Priority for the First Nursing Visit Tool (PREVENT) to assist nurses in prioritizing patients in need of immediate first homecare nursing visits . PREVENT was developed with rigor, using a strong theoretical foundation (transition theory) [ ] and methodology for eliciting experts’ decisions to create clinical decision support tools [ ]. PREVENT was constructed using data mining, regression modeling, and expert homecare nurses’ ratings of example patients who were transitioned from hospital to homecare. The goal was to identify key patient characteristics that are essential to support early homecare admission decision making. Overall, more than 70 patient demographic and clinical characteristics (eg, comorbidities, level and availability of social support, and detailed functional status) were considered for inclusion in the final prediction model from which PREVENT was developed. The final PREVENT CDSS uses 5 factors (including the number of medications, number of comorbid conditions, presence of a wound, presence of a comorbid condition of depression, and patient’s functional status) to produce a recommendation on whether a specific homecare patient should be prioritized for the first homecare nursing visit. See [ - , - ] for more information about the methods for PREVENT CDSS development.
We completed a pilot efficacy study  to measure the efficacy of PREVENT, conducted at a large urban hospital in Brooklyn, New York. In collaboration with the Visiting Nurse Service of New York (VNSNY), we enrolled 176 patients admitted to homecare from the hospital during April and May 2016. In the control phase (n=90 patients), we calculated the PREVENT priority score but did not share the score with the homecare admission staff who influence visit scheduling. In the experimental phase, the PREVENT score was shared with the homecare admission staff (n=86 patients). During this phase, patients identified as high priority received their first homecare nursing visit about a half-day sooner as compared with the control phase (1.8 days vs 2.2 days; P=.09). Rehospitalizations from homecare decreased by almost 50% (9.4% point reduction) when comparing the control (21.1%) and experimental phases (11.7%), with a significant difference between the rehospitalization (survival analysis) curves (log-rank P =.03). We acknowledge that this pilot study had a relatively small sample size and potentially insufficient adjustment for background variables. However, these results were promising in that high-priority patients received their first homecare visit sooner and overall rehospitalization rates were lower.
This manuscript presents our methodology for examining PREVENT’s impact on patient outcomes via a larger and more rigorous effectiveness trial. Our specific study aims are as follows:
- Evaluate the effectiveness of the PREVENT tool on process and patient outcomes. We will test the following hypotheses using survival analysis and logistic regression with propensity score matching: compared with not using the tool in the preintervention phase, when homecare clinicians use the PREVENT tool, high-risk patients in the intervention phase will receive more timely first homecare visits and have decreased incidence of rehospitalization and have decreased emergency department use within 60 days of hospital discharge.
- Explore PREVENT’s reach and adoption by the homecare admission staff and describe the tool’s implementation during homecare admission. Aim 2 will be assessed using mixed methods incorporating homecare admission staff interviews, think-aloud simulations [ ], and analysis of staffing and other relevant data.
Mixed Method Approach
We are using an embedded mixed methods design. We will conduct a pre- and postintervention trial of PREVENT’s integration into clinical practice using homecare admissions from two New York City urban hospitals serving diverse racial and ethnic populations. We will use quantitative methods, including logistic regression and survival analysis, to evaluate the effects of the tool on process and patient outcomes. We will utilize qualitative methods integrated with quantitative methods to gain an in-depth insight into technology adoption and implementation.
On the basis of our consultations with New York-Presbyterian (NYP) hospitals’ leadership and our goal of exploring the effectiveness of the PREVENT system in different settings and among sites serving an ethnically diverse population, we will conduct the study at 2 NYP hospitals: (1) NYP Hospital/Columbia University Irving Medical Center (large academic medical center), a 745-bed adult academic medical center providing emergency, primary, and specialty care in all the major fields of medicine, and (2) NYP Allen Hospital (small community hospital), a 196-bed community hospital serving northern Manhattan, Riverdale, and other communities in the Bronx. As a homecare site, we will use VNSNY—the largest not-for-profit home health agency in the United States serving up to 48,500 patients and health plan members daily.
The study is guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework. The RE-AIM was adopted  to specifically focus on real-world implementation of clinical informatics interventions, such as CDSS. Since its adoption, RE-AIM has been successfully applied in numerous studies of health information technology evaluations [ ]. The components of this framework are as follows: reach of the intervention to a representative proportion of the target population, effectiveness of the intervention, adoption of the intervention across a broad and representative proportion of settings, implementation details, and maintenance of the intervention after implementation. In each phase of implementation, we will use the mixed methods approach for data collection and analysis to identify barriers and facilitators of the PREVENT implementation. summarizes the RE-AIM dimensions with the associated definitions of each component, questions that need to be addressed in each component, and the associated analytic methods to answer those questions.
|RE-AIMa dimension and definition||Questions relevant to this study and aims addressed||Analytic methods||Data sources|
|Reach (individual level): Absolute number, proportion, and representativeness of individuals who participate in the intervention.||1. Were patients for whom PREVENTb was calculated representative of all the eligible patients? (Aim 2)||Quantitative analysis: Comparison of patient demographic and clinical characteristics within the study (preintervention vs intervention groups), and between the study period and annual data (method: t tests or chi-square tests, when applicable).||VNSNYc data repository (including OASISd) |
|Individuals who participate in the intervention.||2. What proportion of admission staff eligible to use the PREVENT actually used it? (Aim 2)||Quantitative analysis: Estimation of how many admission staff used PREVENT out of total eligible admission staff members.||Data from VNSNY admission units|
|Efficacy or effectiveness (individual level): Impact of an intervention on important outcomes, including potential negative effects.||3. What is the effect of PREVENT on process outcomes? (Aim 1)||Quantitative analysis: Estimation of PREVENT’s effect on timing of the first homecare visit (method: survival analysis).||VNSNY data repository (including OASIS)|
|Efficacy or effectiveness (individual level)||4. What is the effect of PREVENT on patient outcomes? (Aim 1)||Quantitative analysis: Estimation of PREVENT’s effect on incidence and time to rehospitalization and EDe use within 30 and 60 days (method: logistic regression and survival analysis).||VNSNY data repository (including OASIS) +RHIOf (health service use data in New York)|
|Adoption (setting and/or organizational level): Absolute number, proportion, and representativeness of settings and intervention agents (people who deliver the program) who are willing to initiate a program.||5. What are the characteristics of the settings that decided to adopt PREVENT? (Aim 2)||Quantitative analysis: Description of the VNSNY and two referring hospitals in terms of location, staffing, and patient population (method: descriptive summary).||VNSNY staffing data and hospitals staffing data; VNSNY data repository|
|Adoption (setting and/or organizational level)||6. How well did the goals of PREVENT fit with the values and expectations of the practice settings? (Aim 2)||Qualitative analysis: Identification of the strategic plans and potential other incentives (such as CMSg readmission reduction program) related to PREVENT (method: descriptive summary).||VNSNY and NYPh strategic plans; information about potential other incentives|
|Implementation (setting and/or organizational level): Setting level—intervention agents’ fidelity to the various elements of an intervention’s protocol, including consistency of delivery as intended and the time and cost of the intervention; individual level—clients’ use of the intervention strategies.||7. How many admission staff members used PREVENT? (Aim 2)||Quantitative analysis: Description of the number of admission staff members who used PREVENT (method: descriptive statistics).||VNSNY data repository|
|Implementation (setting and/or organizational level)||8. What was the user satisfaction with PREVENT? (Aim 2)||Quantitative analysis (method: summary statistics).||Administration of the 12 item End-User Computing Satisfaction Instrument|
|Implementation (setting and/or organizational level)||9. Did users perceive PREVENT as easy to use? (Aim 2)||Qualitative analysis (method: thematic analysis).||Postintervention interviews|
|Implementation (setting and/or organizational level)||10. Did users perceive PREVENT as useful? (Aim 2)||Qualitative analysis (method: thematic analysis).||Postintervention interviews|
|Implementation (setting and/or organizational level)||11. Was there sufficient leadership support? (Aim 2)||Qualitative analysis (method: thematic analysis).||Administration of open-ended questions during think-aloud simulations and postintervention interviews|
|Implementation (setting and/or organizational level)||12. What workflow adjustments needed to be made to streamline PREVENT into routines of daily clinical practice? (Aim 2)||Qualitative analysis (method: thematic analysis).||Research team experience; administration of open-ended questions during think-aloud simulations and postintervention interviews|
|Implementation (setting and/or organizational level)||13. What support, resources, and outside collaborations were needed to implement PREVENT? (Aim 2)||Qualitative analysis (method: thematic analysis).||Research team experience; agency description of PREVENT electronic integration process|
|Implementation (setting and/or organizational level)||14. What technical infrastructure was required to implement the CDSSi? (Aim 2)||Qualitative analysis (method: thematic analysis).||Research team experience; agency description of PREVENT electronic integration process|
|Implementation (setting and/or organizational level)||15. What user training and support services were needed by PREVENT users? (Aim 2)||Qualitative analysis (method: thematic analysis).||Research team experience; data from think-aloud simulations and postintervention interviews|
|Implementation (setting and/or organizational level)||16. What were the potential barriers to successful PREVENT implementation and how were they addressed? (Aim 2)||Qualitative analysis (method: thematic analysis).||Research team experience; administration of open-ended questions during think-aloud simulations and postintervention interviews|
|Implementation (setting and/or organizational level)||17. Did homecare admission staff agree with PREVENT recommendations? (Aim 2)||Quantitative analysis: Comparison of PREVENT recommendations with actual homecare admission staff decisions to provide priority visits and comparison of patient characteristics between cases that disagree versus agree with PREVENT recommendations (method: t tests or chi-square tests, when applicable).||Service use and patient data from VNSNY data repository|
|Implementation (setting and/or organizational level)||18. Did homecare admission staff agree with PREVENT recommendations? (Aim 2)||Qualitative analysis (method: thematic analysis).||Administration of open-ended questions during think-aloud simulations and postintervention interviews and follow-up calls to assess disagreements|
|Maintenance: Setting level—extent to which intervention becomes institutionalized or part of the routine organizational practices; individual level—long-term effects of a program on outcomes for 6 or more months after the most recent intervention contact.||Beyond the scope of this study||Beyond the scope of this study||Beyond the scope of this study|
aRE-AIM: Reach, Effectiveness, Adoption, Implementation, and Maintenance.
bPREVENT: Priority for the First Nursing Visit Tool.
cVNSNY: Visiting Nurse Service of New York.
dOASIS: Outcome and Assessment Information Set.
eED: emergency department.
fRHIO: Regional Health Information Organization.
gCMS: Centers for Medicare & Medicaid Services
hNYP: New York-Presbyterian.
iCDSS: clinical decision support system.
Study Intervention: PREVENT
The PREVENT tool will be integrated with the hospitals’ electronic health record (EHR) via a locally developed system called iNYP, which integrates with the EHR and provides advanced data review capabilities of all EHR data. iNYP is a Java-based service-oriented web app that builds on Columbia University’s 25-year history of clinical information system innovation [, ]. iNYP is available as a custom tab within the commercial hospital EHR (supplementing the native results review capabilities) and is also accessible from a web browser or a mobile device. iNYP is widely used by most clinicians alongside the EHR, including the homecare admission staff. The PREVENT score will be calculated automatically from EHR data that populate the patient discharge summary or other parts of the EHR. We have cross-mapped the elements (eg, number of medications and comorbid conditions) needed for the calculation of the PREVENT score to confirm that the required elements are readily available in the EHR system. VNSNY admission staff will receive an auto-populated field within the homecare referral containing the PREVENT recommendation about visit priority, presented as high priority and medium or low priority. Before any data collection, we will test the accuracy of the PREVENT score on the first 50 priority calculations and correct the EHR integration if any mistakes are found.
Standard VNSNY Patient Admission Workflow
During our preliminary work, we determined that the scheduling and assignment unit assumes responsibility for patient admission to the VNSNY. The unit comprises several admission staff members who are involved in the admission processes, including intake coordinators, clinical associate managers, and schedulers. Homecare admission starts with standard homecare referral signed by the referring physician.
The referrals are passed to the intake coordinators (administrative staff) who enter the referral information into the VNSNY EHR system. Next, clinical field managers give patients a welcome call and coordinate the general start of care dates. After that, schedulers identify the date of a first homecare nursing visit. Each geographic location (based on city boroughs and street addresses) is served by several admission staff members.
The workflow of PREVENT implementation consists of 3 phases: preintervention phase, intervention phase, and postintervention phase.provides an overview of these phases.
During this phase of the study, 3 research activities will be implemented. First, the PREVENT priority score will be automatically calculated for all the patients referred to VNSNY from the 2 hospitals (step 1). Second, the PREVENT score (and priority recommendation based on the score) will be collected but not shared with homecare admission staff over about 3 months (see the Sample Size Calculation section; step 2). Third, after the preintervention phase data are collected, the study team will conduct several 30-min educational sessions for the admission staff about the development and validation of PREVENT and this study (step 3). We will work with the VNSNY scheduling and assignment unit management to identify all the VNSNY staff eligible (15-20 staff) to be exposed to PREVENT’s recommendations during the study. We will ensure that each eligible admission staff member undergoes at least one educational session about the study workflow.
To minimize periodical and time effects, the intervention phase will start at both hospitals on the same date. The PREVENT recommendation will be shared with the homecare intake coordinators for about 3 months (step 4). The intake coordinator will enter the PREVENT recommendation into the special recommendations field of the VNSNY EHR system. This field stores information about any special programs or services patients should receive in homecare, such as recommendations for frontloading of visits. Next, clinical field managers and schedulers will incorporate the PREVENT priority recommendations in their processes related to visit scheduling and patient prioritization. The field clinician will then conduct the first nursing visit. For cases where patient prioritization was not possible, we will ask the admission staff to document why a priority visit could not happen (such as the patient refused or short staffing; step 5).
After completion of data collection, we will continue to share the results of the PREVENT recommendation for approximately 4-6 weeks. During this period, four evaluation activities will be conducted. First, we will use the Postintervention simulation guide (see Study Instruments section) to conduct think-aloud simulations with admission staff who were exposed to the CDSS (step 6). Second, concurrently with the think-aloud simulations, we will continuously assess cases where admission staff did not implement PREVENT recommendations (step 7). We will compare patient characteristics between cases that disagreed versus agreed with PREVENT recommendations. We want to identify additional factors that PREVENT might have missed when assessing a patient’s priority for a visit. Once saturation in responses is achieved and we have a solid number of cases for quantitative disagreement comparisons (about 100 cases of disagreement), we will discontinue sharing the PREVENT recommendation. Third, we will anonymously distribute the End-User Computing Satisfaction Instrument using a secure web-based survey platform (Qualtrics ; step 8). Fourth, we will conduct in-person interviews using the Postintervention phase interview guide with the same respondents who participated in the observations (step 9).
Qualitative interviews and think-aloud simulations will be guided by two robust interview guides we will develop for this study. The guides will incorporate aspects of the RE-AIM framework dimensions as questions. The guides are as follows: (1) postintervention simulation guide (think-aloud protocol) and (2) postintervention phase interview guide. Each interview guide will include semi-structured open-ended questions to be answered by the admission staff. The Postintervention phase interview guide will include questions about PREVENT’s perceived usability and ease of use, leadership support, workflow adjustments, adequacy of training sessions, and barriers to implementation such as any changes the respondent made to his or her regular workflow to use PREVENT’s recommendations.
The End-User Computing Satisfaction Instrument [- ] will be used to quantitatively measure satisfaction. The 12 item instrument measures concepts such as accuracy and ease of use and has been used to evaluate many types of applications, including decision support. A score of 54 corresponds to the 70th percentile. Any concept scoring less than the 70th percentile from either user group will guide future tool revision. See [ - ] for more information on the study instruments.
Methods for Study Aim 1: Evaluate the Effects of PREVENT on Process and Patient Outcomes
Sample Size Calculation
In this study, we will calculate the PREVENT scores for all patients referred to VNSNY from the 2 hospitals (see study setting) in a 3 month period during the preintervention phase (scores not shared) and a 3 month intervention phase (scores shared) for an estimated total of 2094 patients and 1508 high-priority patients, respectively. This calculation is based on the pilot study rehospitalization decrease.presents the minimum detectable difference between high-priority intervention group patients and preintervention high-priority patients.
Patient characteristics will be extracted from the Outcome and Assessment Information Set (OASIS), a standardized assessment tool designed to collect nearly 100 items related to a recipient’s functional status, clinical status, and service needs during a homecare episode. Mandated by CMS since 1999, OASIS is the most comprehensive national data set for homecare patient assessment and outcomes. Data were collected upon admission, every 60 days, if transferred to an inpatient facility, and at discharge. OASIS data will be extracted from the VNSNY EHR and supplemented with additional data such as language spoken and residence county.shows the details of the variables and the sources of the data.
The first visit timing will be extracted from the VNSNY administrative database. In this study, we will examine the impact of PREVENT’s recommendation on the time from the patient’s hospital discharge to the first homecare nursing visit.
|Socio-economic factors||Age, sex, race and ethnicity, language, caregiver support, living arrangements, language spoken, and residence county||OASISa/VNSNYb EHRc|
|Clinical factors||Primary and background diagnoses, limitations in functioning, cognitive status, depressive symptoms, behavioral problems, wounds, pain, sensory status, elimination status, and medications||OASIS/VNSNY EHR|
|Health outcomes||Time from hospital discharge to first homecare visit, rehospitalizations, and EDd visits||Regional Health Information organization/ VNSNY EHR|
aOASIS: Outcome and Assessment Information Set.
bVNSNY: Visiting Nurse Service of New York.
cEHR: electronic health record.
dED: emergency department.
Most often when patients are rehospitalized, they return to the same facility as the initial hospitalization. In this study, it would be the NYP system. We will have access to these data. Seefor more information about rehospitalization data.
Quantitative Analysis Methods
We will use survival analysis methods and logistic regression to estimate the effect of the CDSS on process and patient outcomes. As this is an observational study, we will also conduct propensity score matching (1:1 matching) of high-risk patients between the preintervention and intervention phases using the greedy nearest neighbor algorithm [, ]. See [ , ] for more information about the quantitative methods used in this study.
Methods for Study Aim 2: Explore PREVENT’s Reach and Adoption by the Homecare Admission Staff and Describe the Tool’s Implementation During Homecare Admission
To explore PREVENT’s reach and adoption of the proposed study, we will match qualitative and quantitative analyses of the collected data to gain an in-depth understanding of the topic.provides a description of the RE-AIM aspects, specific questions, analytical methods, and data sources for these analyses.
Think-aloud methodology is a standard approach to elicit data about cognitive reasoning that occurs during a problem-solving task [, ]. The think-aloud method will help answer questions about operational support (ie, 11-12, 15-16, and 18), as specified in . This methodology will be implemented to observe patient admission in a simulated environment with 20 comprehensive and diverse case scenarios of patient admissions (10 high-priority cases and 10 low or medium-priority cases). See [ , ] for more information about the think-aloud methodology.
Qualitative Analysis Method
Thematic analysis is a qualitative descriptive approach for identifying, analyzing, and reporting themes within data [, ]. Researchers will use this approach to analyze data collected during the postintervention interviews and think-aloud simulations. The study research assistant will transcribe each interview into a text file. A different member of the study team will validate portions of transcriptions (20-30%) for quality. We will use qualitative analysis software (NVivio [ ]) to implement the analysis. See [ - ] for more information.
Mixed Methods Analysis
This mixed methods analysis will match qualitative (for exploring the PREVENT’s reach and adoption) and quantitative (to evaluate the effects of PREVENT on process and patient outcomes) findings to provide context so as to gain an in-depth understanding of our experimental results. The qualitative findings will supplement the quantitative findings. For example, to understand aspects of CDSS implementation related to PREVENT’s use and user satisfaction, we will match findings from qualitative questions #9 and 10 () with findings from quantitative questions #7 and 8. Similarly, to understand aspects of CDSS adoption related to setting characteristics and strategic alignment of PREVENT with the setting’s goals, we will match the findings from questions #5 and 6.
The study research protocol was approved by the institutional review board in October 2019. Currently, the study team is working on integrating the PREVENT CDSS into hospital and homecare EHRs. The study team is also analyzing information on patients who are currently being admitted to the VNSNY to create a framework for statistical analysis of this study’s findings. These activities are necessary to conduct study aims 1 and 2. Data collection is planned to start in early 2021.
In this study, we introduced a rigorous methodology for evaluating the implementation of an innovative CDSS, PREVENT, which was developed to assist in determining which patients should be prioritized for the first homecare nursing visit  (more details on the PREVENT tool are presented in ). This methodology was built on the RE-AIM framework and mixed methods approaches, incorporating homecare admission staff interviews, think-aloud simulations [ ], and analysis of staffing and other relevant data. By following the methodology’s steps, we will be able to explore the reach and adoption of PREVENT by the homecare admission staff; describe the implementation phase; identify the potential barriers for implementation; and improve the perceived value (usefulness) and familiarity (ease of use) of PREVENT from the schedulers’, clinicians’, and administrative staffs’ perspectives. In addition, by exploring the technical infrastructure and the process of clinician decision making, we will be able to adjust the workflow and smoothly integrate PREVENT with the EHR system for automated computation of the first homecare nursing visit priority for individual patients.
Overall, our approach for the evaluation of PREVENT as a CDSS implementation consists of 3 major phases: preintervention, intervention, and postintervention phases. It is crucial to measure the efficacy of the CDSS in improving health care outcomes using controlled experimental methods. In the preintervention phase, we will identify the eligible staff for exposure to the CDSS and conduct training to educate them about using the CDSS. The research team will also explore and evaluate the existing infrastructure and workflow of decision making and prepare a rigorous plan for the integration of the CDSS with the clinical workflow.
In the intervention phase, the trained admission staff will be exposed to the CDSS intervention. Monitoring and tracking the staff’s use of the CDSS recommendations and the extent to which the recommendations are overridden is essential to develop appropriate mechanisms to measure CDSS usability and adjust the infrastructure and workflow to meet the needs of administrative staff.
For the postintervention phase, the core step is to evaluate the implementation of the CDSS using appropriate instruments and tools to generate a clear picture of the barriers and facilitators for CDSS uptake and effectiveness. Tools such as qualitative interviews, think-aloud simulations, and End-User Computing Satisfaction Instrument will help the research team measure staff satisfaction and agreement with the CDSS and perceived CDSS usability and ease of use. Other important factors to be evaluated in this phase are access to resources and adjustment of workflow and technical infrastructure.
Strengths, Limitations, and Alternatives to the Methodology
We considered several options for the study design. We could have used a stepped wedge cluster randomized trial . One prerequisite for such a design is the ability to randomize the intervention among clearly defined clusters of hospital units or homecare agencies. In this study, homecare admission staff consists of 3 types of professionals (intake coordinators, clinical associate managers, and schedulers), some of whom work together on a constant basis in the same office. Thus, isolating selected admission staff is challenging, with a high risk of contamination.
Another option was to conduct a randomized controlled trial of PREVENT. This study design would require randomization at the patient level such that PREVENT recommendations would be shared for half of the randomly selected patients but not the others. Similar to the stepped wedge cluster design challenge, clear randomization would be unlikely due to contamination. The proposed quasi-experimental pre-post design (with propensity score matching) is similar to our pilot efficacy study  and addresses the contamination challenge of the abovementioned approaches. By matching patients in the preintervention and intervention phases based on their propensity score, we will account for potential differences in these 2 patient groups, which we were not always able to do in the pilot study. This design, guided by the RE-AIM framework, is preferable for an effectiveness study of real-world CDSS implementation.
Additional study limitations include the relatively limited generalizability of study findings, as the study involves 2 hospitals in a large academic hospital system in New York City. In addition, our ability to draw causal inferences is somewhat limited in quasi-experimental pre-post study designs.
Innovation and Impact
Our study is innovative in the seamless use of CDSS and patient-centeredness:
- Our work is focused on building and evaluating one of the first evidence-based CDSS for homecare in the United States. The majority of hospitals and many homecare agencies across the nation use some type of EHR [ ]. Tools such as PREVENT are becoming increasingly important in the effort to standardize care among agencies and avoid negative patient outcomes.
- For decades, the homecare industry has promoted providing the first nursing visit close to hospital discharge as an effective strategy to prevent hospitalizations. PREVENT will provide the first CDSS in homecare to assist agencies with implementing this important intervention.
- PREVENT will help highlight unique patient characteristics to support person-centered care. Our work is intended to change the paradigm in homecare by implementing 3 out of 4 key characteristics of the homecare agency of the future recently identified by the Institute of Medicine report The Future of Home Health Care [ ]. The 3 characteristics are the agency being patient and person centered, seamlessly connected, and technology enabled.
Study results may have the potential to (1) standardize and individualize nurse decision making by using cutting-edge technology and (2) improve patient outcomes in the understudied homecare setting.
Conclusions and Recommendations
This manuscript presents a protocol of a CDSS PREVENT study aimed at improving the outcomes of patients admitted to homecare services. We strongly encourage other researchers who study the effects of CDSS in clinical practice to apply similar mixed qualitative and quantitative methodologies in their studies. The application of mixed methods can enable researchers to gain an in-depth understanding of the complex socio-technological aspects of CDSS use in clinical practice. In turn, such comprehensive understanding can improve long-term effective use of CDSS in clinical settings.
Research reported in this publication was supported by the National Institute of Nursing Research of the National Institutes of Health under Award Number R01NR018831. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Conflicts of Interest
KB and MT are co-inventors of a decision support tool called “Priority for the First Nursing Visit Tool” (PREVENT), which under governing University policy is Clinical Trial intellectual property being studied in this research (“I Interest”). The intellectual property is assigned to the University of Pennsylvania.
Development of the Priority for the First Nursing Visit Tool.DOCX File , 16 KB
Study instruments.DOCX File , 15 KB
Minimum detectable difference between high priority intervention group patients versus pre-intervention high priority patients.DOCX File , 15 KB
Rehospitalization data.DOCX File , 18 KB
Quantitative analysis methods.DOCX File , 14 KB
Think-aloud methodology.DOCX File , 14 KB
Qualitative analysis methods.DOCX File , 15 KB
Peer-Review Reports from National Institute of Health.PDF File (Adobe PDF File), 250 KB
- The Medicare Payment Advisory Commission. Report to the Congress- Medicare Payment Policy. Washington, DC: Home Health Care Services; 2018.
- Basic Statistics About Home Care. 2010. URL: http://www.nahc.org/wp-content/uploads/2017/10/10hc_stats.pdf [accessed 2020-09-17]
- The Medicare Payment Advisory Commission. Home Health Care Services. 2018. URL: http://arooj.in/ [accessed 2020-09-17]
- O'Connor M, Hanlon A, Bowles KH. Impact of frontloading of skilled nursing visits on the incidence of 30-day hospital readmission. Geriatr Nurs 2014;35(2 Suppl):S37-S44 [FREE Full text] [CrossRef] [Medline]
- Home Health Compare. Centers for Medicare and Medicaid Services. 2016. URL: https://www.medicare.gov/homehealthcompare/search.html [accessed 2020-09-17]
- Rosati RJ, Huang L. Development and testing of an analytic model to identify home healthcare patients at risk for a hospitalization within the first 60 days of care. Home Health Care Serv Q 2007;26(4):21-36. [CrossRef] [Medline]
- Vasquez MS. Preventing Rehospitalization through effective home health nursing care. Prof Case Manag 2009;14(1):32-38. [CrossRef] [Medline]
- Murtaugh CM, Deb P, Zhu C, Peng TR, Barrón Y, Shah S, et al. Reducing readmissions among heart failure patients discharged to home health care: effectiveness of early and intensive nursing services and early physician follow-up. Health Serv Res 2017 Aug;52(4):1445-1472 [FREE Full text] [CrossRef] [Medline]
- Topaz M, Trifilio M, Maloney D, Bar-Bachar O, Bowles KH. Improving patient prioritization during hospital-homecare transition: a pilot study of a clinical decision support tool. Res Nurs Health 2018 Oct;41(5):440-447. [CrossRef] [Medline]
- Markley J, Sabharwal K, Wang Z, Bigbee C, Whitmire L. A community-wide quality improvement project on patient care transitions reduces 30-day hospital readmissions from home health agencies. Home Healthc Nurse 2012 Mar;30(3):E1-11. [CrossRef] [Medline]
- McDonald MV, King LJ, Moodie M, Feldman PH. Exploring diabetic care deficiencies and adverse events in home healthcare. J Healthc Qual 2008;30(6):5-12. [CrossRef] [Medline]
- Sockolow P, Wojciechowicz C, Holmberg A, Bass EJ, Potashnik S, Yang Y, et al. Home care admission information: what nurses need and what nurses have. A mixed methods study. Stud Health Technol Inform 2018;250:164-168. [Medline]
- Sockolow P, Bass EJ, Eberle CL, Bowles KH. Homecare nurses' decision-making during admission care planning. Stud Health Technol Inform 2016;225:28-32. [Medline]
- Irani E, Hirschman K, Cacchione P, Bowles K. Home health nurse decision-making regarding visit intensity planning for newly admitted patients: a qualitative descriptive study. Home Health Care Serv Q 2018;37(3):211-231 [FREE Full text] [CrossRef] [Medline]
- Jones C, Jones J, Richard A, Bowles K, Lahoff D, Boxer R, et al. 'Connecting the dots': a qualitative study of home health nurse perspectives on coordinating care for recently discharged patients. J Gen Intern Med 2017 Oct;32(10):1114-1121 [FREE Full text] [CrossRef] [Medline]
- Hirdes J, Poss J, Curtin-Telegdi N. The method for assigning priority levels (MAPLe): a new decision-support system for allocating home care resources. BMC Med 2008 Mar 26;6:9 [FREE Full text] [CrossRef] [Medline]
- Maly MB, Lawrence S, Jordan MK, Davies WJ, Weiss MJ, Deitrick L, et al. Prioritizing partners across the continuum. J Am Med Dir Assoc 2012 Nov;13(9):811-816. [CrossRef] [Medline]
- Topaz M. Developing a Tool to Support Decisions on Patient Prioritization at Admission to Home Health Care. University of Pennsylvania. 2014. URL: https://repository.upenn.edu/edissertations/1473/ [accessed 2020-09-17]
- Topaz M. The hitchhiker's guide to nursing informatics theory: using the data-knowledge-information-wisdom framework to guide informatics research. Online J Nurs Informatics 2013;17(3):-. [CrossRef]
- Bowles KH, Ratcliffe S, Potashnik S, Topaz M, Holmes J, Shih N, et al. Using electronic case summaries to elicit multi-disciplinary expert knowledge about referrals to post-acute care. Appl Clin Inform 2016;7(2):368-379 [FREE Full text] [CrossRef] [Medline]
- Hosmer D, Lemeshow S, Sturdivant R. Applied Logistic Regression. 3rd ed. Wiley 2013:-. [CrossRef]
- Ma C, Shang J, Miner S, Lennox L, Squires A. The prevalence, reasons, and risk factors for hospital readmissions among home health care patients: a systematic review. Home Health Care Manag Pract 2017 Nov 15;30(2):83-92. [CrossRef]
- O'Connor M. Hospitalization among medicare-reimbursed skilled home health recipients. Home Health Care Manag Pract 2012 Feb;24(1):27-37 [FREE Full text] [CrossRef] [Medline]
- Eccles D, Arsal G. The think aloud method: what is it and how do I use it? Qual Res Sport Exerc Heal 2017;9(4):514-531.
- Bakken S, Ruland CM. Translating clinical informatics interventions into routine clinical care: how can the RE-AIM framework help? J Am Med Inform Assoc 2009;16(6):889-897 [FREE Full text] [CrossRef] [Medline]
- Leveille SG, Walker J, Ralston JD, Ross SE, Elmore JG, Delbanco T. Evaluating the impact of patients' online access to doctors' visit notes: designing and executing the OpenNotes project. BMC Med Inform Decis Mak 2012 Apr 13;12:32 [FREE Full text] [CrossRef] [Medline]
- Tullai-McGuinness S, Madigan EA, Fortinsky RH. Validity testing the outcomes and assessment information set (OASIS). Home Health Care Serv Q 2009;28(1):45-57 [FREE Full text] [CrossRef] [Medline]
- Johnson S, Friedman C, Cimino JJ, Clark T, Hripcsak G, Clayton PD. Conceptual data model for a central patient database. Proc Annu Symp Comput Appl Med Care 1991:381-385 [FREE Full text] [Medline]
- Hripcsak G, Cimino JJ, Sengupta S. WebCIS: large scale deployment of a web-based clinical information system. Proc AMIA Symp 1999:804-808 [FREE Full text] [Medline]
- Qualitative Research Software. Qualtrics Research Core. 2020. URL: https://www.qualtrics.com/au/core-xm/ [accessed 2020-09-17]
- Doll WJ, Torkzadeh G. The measurement of end-user computing satisfaction. MIS Q 1988 Jun;12(2):259. [CrossRef]
- Doll WJ, Deng X, Raghunathan T, Torkzadeh G, Xia W. The meaning and measurement of user satisfaction: a multigroup invariance analysis of the end-user computing satisfaction instrument. J Manag Inf Syst 2014 Dec 8;21(1):227-262. [CrossRef]
- Doll WJ, Xia W, Torkzadeh G. A confirmatory factor analysis of the end-user computing satisfaction instrument. MIS Q 1994 Dec;18(4):453. [CrossRef]
- Austin PC. An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res 2011 May;46(3):399-424 [FREE Full text] [CrossRef] [Medline]
- Austin PC. A comparison of 12 algorithms for matching on the propensity score. Stat Med 2014 Mar 15;33(6):1057-1069 [FREE Full text] [CrossRef] [Medline]
- Someren MV. The Think Aloud Method: A Practical Guide to Modelling Cognitive Processes. New York, USA: Academic Press; 1994.
- Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
- Vaismoradi M, Turunen H, Bondas T. Content analysis and thematic analysis: implications for conducting a qualitative descriptive study. Nurs Health Sci 2013 Sep;15(3):398-405. [CrossRef] [Medline]
- NVivo Qualitative data Analysis Software. QSR International Pty Ltd. 2018. URL: https://www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home [accessed 2020-09-17]
- Byrne M. Data analysis strategies for qualitative research. AORN J 2001 Dec;74(6):904-905. [CrossRef] [Medline]
- Hemming K, Haines TP, Chilton PJ, Girling AJ, Lilford RJ. The stepped wedge cluster randomised trial: rationale, design, analysis, and reporting. Br Med J 2015 Feb 6;350:h391 [FREE Full text] [CrossRef] [Medline]
- Office ONCFIT. Adoption of Electronic Health Record Systems among U.S. Non-Federal Acute Care Hospitals. - 2016:2016-2015.
- Landers S, Madigan E, Leff B, Rosati R, McCann B, Hornbake R. The Future of Home Health Care. Home Health Care Manag Pract. SAGE PublicationsSage CA: Los Angeles, CA; 2016:262.
|CDSS: clinical decision support system|
|EHR: electronic health record|
|MAPLe: Method for Assigning Priority Levels|
|NYP: New York-Presbyterian|
|OASIS: Outcome and Assessment Information Set|
|PREVENT: Priority for the First Nursing Visit Tool|
|RE-AIM: Reach, Effectiveness, Adoption, Implementation, and Maintenance|
|RN: registered nurses|
|VNSNY: Visiting Nurse Service of New York|
Edited by G Eysenbach; submitted 12.05.20; peer-reviewed by R Pryss, S Sabarguna; comments to author 20.06.20; revised version received 25.07.20; accepted 08.08.20; published 22.01.21Copyright
©Maryam Zolnoori, Margaret V McDonald, Yolanda Barrón, Kenrick Cato, Paulina Sockolow, Sridevi Sridharan, Nicole Onorato, Kathryn Bowles, Maxim Topaz. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 22.01.2021.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.