This paper is in the following e-collection/theme issue:

Usage Data and Process Outcomes in eHealth Trials (56) Clinical Information and Decision Making (1314) Decision Support for Health Professionals (1127)

Published on in Vol 8, No 6 (2019): June

A Pragmatic Cluster Randomized Trial of an Electronic Clinical Decision Support System to Improve Chronic Kidney Disease Management in Primary Care: Design, Rationale, and Implementation Experience

A Pragmatic Cluster Randomized Trial of an Electronic Clinical Decision Support System to Improve Chronic Kidney Disease Management in Primary Care: Design, Rationale, and Implementation Experience

A Pragmatic Cluster Randomized Trial of an Electronic Clinical Decision Support System to Improve Chronic Kidney Disease Management in Primary Care: Design, Rationale, and Implementation Experience

Authors of this article:

Elaine C Khoong1 Author Orcid Image ;   Leah Karliner2, 3 Author Orcid Image ;   Lowell Lo4 Author Orcid Image ;   Marilyn Stebbins5 Author Orcid Image ;   Andrew Robinson6 Author Orcid Image ;   Sarita Pathak2, 3 Author Orcid Image ;   Jasmine Santoyo-Olsson2 Author Orcid Image ;   Rebecca Scherzer7 Author Orcid Image ;   Carmen A Peralta4, 7, 8 Author Orcid Image
Article Authors Cited by (10) Tweetations (4) Metrics

    Protocol

    1Division of General Internal Medicine at Zuckerberg San Francisco General Hospital, Department of Medicine, University of California San Francisco, San Francisco, CA, United States

    2Division of General Internal Medicine, Department of Medicine, University of California San Francisco, San Francisco, CA, United States

    3Multiethnic Health Equity Research Center, University of California San Francisco, San Francisco, CA, United States

    4Division of Nephrology, Department of Medicine, University of California San Francisco, San Francisco, CA, United States

    5School of Pharmacy, University of California San Francisco, San Francisco, CA, United States

    6University of California San Francisco, San Francisco, CA, United States

    7Kidney Health Research Collaborative, San Francisco Veterans Affairs Medical Center, University of California San Francisco, San Francisco, CA, United States

    8Cricket Health, San Francisco, CA, United States

    Corresponding Author:

    Elaine C Khoong, MD, MS

    Division of General Internal Medicine at Zuckerberg San Francisco General Hospital

    Department of Medicine

    University of California San Francisco

    Ward 13, Building 10, Floor 3

    1001 Potrero Avenue

    San Francisco, CA, 94110

    United States

    Phone: 1 4152063188

    Email: elaine.khoong@ucsf.edu


    Background: The diagnosis of chronic kidney disease (CKD) is based on laboratory results easily extracted from electronic health records; therefore, CKD identification and management is an ideal area for targeted electronic decision support efforts. Early CKD management frequently occurs in primary care settings where primary care providers (PCPs) may not implement all the best practices to prevent CKD-related complications. Few previous studies have employed randomized trials to assess a CKD electronic clinical decision support system (eCDSS) that provided recommendations to PCPs tailored to each patient based on laboratory results.

    Objective: The aim of this study was to report the trial design and implementation experience of a CKD eCDSS in primary care.

    Methods: This was a 3-arm pragmatic cluster-randomized trial at an academic general internal medicine practice. Eligible patients had 2 previous estimated-glomerular-filtration-rates by serum creatinine (eGFRCr) <60 mL/min/1.73m2 at least 90 days apart. Randomization occurred at the PCP level. For patients of PCPs in either of the 2 intervention arms, the research team ordered triple-marker testing (serum creatinine, serum cystatin-c, and urine albumin-creatinine-ratio) at the beginning of the study period, to be completed when acquiring labs for regular clinical care. The eCDSS launched for PCPs and patients in the intervention arms during a regular PCP visit subsequent to completing the triple-marker testing. The eCDSS delivered individualized guidance on cardiovascular risk-reduction, potassium and proteinuria management, and patient education. Patients in the eCDSS+ arm also received a pharmacist phone call to reinforce CKD-related education. The primary clinical outcome is blood pressure change from baseline at 6 months after the end of the trial, and the main secondary outcome is provider awareness of CKD diagnosis. We also collected process, patient-centered, and implementation outcomes.

    Results: A multidisciplinary team (primary care internist, nephrologists, pharmacist, and informaticist) designed the eCDSS to integrate into the current clinical workflow. All 81 PCPs contacted agreed to participate and were randomized. Of 995 patients initially eligible by eGFRCr, 413 were excluded per protocol and 58 opted out or withdrew, resulting in 524 patient participants (188 usual care; 165 eCDSS; and 171 eCDSS+). During the 12-month intervention period, 53.0% (178/336) of intervention patient participants completed triple-marker labs. Among these, 138/178 (77.5%) had a PCP appointment after the triple-marker labs resulted; the eCDSS was opened for 73.9% (102/138), with orders or education signed for 81.4% (83/102).

    Conclusions: Successful integration of an eCDSS into primary care workflows and high eCDSS utilization rates at eligible visits suggest this tailored electronic approach is feasible and has the potential to improve guideline-concordant CKD care.

    Trial Registration: ClinicalTrials.gov NCT02925962; https://clinicaltrials.gov/ct2/show/NCT02925962 (Archived by WebCite at http://www.webcitation.org/78qpx1mjR)

    International Registered Report Identifier (IRRID): DERR1-10.2196/14022

    JMIR Res Protoc 2019;8(6):e14022

    doi:10.2196/14022

    Keywords

    chronic kidney disease; clinical decision support systems; pragmatic clinical trial; electronic health records 


    Management of Early Chronic Kidney Disease

    Chronic kidney disease (CKD) is common in adults; in the United States, 14% of all adults and nearly half of individuals aged ≥70 years have CKD [1]. CKD is an important predictor of morbidity and cardiovascular mortality [2]. Although most patients with early CKD are seen by primary care providers (PCPs), studies have consistently shown that patients and PCPs remain largely unaware of the patient’s CKD diagnosis until the disease is more advanced [3,4]. Even when early CKD is recognized, PCPs frequently are unaware of the best practices for risk stratification and prevention of CKD-related complications [3]. In particular, both CKD staging and complication risk stratification are greatly improved using a triple marker strategy: urine albumin-creatinine ratio (ACR), estimated glomerular filtration rate (eGFR) based on both serum creatinine levels (eGFRCr) and cystatin-c levels (eGFRCys) [5-9]. Early detection of CKD and risk stratification enable clinicians and patients to take individualized actions that improve outcomes and have the potential to attenuate progression, such as blood pressure management and renin-angiotensin blockade [10-13], statin therapy [13-16], avoidance of nephrotoxic medications [17,18], and glucose management in individuals with diabetes [19-21].

    Approaches to Increase Guideline-Concordant Early Chronic Kidney Disease Management

    With the adoption of electronic health records (EHRs), there have been opportunities to implement low-cost interventions to improve patient care. Given that CKD diagnosis is based on laboratory results easily extracted from EHRs, CKD identification and management is an ideal area for targeted electronic efforts [22,23]. Previous studies have shown the feasibility of electronic clinical decision support systems (eCDSS) to improve guideline-concordant care for patients with CKD [24-29]. However, most studies have not been randomized nor have they provided individualized recommendations for patients based on laboratory results. Instead, they have utilized standard reminders or checklists. In addition, concerns about the eCDSS being burdensome or disruptive to workflow have hindered the development and testing of more complex, individualized eCDSS [23,30,31].

    Team-based approaches for providing chronic disease management in primary care practice have also been shown to improve patient outcomes [32,33]. For example, blood pressure (BP) control is improved when managed by nonphysician members of the team, including nurses and pharmacists [34-37]. Initial studies have suggested that CKD, like other chronic diseases, may also benefit from a team-based approach [38,39].

    Trial Aims

    Given the promise of both eCDSS and team-based care to improve guideline-concordant CKD care in primary care practice, we designed a pragmatic cluster randomized 3-arm trial (usual care; eCDSS; and eCDSS plus pharmacist follow-up). This trial aimed to assess the feasibility and impact of a CKD eCDSS with individualized recommendations and any additional benefit of pharmacist outreach to follow-up compared with usual care. In this paper, we have reported the study design and initial implementation outcomes of this trial.


    Overall Design

    This was a 3-arm cluster randomized controlled trial with randomization at the provider level. There was 1 usual care control arm and 2 intervention arms. For patients of PCPs in the intervention arms, the research team ordered triple-marker testing (serum creatinine, serum cystatin-c, and urine ACR) at the beginning of the study period, to be completed when the patient visited the lab for regular clinical care. The eCDSS launched for PCPs and patients in the intervention arms during a regular PCP visit subsequent to completing the triple-marker testing. Patients in the eCDSS+ arm also received a pharmacist phone call to reinforce CKD-related education after a PCP visit in which the eCDSS was utilized. We planned for an 18-month trial beginning October 4, 2017, that included a 12-month intervention period and subsequent 6-month follow-up period. This trial was registered on ClinicalTrials.gov (NCT02925962). The University of California San Francisco Human Research Protection Program approved the protocol for this study.

    Setting

    This study was conducted at a general internal medicine practice (with 2 locations) at the University of California San Francisco that cares for a diverse population of more than 24,000 adult patients. The PCPs in this practice include faculty attending physicians, resident physicians, and nurse practitioners (NPs). This practice is a certified primary care medical home (PCMH) that follows a team-based approach to care; each of the 10 teams includes 10 to 14 part-time PCPs, 2 medical assistants, a licensed vocational nurse, and administrative staff. Additional personnel of the PCMH include 5 registered nurses, 2 pharmacy technicians, and a clinical pharmacist available 1 day a week. The EHR used within this practice is Epic (EpicCare, Epic Systems).

    Pilot Activities

    To optimize the chance of a successful intervention, we worked with a multidisciplinary team (general internist, informatics specialist, and nephrologists) to develop the eCDSS logic. The team focused on areas with the strongest evidence base for CKD management in primary care and developed the logic to apply guideline recommendations in real-time for individual patients. In the first step, patients were classified as either at high risk (eGFRCys <60 mL/min and/or ACR >30 mcg/mg) or low risk (eGFRCys >60 mL/min and ACR <30 mcg/mg) for experiencing CKD-related complications [5,17]. After risk stratification, the eCDSS logic focused on 5 domains of CKD care for high-risk patients: (1) cardiovascular risk reduction; (2) potassium management; (3) proteinuria management; (4) appropriateness for nephrology referral; and (5) patient education. For low-risk patients, the logic defaulted to a recommendation for repeat triple-marker testing in 6 months.

    We elicited feedback from PCPs in focus group settings during development of the eCDSS and during implementation planning, which is a recommended practice to increase adoption and uptake of eCDSS [23]. Focus group discussions centered on eCDSS messaging and orders, as well as ways to facilitate integration into the usual clinical workflow.

    Eligibility and Selection of Participants

    All providers practicing at the general internal medicine practice with a primary care panel were eligible for inclusion and randomization.

    Eligible patients were identified using a previously described and validated algorithm [40]. (Patients were considered eligible for inclusion if they were aged 18 to 80 years; had a preferred language of English, Spanish, or Chinese; had at least 2 outpatient eGFRCr=30 to 59 milliliters/minute (mL/min) at least 90 days apart, one of which was within the 12 months before August 14, 2017 (when providers were first contacted for study recruitment); and had a primary care visit in the past 18 months. (see Figure 1 for the Consolidated Standards of Reporting Trial diagram) Participants were automatically excluded (based on EHR data) if they were deceased; diagnosed with end-stage renal disease; engaged with a nephrology clinic with 2 or more visits in the past 12 months; kidney transplant recipients; or diagnosed with dementia. PCPs were then sent a list of potential participants for inclusion and asked to exclude participants for additional criteria that were unreliably captured when extracted from the EHR: current pregnancy, life expectancy <6 months, limited communication ability owing to impaired cognition or severe mental illness, New York Heart Association Class III/IV heart failure, or known ejection fraction <25%. PCPs could also exclude any other participants they felt would be inappropriate for study staff to contact.

    ‎
    Figure 1. Flow diagram of participant selection. Note that 1 clinician who was randomized did not ultimately participate because their only eligible patient opted out. eGFRCr: estimated glomerular filtration rate by serum creatinine; eCDSS: electronic clinical decision support system trial arm; eCDSS+: electronic clinical decision support system and pharmacist follow-up trial arm; N/A: not applicable.
    View this figure

    Recruitment and Consent Process

    We recruited PCPs with eligible patients by eGFRCr by email between August 14, 2017 and September 8, 2017. PCPs were given 2 weeks to either opt-out entirely or exclude individual patients. We then randomized participating PCPs as described below. Participation letters were mailed to eligible intervention arm patients between September 1, 2017 and September 27, 2017, and patients were given 2 weeks to either return an opt-out card or call the study coordinator to opt out. Patients opting out after the study start date of October 4, 2017, were considered withdrawn from the study.

    The usual care group PCPs received no additional contact beyond recruitment from the study team, and the usual care patients were never contacted directly by the study team. Usual care PCPs did have access to the same labs and medications to use at their clinical discretion without any recommendations or direction from the study team.

    Randomization and Blinding

    We utilized block randomization at the PCP level based on panel size. The study statistician randomized PCP participants to each of the 3 study arms using an automated procedure that accounted for cluster size, to assure balance both in terms of number of patients and number of providers per arm. Eligible patient participants were assigned to study arms based on their PCP’s assignment. The study statistician was blinded to the identification of the PCPs and patients and will be similarly blinded for the analyses.

    Intervention

    Before the eCDSS rollout but after randomization, all participating PCPs randomized to an intervention arm were asked to watch an educational video that provided background on guideline-based CKD care, as well as information on what to expect when an enrolled patient participant came in for a visit. PCPs were given an incentive of US $10 and a chocolate bar upon watching the video. Overall, 69.8% (72.0% eCDSS and 67.9% eCDSS+) viewed the video.

    Before the eCDSS rollout, between October 5, 2017 and October 12, 2017, one of the study investigators (LK) ordered triple-marker testing for each patient participant from the eCDSS arm (n=169) and each patient participant from the eCDSS+ arm (n=177). We excluded 48 patients who opted out. In total, 10 patients withdrew later.

    After the triple-marker testing was ordered, the next time the patient participant visited the lab for their regular clinical care, the triple-marker labs were also collected. We programmed the eCDSS to trigger the first time that the patient participant visited their PCP after all 3 lab results were available. (see Figure 2 for the study workflow) The eCDSS best practice advisory (BPA) appeared above the current medications in the electronic chart (Figures 3 and 4). PCPs could choose to open the BPA by clicking an “accept” button. We also allowed PCPs to view the recommendations by hovering over the BPA before accepting it. For patient participants categorized as low risk, when the PCP accepted the BPA, triple-marker labs with an expected completion date 6 months hence prepopulated were automatically pending for the PCP to sign. For patient participants categorized as high risk, when the PCP accepted the BPA, a SmartSet (EpicCare, Epic Systems) of tailored orders and recommendations appeared (Figure 5). If the PCP did not open the BPA and sign the orders in the SmartSet during the first PCP visit after the triple-marker labs were available, the BPA would trigger at up to 2 additional PCP visits during the study period. The eCDSS did not trigger if the patient participant saw a provider who was not their PCP.

    ‎
    Figure 2. Flow diagram of study workflow for intervention arms. BPA: best practice advisory; PCP: primary care provider.
    View this figure
    ‎
    Figure 3. Electronic clinical decision support system trial arm low-risk best practice advisory. CKD: chronic kidney disease; Cr: serum creatinine; eGFR: estimated glomerular filtration rate.
    View this figure
    ‎
    Figure 4. Electronic clinical decision support system trial arm high-risk best practice advisory. BP: blood pressure; CKD: chronic kidney disease; mg/g: milligrams per grams.
    View this figure
    ‎
    Figure 5. Electronic clinical decision support system trial arm example SmartSet. ACEi/ARB: angiotensin converting enzyme inhibitor / angiotensin II receptor blocker; AVS: after visit summary; CKD: chronic kidney disease; eCDSS: electronic clinical decision support system; CDSS+: electronic clinical decision support system and pharmacist follow-up trial arm; K+: potassium; mg/g: milligrams per grams; NSAID: nonsteroidal anti-inflammatory drug; OTC: over-the-counter;.
    View this figure

    In addition to a reminder of BP targets in CKD, the SmartSet for patient participants classified as high risk for progression and/or complications of CKD used real-time EHR lab and medication data to recommend any of the following that applied:

    1. Cardiovascular risk reduction: recommendation for use of statin in all individuals aged ≥50 years not already on a statin, as recommended by Kidney Disease: Improving Global Outcomes (KDIGO) guidelines [17].
    2. Potassium management: diet and diuretic recommendations.
    3. Proteinuria management: initiation or titration of renin-angiotensin blockade with angiotensin converting enzyme inhibitors (ACEi) or angiotensin II receptor blockers (ARB).
    4. Nephrology referral for highest risk participants: highest risk was defined as one of: confirmed eGFRCys <30 mL/min; potassium >5.5 mEq/L; ACR >300 mcg/mg, systolic blood pressure (SBP) >150 millimeters of mercury (mm Hg) despite 3 agents including a diuretic; >3% risk of five-year progression kidney failure based on Tangri equation [41].
    5. Patient education: patient education materials to populate in the after-visit summary (AVS) focused on CKD general information, avoidance of nonsteroidal anti-inflammatory drugs (NSAIDs), and dietary recommendations.

    The second intervention arm (eCDSS+) included a pharmacist follow-up visit by telephone. If the SmartSet was signed during the PCP visit, a pharmacist would call the participant within 2 weeks of the visit to reinforce CKD-related teaching and medication changes from the visit as well as complete a comprehensive medication review. Information on the telephone encounter was documented in the EHR and sent to the PCP; any urgent clinical issues identified during the call were highlighted for the PCP as needing their follow-up. Providers in the eCDSS+ group also were encouraged to make a warm handoff to the pharmacist by including anticipatory guidance about the pharmacist phone call within the after-visit summary and distributing a business card with the pharmacist’s photo and phone number for their patient participants at the time of using the SmartSet.

    Given that the study physician ordered labs at the start of the intervention period, safety checks were put into place to ensure adequate review of the laboratory results. A nephrologist (LL) from the study team reviewed weekly laboratory results specifically to identify the following: eGFRCr decline >30% from baseline, ACR ≥1000 mcg/mg, adherence to nephrology referrals, and any discordance >30% between eGFRCr and eGFRCys. In all of these situations, the nephrologist contacted the PCP to ensure appropriate follow-up. Specifically, in cases of discordance between the 2 types of eGFR levels, the nephrologist advised PCPs to dose medications based on eGFR­Cys when the clinical scenario suggested that eGFRcr may not be accurate.

    Strategies to Encourage Intervention and Behavior Change

    Implementation studies have shown that interventions designed with a theoretical basis are more likely to be successful [42,43]. Therefore, we utilized several theory-based strategies to encourage provider uptake of the eCDSS and participant uptake of provider and/or pharmacist recommendations. We classify these strategies using the capability, opportunity, motivation, behavior framework (or COM-B), which asserts that capability, opportunity, and motivation are essential conditions that impact behavior [44,45].

    To encourage provider uptake and use of CKD guidelines, we addressed capability barriers (such as knowledge about the guidelines or the eCDSS or forgetting to discuss CKD owing to limited bandwidth). During the recruitment phase, we educated eligible PCPs about the prevalence of CKD within their panel by sending a list of their potentially eligible participants along with information about the importance of recognizing and managing CKD early. Shortly before intervention implementation, we sent a training video about the CKD guidelines and the eCDSS to the participating intervention PCPs. The eCDSS itself also served as a prompt to discuss CKD during the visits.

    To address PCP opportunity barriers, we designed the eCDSS to fit into the physician workflow and to contain all the necessary CKD-related orders and patient education in one easily accessible place during a clinical visit.

    To address PCP motivation barriers, we provided a small incentive to the participating intervention PCPs who watched the training video. Finally, to enhance use of the BPA, study personnel sent an individual email reminder to the intervention PCP the day before each eligible patient participant visit.

    Although the eCDSS itself was a provider-facing intervention, we used strategies to encourage patient participant behavior change. To address capability barriers (such as knowledge of CKD or forgetting the provider recommendations), the CDSS SmartSet had CKD-based patient educational handouts in the patient’s preferred language. The pharmacist phone call was also designed to reinforce the knowledge provided at the office visit and motivate the patient participant to adhere to medication and avoid NSAIDS.

    Data Collection

    For patient participants, the EHR was used to collect baseline data about demographic characteristics (age, sex, race-ethnicity, preferred language, and insurance status); medical co-morbidities (cerebrovascular disease, congestive heart failure, coronary artery disease, diabetes mellitus, hyperlipidemia, and hypertension); current medication prescriptions (statin, ACEi/ARB, and diuretics); previous documentation of CKD on problem list and/or as a visit diagnosis, most recent blood pressure, eGFR­­Cr, eGFR­­Cys (if available), and urine ACR (if available) before enrollment.

    Primary outcome data on BP will be extracted from the EHR. Baseline BP was defined as the most recent ambulatory BP measurement before final enrollment on October 4, 2017. All patient participant ambulatory BP measurements were collected during the intervention period and will continue to be collected for the 6 months after the end of the intervention.

    Secondary outcomes will be collected primarily using automated EHR data extraction algorithms. The outcomes acquired through data extraction include all secondary outcomes except for provider and patient-centered outcomes, which are collected via surveys. Up to 25 participants in each of the 3 arms were surveyed to assess patient knowledge about CKD and NSAID avoidance. Patient participants in the eCDSS arm were surveyed 2 weeks after a visit in which the SmartSet was utilized; those in the eCDSS+ arm were surveyed 2 weeks after their pharmacist follow-up call, and those in the usual care arm were surveyed 2 weeks after a visit with their PCP during the study period. All participating intervention PCPs will be surveyed about their perception of the feasibility and utility of the eCDSS and pharmacist calls (for CDSS+ providers).

    Outcomes

    Primary Outcome

    A summary of pre-planned study outcomes is shown in Textbox 1. The primary clinical outcome of this study is BP change from baseline. This outcome will be assessed as a continuous variable (separately for both diastolic and systolic BP change from baseline) and dichotomous variable (sustained control, defined as BP <140/90 in 2 or more consecutive visits during the trial). As this study was initiated before the 2017 BP guidelines, <140/90 was used to define control; this value also aligns with current KDIGO recommendations [17,46]. Rates of control at the end of the intervention period and 6 months later will be assessed based on the most recent BP measurement carried forward.

    Outcome measures. ACEi/ARB: angiotensin converting enzyme inhibitor / angiotensin II receptor blocker; BPA: best practice advisory; CKD: chronic kidney disease; PCP: primary care provider; NSAID: nonsteroidal anti-inflammatory drug.

    Primary outcome: blood pressure

    • Systolic blood pressure change
    • Diastolic blood pressure change
    • Controlled blood pressure (defined as <140/90)

    Secondary outcomes

    • Process
      • PCP awareness of CKD diagnosis (CKD inclusion on problem list or visit diagnosis)
      • ACEi/ARB utilization for albuminuria
      • Statin therapy for eligible high-risk CKD patients
        • Initiation of statin therapy
        • Total use of statin therapy
    • Patient-centered
      • Knowledge about CKD
      • Awareness of NSAID avoidance in CKD

    Implementation outcomes

    • Reach
      • Recruitment rate of providers and patients
      • Proportion that complete triple marker screen
      • Proportion with PCP visit after complete triple marker screening
      • Proportion of participants where BPA is available for PCP
      • Proportion of participants identified as high- vs low-risk CKD
    • Adoption
      • Proportion where PCP signed Smart Set when BPA was available
      • Proportion of CDSS+ participants that received pharmacist phone call after PCP signed Smart Set
    • Implementation
      • Proportion of orders signed or patient education provided by PCP after opening Smart Set
      • Proportion referred to nephrology and followed up with nephrology after signing Smart Set
      • Proportion of low-risk CKD patients that receive repeat triple screen
    • Maintenance
      • PCP satisfaction with eCDSS
      • PCP intent to continue using eCDSS
    Textbox 1. Outcome measures. ACEi/ARB: angiotensin converting enzyme inhibitor / angiotensin II receptor blocker; BPA: best practice advisory; CKD: chronic kidney disease; PCP: primary care provider; NSAID: nonsteroidal anti-inflammatory drug.
    Secondary Outcomes

    Our main secondary outcome is PCP awareness of the patient participant’s CKD diagnosis, as measured by inclusion of CKD-related International Statistical Classification of Diseases and Related Health Problems-10th Revision codes on the problem list or as a visit diagnosis. Additional secondary outcomes include process of care, patient-centered, and implementation outcomes.

    There are 2 main process of care outcomes: (1) the proportion of patient participants with albuminuria prescribed ACEi/ARB and (2) appropriate use of statins. Appropriate statin use is defined both as the proportion of all patient participants aged ≥50 years prescribed statin therapy and the proportion of participants initiated on statin therapy after study start date.

    Patient-centered outcomes include patient knowledge about CKD and NSAID avoidance as measured by patient surveys.

    In addition to the effectiveness outcomes described above, the implementation outcomes are described in detail in Textbox 1 using the reach, effectiveness, adoption, implementation, and maintenance (or RE-AIM) framework to highlight measures crucial to successful implementation [47]. In this paper, we report initial implementation results, including all of the Reach outcomes and the first Adoption outcome (PCP SmartSet use).

    Analyses

    Baseline demographic and clinical characteristics will be summarized and study balance by study arm will be assessed using descriptive statistics as well as methods that account for the lack of independence among patients in the same cluster [48]. Balance will be assessed with logit link generalized estimating equation (GEE) for categorical variables and linear GEE models for continuous variables. Clustering will be accounted for using an exchangeable correlation matrix with robust standard errors. We anticipate controlling for characteristics that are associated with outcomes but differ at baseline between arms and using multivariable regression analyses with GEE to account for cluster randomization in the reporting of the final outcomes of our study. Sensitivity analyses will include as treated analyses with inverse probability weighting to determine impact of participants changing providers/study arms as well as to determine impact of any differences in the number of assessments or clinic visits by study arm. Primary analyses will follow intention-to-treat principles.

    Sample Size and Power Calculation

    Based on preliminary data from our institution’s EHR, it was determined that there would be a maximum of 1400 participants in the practice who could meet inclusion criteria. The study was powered for the clinical outcome of BP change, with calculations performed using the clustersampsi command for Stata version 11.2. We assumed a 2-tailed alpha level of 0.05, an intraclass correlation coefficient of 0.025 based on studies in primary care settings [49,50], and a standard deviation of 5 mmHg. Assuming 23 clinicians per arm with at least 15 patients per clinician (ie, 345 patients per arm), we estimated that we would have 80% power to detect a difference as small as 1.27 mmHg between arms.


    Implementation Outcomes: Reach

    Implementation outcomes at the end of the 12-month intervention period on October 4, 2018, are summarized below:

    Recruitment Rate

    At the provider level, all 81 eligible PCPs (47 faculty attending physicians, 31 resident physicians, and 3 NPs) agreed to participate (100%). In total, 79 of the 81 providers (98%) were included because 2 providers had no remaining eligible participants after exclusion and opt-outs. At the patient level, 995 patients who met the initial eligibility criteria were identified. A total of 316 / 995 (31.8%) potential patient participants were excluded using automated algorithms, and clinicians excluded another 90 patients (Figure 1). A total of 582 patient participants were distributed to the 3 arms based on provider randomization. After an additional 3 patients were excluded (owing to patient deaths not recorded in EHR before randomization) and 55 / 582 patients (9.5%) opted out or withdrew, 524 patient participants (90.0% of eligible 582 participants) were included in the study: 188 usual care, 165 eCDSS, and 171 eCDSS+.

    Baseline patient participant characteristics are shown in Table 1. The 55 patients that opted out/withdrew were similar to the patient participants across multiple traits—including age, gender, race/ethnicity, language preference, and co-morbidities—except they had lower renal function on average: eGFRCr (52.7 [SD 11.7] versus 56.0 [SD 11.8]; P=.046).

    Table 1. Baseline characteristics of all included patients (N=524).
    Patient characteristicValues
    Age (years), mean (SD)70.3 (8.9)
    Female, n (%)236 (45)
    Race/ethnicity, n (%)

    		White277 (53)

    		Asian118 (23)

    		Black/African American70 (13)

    		Hispanic38 (7)

    		Other20 (4)
    Preferred language, n (%)

    		English469 (90)

    		Chinese39 (7)

    		Spanish16 (3)
    Insurance, n (%)

    		Private insurance146 (28)

    		Medicare215 (41)

    		Medicaid163 (31)
    Medical co-morbidities, n (%)

    		Cerebrovascular disease36 (7)

    		Congestive heart failure40 (8)

    		Coronary artery disease95 (18)

    		Diabetes mellitus199 (38)

    		Hyperlipidemia302 (58)

    		Hypertension377 (72)
    Medication use, n (%)

    		Angiotensin converting enzyme inhibitor/angiotensin II receptor blocker319 (61)

    		Diuretic199 (38)

    		Statin353 (67)
    CKD-related outcomes, n (%)

    		Chronic kidney disease (CKD) on problem list61 (12)

    		CKD on problem list or primary care visit diagnosis by ICD-10c codes247 (47)
    Systolic BPa (mm Hg), median (Q1-Q3)128 (117-140)
    Diastolic BP (mm Hg), median (Q1-Q3)68 (62.5-75)
    BP controlledb (<140/90), n (%)373 (71)
    Estimated glomerular filtration rate based by serum creatinine (mL/min), mean (SD)56.0 (11.8)

    aBP: blood pressure.

    bBP control reported based on only the most recent measurement before the start of the study.

    cICD-10: International Classification of Diseases, Tenth Revision, Clinical Modification.

    Completion of Triple Marker

    Of the 336 patient participants in the intervention arms, 178 (53.0%) completed the triple-marker screening.

    PCP Visits

    Of the 178 patient participants who completed a triple-marker screen, 138 (77.5%) had a PCP visit during the 12-month intervention period. This was 41.1% (138/336) of all intervention participants.

    PCP BPA Use

    Of the 138 participants with a PCP visit, the BPA was opened for 102 participants (73.9%). This was 30.3% (102/336) of all intervention participants.

    Implementation Outcomes: Adoption

    SmartSet Use

    Of the 102 participants in which the BPA was opened, orders were signed or patient education was given from the SmartSet for 83 participants (81.4%).


    Principal Findings

    In this report, we describe the rationale, design, and initial implementation outcomes of a 3-arm pragmatic trial that assessed the feasibility and preliminary effectiveness of an eCDSS to improve CKD management in primary care compared with usual care. This study builds on previous pragmatic trials focused on EHR-based interventions to improve CKD management [28,29,51,52] and supports the feasibility of using EHRs to identify study participants, intervene in early CKD management, and measure study outcomes. Given its design as a pragmatic trial (per Pragmatic Explanatory Continuum Indicator Summary, or PRECIS, criteria [53]), we found high rates of participation by providers and low opt-out rates by patients. We also found high rates of eCDSS use by providers.

    This study has many innovative components. We utilized a 3-armed trial design, which included a usual care group as well as a group with pharmacist-patient engagement after the provider-facing electronic intervention. In addition, our clinical eCDSS was mostly automated and individualized to patients based on laboratory results, rather than a generic CDSS for all participants. We also implemented the recommended triple-marker risk stratification approach to better characterize participants’ risk for CKD progression. Our participants included non-English speakers, who have been excluded from many other trials. This trial also included multiple types of outcomes allowing for a deeper understanding of the intervention’s impact on care processes. Reviews of eCDSS have found that few studies assess unintended consequences or adverse effects of a CDSS, which can be widespread as CDSS interventions often automate multiple actions [54,55]. This trial integrated safety measures and their tracking by a nephrologist allowing not only for measurement of safety but also the ability to ensure that any new concerning clinical findings found owing to the trial would be addressed. As previously recommended, it is important to first pilot interventions before widespread implementation to ensure there are no adverse effects in a small population where safety can be more easily assessed [56].

    One of the greatest challenges of this trial was its dependence on data analysts experienced at working with the EHR to program and implement the intervention as well as extract outcomes data. The study team devoted substantial time to troubleshoot unanticipated problems with the eCDSS programming and capture of outcomes. Every update or change to the EHR processes had the potential to disrupt the trial implementation. The team regularly validated the eCDSS and data capture through manual chart reviews. The inaccuracies of EHR data extraction are well-documented [57]. Despite the presence of an explicit trial infrastructure in our EHR, implementation of the intervention and data collection was far from seamless. Studies that rely on the EHR to deliver interventions and collect outcomes must ensure adequate time and resources for a multidisciplinary team to validate the intervention and review data inaccuracies in an ongoing manner for all EHR-based trials.

    Comparison With Previous Work

    In this trial, we found much higher reach and adoption of eCDSS by PCPs than previous studies where adoption rates were frequently <50% [55,58]. We believe there are multiple explanations. Few electronic alerts are currently used in this practice, so providers have not yet experienced fatigue associated with the growing presence of alerts [59]. Moreover, the eCDSS was designed by a multidisciplinary team, facilitating buy-in from all stakeholders and ensuring all perspectives were considered in determining how the eCDSS would impact workflow. Finally, study staff (including a lead investigator who works within the primary care practice) served as on-the-ground champions and made significant efforts to educate, remind, and incentivize PCPs.

    We experienced many of the same challenges as other pragmatic trials during implementation of this EHR-dependent intervention. There was a period of time between provider recruitment, randomization, participants being contacted to opt out, and the study start date; as a result, 18 patients passed away before randomization and 2 additional patients passed away before the start of the study. Importantly, nearly half of the participants in the intervention arms never completed the triple-marker screen and therefore never had the opportunity to receive the eCDSS interventions. Another one-fifth of participants who completed the triple-marker screen did not have a PCP visit after the triple-marker screening and also did not receive the eCDSS. These barriers, common in many eCDSS trials, prevented the intervention group from receiving the desired intervention.

    Limitations

    As a pilot study, this study was limited by its inclusion of patients and providers from a single institution. Therefore, both the intervention and strategies to encourage intervention uptake were adapted to meet the needs of this single institution that uses Epic Systems’ EHR. However, we have found that the patient population in this clinic is similar to others [60], and Epic is a widely used EHR in multiple health systems. As a pragmatic trial, our sample size was smaller than initially anticipated and will thus ultimately impact our power to detect changes in our primary outcome. The short follow-up period for this pilot study will also limit our ability to assess some clinical outcomes. However, we were still able to use this pilot study to determine the feasibility of this intervention.

    Conclusions

    As the prevalence of CKD grows, primary care teams will increasingly be responsible for management of CKD. Clinical decision support tools may be a low-cost effective solution to enhance guideline-concordant care for this underdiagnosed disease. We describe the design and implementation of a pragmatic clustered randomized controlled trial to evaluate the feasibility and effectiveness an EHR-embedded electronic clinical decision support tool to improve management of CKD in primary care. Results from this study can guide design of future pragmatic eCDSS trials to improve CKD care.

    Acknowledgments

    LK and CP received funding from the National Institute of Diabetes and Digestive and Kidney Diseases (1R18DK110959). EK received funding through the National Research Service Award (T32HP19025). The funders played no role in review or approval of this manuscript for publication. The authors wish to thank the PCPs and patients who participated in this study.

    Authors' Contributions

    EK analyzed and interpreted data and drafted the manuscript. LK, LL, and MS conceived and designed the study; interpreted data; and substantively revised the manuscript. AR, SP, JS acquired and analyzed data. RS designed the study; analyzed data; and substantively revised the manuscript. CP conceived and designed the study; interpreted data; and substantively revised the manuscript. All authors read and approved the final manuscript.

    Conflicts of Interest

    CP is currently the chief medical officer of Cricket Health. The other authors declare they have no conflicts of interest.

    ‎

    Multimedia Appendix 1

    CONSORT‐EHEALTH checklist (V 1.6.1).

    PDF File (Adobe PDF File), 3MB
    1. Centers for Disease Control and Prevention. Chronic Kidney Disease Initiative   URL: https://www.cdc.gov/ckd/ [accessed 2019-05-29] [WebCite Cache]
    2. United States Renal Data System. American Journal of Kidney Diseases. Bethesda, MD: National Institutes of Health, National Institute of Diabetes and Digestive and Kidney Diseases; 2018. Morbidity and Mortality in Patients with CKD   URL: https://www.ajkd.org/article/S0272-6386(17)30077-X/pdf [accessed 2019-05-31] [WebCite Cache]
    3. Abdel-Kader K, Greer RC, Boulware LE, Unruh ML. Primary care physicians' familiarity, beliefs, and perceived barriers to practice guidelines in non-diabetic CKD: a survey study. BMC Nephrol 2014 Apr 22;15:64 [FREE Full text] [CrossRef] [Medline]
    4. Tuot DS, Plantinga LC, Hsu C, Jordan R, Burrows NR, Hedgeman E, Centers for Disease Control Chronic Kidney Disease Surveillance Team. Chronic kidney disease awareness among individuals with clinical markers of kidney dysfunction. Clin J Am Soc Nephrol 2011 Aug;6(8):1838-1844 [FREE Full text] [CrossRef] [Medline]
    5. Peralta CA, Shlipak MG, Judd S, Cushman M, McClellan W, Zakai NA, et al. Detection of chronic kidney disease with creatinine, cystatin C, and urine albumin-to-creatinine ratio and association with progression to end-stage renal disease and mortality. J Am Med Assoc 2011 Apr 20;305(15):1545-1552 [FREE Full text] [CrossRef] [Medline]
    6. Shlipak MG, Sarnak MJ, Katz R, Fried LF, Seliger SL, Newman AB, et al. Cystatin C and the risk of death and cardiovascular events among elderly persons. N Engl J Med 2005 May 19;352(20):2049-2060. [CrossRef] [Medline]
    7. Peralta CA, Muntner P, Scherzer R, Judd S, Cushman M, Shlipak MG. A risk score to guide cystatin C testing to detect occult-reduced estimated glomerular filtration rate. Am J Nephrol 2015;42(2):141-147 [FREE Full text] [CrossRef] [Medline]
    8. Inker LA, Schmid CH, Tighiouart H, Eckfeldt JH, Feldman HI, Greene T, CKD-EPI Investigators. Estimating glomerular filtration rate from serum creatinine and cystatin C. N Engl J Med 2012 Jul 5;367(1):20-29 [FREE Full text] [CrossRef] [Medline]
    9. Shlipak MG, Matsushita K, Ärnlöv J, Inker LA, Katz R, Polkinghorne KR, CKD Prognosis Consortium. Cystatin C versus creatinine in determining risk based on kidney function. N Engl J Med 2013 Sep 5;369(10):932-943 [FREE Full text] [CrossRef] [Medline]
    10. Appel LJ, Wright JT, Greene T, Agodoa LY, Astor BC, Bakris GL, AASK Collaborative Research Group. Intensive blood-pressure control in hypertensive chronic kidney disease. N Engl J Med 2010 Sep 2;363(10):918-929 [FREE Full text] [CrossRef] [Medline]
    11. Xie X, Atkins E, Lv J, Bennett A, Neal B, Ninomiya T, et al. Effects of intensive blood pressure lowering on cardiovascular and renal outcomes: updated systematic review and meta-analysis. Lancet 2016 Jan;387(10017):435-443. [CrossRef]
    12. SPRINT Research Group, Wright JT, Williamson JD, Whelton PK, Snyder JK, Sink KM, et al. A randomized trial of intensive versus standard blood-pressure control. N Engl J Med 2015 Nov 26;373(22):2103-2116 [FREE Full text] [CrossRef] [Medline]
    13. Upadhyay A, Earley A, Haynes S, Uhlig K. Systematic review: blood pressure target in chronic kidney disease and proteinuria as an effect modifier. Ann Intern Med 2011 Apr 19;154(8):541-548. [CrossRef] [Medline]
    14. Sanguankeo A, Upala S, Cheungpasitporn W, Ungprasert P, Knight EL. Effects of statins on renal outcome in chronic kidney disease patients: a systematic review and meta-analysis. PLoS One 2015;10(7):e0132970 [FREE Full text] [CrossRef] [Medline]
    15. Cholesterol Treatment Trialists' (CTT) Collaboration, Herrington W, Emberson J, Mihaylova B, Blackwell L, Reith C, et al. Impact of renal function on the effects of LDL cholesterol lowering with statin-based regimens: a meta-analysis of individual participant data from 28 randomised trials. Lancet Diabetes Endocrinol 2016 Dec;4(10):829-839 [FREE Full text] [CrossRef] [Medline]
    16. Grundy SM, Stone NJ, Bailey AL, Beam C, Birtcher KK, Blumenthal RS, et al. 2018 AHA/ACC/AACVPR/AAPA/ABC/ACPM/ADA/AGS/APhA/ASPC/NLA/PCNA Guideline on the Management of Blood Cholesterol: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. J Am Coll Cardiol 2018 Nov 8. [CrossRef] [Medline]
    17. Stevens PE, Levin A, Kidney Disease: Improving Global Outcomes Chronic Kidney Disease Guideline Development Work Group Members. Evaluation and management of chronic kidney disease: synopsis of the kidney disease: improving global outcomes 2012 clinical practice guideline. Ann Intern Med 2013 Jun 4;158(11):825-830. [CrossRef] [Medline]
    18. Inker LA, Astor BC, Fox CH, Isakova T, Lash JP, Peralta CA, et al. KDOQI US commentary on the 2012 KDIGO clinical practice guideline for the evaluation and management of CKD. Am J Kidney Dis 2014 May;63(5):713-735. [CrossRef] [Medline]
    19. Bash LD, Selvin E, Steffes M, Coresh J, Astor BC. Poor glycemic control in diabetes and the risk of incident chronic kidney disease even in the absence of albuminuria and retinopathy: Atherosclerosis Risk in Communities (ARIC) Study. Arch Intern Med 2008 Dec 8;168(22):2440-2447 [FREE Full text] [CrossRef] [Medline]
    20. Shurraw S, Hemmelgarn B, Lin M, Majumdar SR, Klarenbach S, Manns B, Alberta Kidney Disease Network. Association between glycemic control and adverse outcomes in people with diabetes mellitus and chronic kidney disease: a population-based cohort study. Arch Intern Med 2011 Nov 28;171(21):1920-1927. [CrossRef] [Medline]
    21. Fioretto P, Bruseghin M, Berto I, Gallina P, Manzato E, Mussap M. Renal protection in diabetes: role of glycemic control. J Am Soc Nephrol 2006 Apr;17(4 Suppl 2):S86-S89 [FREE Full text] [CrossRef] [Medline]
    22. Drawz PE, Archdeacon P, McDonald CJ, Powe NR, Smith KA, Norton J, et al. CKD as a model for improving chronic disease care through electronic health records. Clin J Am Soc Nephrol 2015 Aug 7;10(8):1488-1499 [FREE Full text] [CrossRef] [Medline]
    23. Patwardhan MB, Kawamoto K, Lobach D, Patel UD, Matchar DB. Recommendations for a clinical decision support for the management of individuals with chronic kidney disease. Clin J Am Soc Nephrol 2009 Feb;4(2):273-283 [FREE Full text] [CrossRef] [Medline]
    24. Abdel-Kader K, Fischer GS, Li J, Moore CG, Hess R, Unruh ML. Automated clinical reminders for primary care providers in the care of CKD: a small cluster-randomized controlled trial. Am J Kidney Dis 2011 Dec;58(6):894-902 [FREE Full text] [CrossRef] [Medline]
    25. Mendu ML, Schneider LI, Aizer AA, Singh K, Leaf DE, Lee TH, et al. Implementation of a CKD checklist for primary care providers. Clin J Am Soc Nephrol 2014 Sep 5;9(9):1526-1535 [FREE Full text] [CrossRef] [Medline]
    26. Litvin CB, Hyer JM, Ornstein SM. Use of clinical decision support to improve primary care identification and management of chronic kidney disease (CKD). J Am Board Fam Med 2016;29(5):604-612 [FREE Full text] [CrossRef] [Medline]
    27. Ennis J, Gillen D, Rubenstein A, Worcester E, Brecher ME, Asplin J, et al. Clinical decision support improves physician guideline adherence for laboratory monitoring of chronic kidney disease: a matched cohort study. BMC Nephrol 2015 Oct 15;16:163 [FREE Full text] [CrossRef] [Medline]
    28. Nash DM, Ivers NM, Young J, Jaakkimainen RL, Garg AX, Tu K. Improving Care for Patients with or at risk for chronic kidney disease using electronic medical record interventions: a pragmatic cluster-randomized trial protocol. Can J Kidney Health Dis 2017;4:2054358117699833 [FREE Full text] [CrossRef] [Medline]
    29. Carroll JK, Pulver G, Dickinson LM, Pace WD, Vassalotti JA, Kimminau KS, et al. Effect of 2 clinical decision support strategies on chronic kidney disease outcomes in primary care: a cluster randomized trial. JAMA Netw Open 2018 Oct 5;1(6):e183377 [FREE Full text] [CrossRef] [Medline]
    30. Gulla J, Neri PM, Bates DW, Samal L. User requirements for a chronic kidney disease clinical decision support tool to promote timely referral. Int J Med Inform 2017 Dec;101:50-57 [FREE Full text] [CrossRef] [Medline]
    31. Osheroff JA, Teich JM, Middleton B, Steen EB, Wright A, Detmer DE. A roadmap for national action on clinical decision support. J Am Med Inform Assoc 2007;14(2):141-145 [FREE Full text] [CrossRef] [Medline]
    32. Milani RV, Lavie C. Health care 2020: reengineering health care delivery to combat chronic disease. Am J Med 2015 Apr;128(4):337-343. [CrossRef]
    33. Körner M, Bütof S, Müller C, Zimmermann L, Becker S, Bengel J. Interprofessional teamwork and team interventions in chronic care: a systematic review. J Interprof Care 2016;30(1):15-28. [CrossRef] [Medline]
    34. Carter BL, Rogers M, Daly J, Zheng S, James PA. The potency of team-based care interventions for hypertension: a meta-analysis. Arch Intern Med 2009 Oct 26;169(19):1748-1755 [FREE Full text] [CrossRef] [Medline]
    35. McLean DL, McAlister FA, Johnson JA, King KM, Makowsky MJ, Jones CA, SCRIP-HTN Investigators. A randomized trial of the effect of community pharmacist and nurse care on improving blood pressure management in patients with diabetes mellitus: study of cardiovascular risk intervention by pharmacists-hypertension (SCRIP-HTN). Arch Intern Med 2008 Nov 24;168(21):2355-2361. [CrossRef] [Medline]
    36. Glynn L, Murphy A, Smith S, Schroeder K, Fahey T. Interventions used to improve control of blood pressure in patients with hypertension. Cochrane Database Syst Rev 2010 Mar 17(3):CD005182. [CrossRef] [Medline]
    37. Margolis KL, Asche SE, Bergdall AR, Dehmer SP, Groen SE, Kadrmas HM, et al. Effect of home blood pressure telemonitoring and pharmacist management on blood pressure control: a cluster randomized clinical trial. J Am Med Assoc 2013 Jul 3;310(1):46-56 [FREE Full text] [CrossRef] [Medline]
    38. Greer R, Boulware LE. Reducing CKD risks among vulnerable populations in primary care. Adv Chronic Kidney Dis 2015 Jan;22(1):74-80. [CrossRef]
    39. Johns TS, Yee J, Smith-Jules T, Campbell RC, Bauer C. Interdisciplinary care clinics in chronic kidney disease. BMC Nephrol 2015 Oct 12;16:161 [FREE Full text] [CrossRef] [Medline]
    40. Frigaard M, Rubinsky A, Lowell L, Malkina A, Karliner L, Kohn M, et al. Validating laboratory defined chronic kidney disease in the electronic health record for patients in primary care. BMC Nephrol 2019 Jan 3;20(1):3 [FREE Full text] [CrossRef] [Medline]
    41. Tangri N. A predictive model for progression of chronic kidney disease to kidney failure. J Am Med Assoc 2011 Apr 20;305(15):1553. [CrossRef] [Medline]
    42. Sales A, Smith J, Curran G, Kochevar L. Models, strategies, and tools. Theory in implementing evidence-based findings into health care practice. J Gen Intern Med 2006 Feb;21(Suppl 2):S43-S49 [FREE Full text] [CrossRef] [Medline]
    43. Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N. Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol 2005 Feb;58(2):107-112. [CrossRef] [Medline]
    44. Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci 2011;6:42 [FREE Full text] [CrossRef] [Medline]
    45. Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med 2013 Aug;46(1):81-95. [CrossRef] [Medline]
    46. Whelton PK, Carey RM, Aronow WS, Casey DE, Collins KJ, Himmelfarb CD, et al. ACC/AHA/AAPA/ABC/ACPM/AGS/APhA/ASH/ASPC/NMA/PCNA guideline for the prevention, detection, evaluation, and management of high blood pressure in adults: executive summary: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Practice Guidelines. J Am Coll Cardiol 2018 May 15;71(19):2199-2269 [FREE Full text] [CrossRef] [Medline]
    47. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999 Sep;89(9):1322-1327. [Medline]
    48. Ma J, Raina P, Beyene J, Thabane L. Comparison of population-averaged and cluster-specific models for the analysis of cluster randomized trials with missing binary outcomes: a simulation study. BMC Med Res Methodol 2013 Jan 23;13:9 [FREE Full text] [CrossRef] [Medline]
    49. Yelland LN, Salter AB, Ryan P, Laurence CO. Adjusted intraclass correlation coefficients for binary data: methods and estimates from a cluster-randomized trial in primary care. Clin Trials 2011 Feb;8(1):48-58. [CrossRef] [Medline]
    50. Killip S, Mahfoud Z, Pearce K. What is an intracluster correlation coefficient? Crucial concepts for primary care researchers. Ann Fam Med 2004;2(3):204-208 [FREE Full text] [Medline]
    51. Navaneethan SD, Jolly SE, Schold JD, Arrigain S, Nakhoul G, Konig V, et al. Pragmatic randomized, controlled trial of patient navigators and enhanced personal health records in CKD. Clin J Am Soc Nephrol 2017 Sep 7;12(9):1418-1427 [FREE Full text] [CrossRef] [Medline]
    52. Peralta CA, Frigaard M, Rubinsky AD, Rolon L, Lo L, Voora S, et al. Implementation of a pragmatic randomized trial of screening for chronic kidney disease to improve care among non-diabetic hypertensive veterans. BMC Nephrol 2017 Dec 12;18(1):132 [FREE Full text] [CrossRef] [Medline]
    53. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. Br Med J 2015 May 8;350:h2147. [Medline]
    54. Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Effect of clinical decision-support systems: a systematic review. Ann Intern Med 2012 Jul 3;157(1):29-43. [CrossRef] [Medline]
    55. Lobach D, Sanders GD, Bright TJ, Wong A, Dhurjati R, Bristow E, et al. Enabling Health Care Decisionmaking Through Clinical Decision Support and Knowledge Management. Rockville, MD: Agency for Healthcare Research and Quality; 2012.   URL: https://effectivehealthcare.ahrq.gov/sites/default/files/pdf/clinical-decision-support_research.pdf [accessed 2019-05-29] [WebCite Cache]
    56. Beeler PE, Bates DW, Hug BL. Clinical decision support systems. Swiss Med Wkly 2014;144:w14073. [CrossRef] [Medline]
    57. Chan KS, Fowles JB, Weiner JP. Review: electronic health records and the reliability and validity of quality measures: a review of the literature. Med Care Res Rev 2010 Oct;67(5):503-527. [CrossRef] [Medline]
    58. McCullagh L, Mann D, Rosen L, Kannry J, McGinn T. Longitudinal adoption rates of complex decision support tools in primary care. Evid Based Med 2014 Dec;19(6):204-209. [CrossRef] [Medline]
    59. Kesselheim AS, Cresswell K, Phansalkar S, Bates DW, Sheikh A. Clinical decision support systems could be modified to reduce 'alert fatigue' while still minimizing the risk of litigation. Health Aff (Millwood) 2011 Dec;30(12):2310-2317 [FREE Full text] [CrossRef] [Medline]
    60. Jolly SE, Navaneethan SD, Schold JD, Arrigain S, Sharp JW, Jain AK, et al. Chronic kidney disease in an electronic health record problem list: quality of care, ESRD, and mortality. Am J Nephrol 2014;39(4):288-296 [FREE Full text] [CrossRef] [Medline]

    ‎
    ACEi: angiotensin converting enzyme inhibitor
    ACR: urine albumin-creatinine ratio
    ARB: angiotensin II receptor blocker
    BP: blood pressure
    BPA: best practice advisory
    CKD: chronic kidney disease
    eCDSS: electronic clinical decision support system trial arm
    eCDSS+: electronic clinical decision support system and pharmacist follow-up trial arm
    eGFR: estimated glomerular filtration rate
    eGFR Cr: estimated glomerular filtration rate by serum creatinine
    eGFR Cys: estimated glomerular filtration rate by serum cystatin-c
    EHR: electronic health record
    GEE: generalized estimating equation
    ICD-10: International Classification of Diseases, Tenth Revision, Clinical Modification
    KDIGO: Kidney Disease: Improving Global Outcomes
    NP: nurse practitioner
    NSAID: nonsteroidal anti-inflammatory drug
    PCMH: primary care medical home
    PCP: primary care provider

    Edited by G Eysenbach; submitted 21.03.19; peer-reviewed by C Litvin, D Nash; comments to author 27.04.19; revised version received 03.05.19; accepted 05.05.19; published 07.06.19

    Copyright

    ©Elaine C Khoong, Leah Karliner, Lowell Lo, Marilyn Stebbins, Andrew Robinson, Sarita Pathak, Jasmine Santoyo-Olsson, Rebecca Scherzer, Carmen A Peralta. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 07.06.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.