Published on in Vol 5, No 2 (2016): Apr-Jun

Using an Electronic Decision Support Tool to Reduce Inappropriate Polypharmacy and Optimize Medicines: Rationale and Methods

Using an Electronic Decision Support Tool to Reduce Inappropriate Polypharmacy and Optimize Medicines: Rationale and Methods

Using an Electronic Decision Support Tool to Reduce Inappropriate Polypharmacy and Optimize Medicines: Rationale and Methods

Protocol

1New Zealand’s National School of Pharmacy, University of Otago, Dunedin, New Zealand

2Department of General Practice and Rural Health, Dunedin School of Medicine, University of Otago, Dunedin, New Zealand

3Dunedin School of Medicine, University of Otago, Dunedin, New Zealand

4Best Practice Advocacy Centre (BPAC), Dunedin, New Zealand

Corresponding Author:

Amber Young, BPharm, PGCertPharm, PGCertGPP

New Zealand’s National School of Pharmacy

University of Otago

PO Box 56

Dunedin, 9054

New Zealand

Phone: 64 34797321

Fax:64 3 479 7034

Email: amber.young@otago.ac.nz


Background: Polypharmacy and inappropriate continuation of medicines can lead to a significant risk of adverse drug events and drug interactions with patient harm and escalating health care costs as a result. Thorough review of patients’ medications focusing on the need for each drug can reduce the potential for harm. Limitations in performing effective medicine reviews in practice include consultation time constraints and funding for pharmacy services. We will aim to overcome these problems by designing an automatic electronic decision support tool (the medicines optimization/review and evaluation (MORE) module) that is embedded in general practice electronic records systems. The tool will focus on medicines optimization and reducing polypharmacy to aid prescribers in reviewing medicines and improve patient outcomes.

Objective: The objectives of this study are: (1) to develop an electronic decision support tool to assist prescribers in performing clinical medication reviews with a particular focus on patients experiencing multimorbidity and polypharmacy, and (2) evaluate and assess the use of the electronic decision support tool, providing pilot data on its usefulness in supporting prescribers during consultations with patients.

Methods: The first three study phases involve development of clinical rules outlining clinical interventions and the creation and validation of the MORE decision support tool. Phase four is a community-based, single-blind, prospective, 6-month controlled trial involving two interventions and two control general practices, matched for practice demographics. We will be measuring the number of times prescribers engage with the tool, total number of interventions suggested by the tool, and total number of times prescribers change medicines in response to recommendations. There will also be prospective follow-up of patients in the intervention group to examine whether changes to medications are upheld, and to determine the number of hospitalizations or emergency department visits within 6 months of a medicine intervention. Comparisons between control and intervention practices will measure the changes in proportions of patients with polypharmacy and inappropriately prescribed medicines before and after the introduction of the electronic decision support tool, proportions of patients receiving appropriate treatment in each practice, and changed, maintained, or improved health status, hospitalizations, and deaths in the study year. Initiation rates of inappropriately prescribed medicines will be measured as a secondary outcome. As well as external assessment of the extent of use and application of the tool, prescribers will receive monthly practice progress reports detailing the proportion of their patients experiencing polypharmacy and taking inappropriately prescribed medicines identified for review.

Results: Phase one has now been completed and the decision support tool is under development. Final data analysis is expected to be available in December 2016.

Conclusions: This study will establish whether the MORE decision support tool stands up to real world conditions and promotes changes in prescribing practice.

JMIR Res Protoc 2016;5(2):e105

doi:10.2196/resprot.5543

Keywords



Background

As the New Zealand population is rapidly ageing, there are increasing numbers of patients with more long-term conditions and taking more medicines. With declining organ function and introduction of comorbidities, increasing age may also influence the suitability of a patient’s long-term medications. Periodic assessment of patients’ medications should be undertaken, keeping in mind their remaining life expectancy, time until benefit of treatment, treatment targets, and goals of care [1].

Polypharmacy can be defined as the concurrent use of five or more medicines, and excessive polypharmacy, the use of 10 or more concurrent medicines [2]. Inappropriate polypharmacy occurs when more medicines are prescribed than are clinically indicated or when medicines are inappropriately continued [3]. Increasing the number of prescribed medicines greatly increases the risk of drug interactions and adverse drug events, resulting in iatrogenic patient morbidity [3-5]. The medical mantra of ‘First do no harm’ is at risk when patients’ are in danger of multimorbidity from cumulative prescribing of inappropriate medicines, particularly when this is compounded with altered pharmacodynamics from declining renal and hepatic function with age. Overall, there is a danger that patients’ medication regimens may begin to pose more risks than benefits to their health [5]. Between 2013 and 2014, 8.5% of the New Zealand population received five or more medicines and 2.6% received 11 or more medicines [2]. These proportions of the population with polypharmacy and hyperpolypharmacy are increasing in all age groups every year [2], particularly for 40- to 60-year olds [6].

Thorough review of patients’ medications focusing on the need for each drug can reduce the potential for harm [5,7]. A 2014 Cochrane review found that interventions to improve appropriate polypharmacy are beneficial in reducing inappropriate prescribing [3]. Despite the evidence, most people taking multiple medicines do not receive an annual comprehensive medicines review due to general practitioners’ limited consultation times. Tools exist to assist a review but are infrequently used due to being complex and time-consuming [8,9]. A scheme for collaborative medications reviews involving pharmacists exists in New Zealand (Medicines Therapy Assessment) but is funded in few regions. This deficiency is similar worldwide as many international models of primary care do not promote intensive medication reviews by a clinical pharmacist.

This study will attempt to overcome the problems in completing medication reviews by designing an automatic electronic decision support tool. These tools have been shown to influence prescriber performance, improve quality of care and patient outcomes [10], and reduce inappropriate prescribing [3]. The medicines optimization/review and evaluation (MORE) decision support tool will focus on medicines optimization to aid prescribers in reviewing medicines and improve patient outcomes while reducing inappropriate polypharmacy. The electronic decision support software will also provide continuous and reproducible medication reviews [11].

Recent studies have demonstrated benefits of electronic decision support tools, but the technology itself can become a burden on physicians’ time and patient management [12-15]. For successful implementation, decision support tools must be fast, reliable, and able to integrate into existing systems used in practice.

The 2012 review by Clyne et al [12] demonstrated that clinical decision support has potential to improve safe and effective prescribing in many different health care settings. Likewise, the review by Topinkova et al [16] from the same year found that decision support reduces prescribing errors. However, they also concluded that the real effect of these systems requires further study, focusing on health outcomes such as overall health care cost and patient morbidity and mortality. Further research is required to evaluate the acceptability of alerts to prescribers’ [16]. The 2014 Cochrane review, examined studies using specific validated screening tools or instruments, but did not include research on how doctors interact with the specific interventions, and based their recommendations for implementation of change on the recommendations presented to them [3].

This study will use an electronic decision support platform currently available in over 80% of New Zealand general practices and compatible with the most common patient management system in use. We expect that prescribers’ familiarity with the decision support platform will enable immediate uptake and application of the MORE tool. We will also investigate the interaction of the doctor with the decision support tool to evaluate its’ effectiveness and usefulness in practice.

In summary, the significance of this study is the development of a new decision support tool that uses each patient’s medical information to automatically undertake a medicines review and assist prescribers with patient management. Personalized medicine management strategies help ensure that patients receive the most appropriate care and avoids the risk of ‘alert fatigue’. Broad use of our tool is intended to improve clinical outcomes, and reduce health care use and cost. The evaluation and review of users’ feedback will enable the production of a practical and efficient decision support tool.

Research Aims

The purpose of this study is to develop and test an electronic decision support tool (the MORE module) designed to assist prescribers in performing clinical medication reviews for patients experiencing multimorbidity and polypharmacy. The study will (1) provide pilot data on its usefulness in supporting prescribers during consultations with patients, and (2) test the hypothesis that there is no difference in the identification and management of patients with multiple medicines between the practices implementing the MORE decision support tool and usual care practices.


Phase One: The Development of the Clinical Rules for the MORE Decision Support Tool

Phase one involved the development of the intervention components to be incorporated into the MORE decision support tool. Literature reviews, internationally validated tools, and prescribing resources were used to identify possible intervention components (see Figure 1). A clinical advisory group was set up to provide expert opinion and direction for the development of the intervention components. This group consisted of 10 clinicians.

The intervention components focused on specific areas for change, including:

1. Reducing polypharmacy by targeting medicines with limited effectiveness, such as long-term use of benzodiazepines for insomnia, or antipsychotics in dementia

2. Stopping duplicate medication classes, such as duplicate antidepressant therapy

3. Reviewing doses and monitoring certain medications such as proton pump inhibitors, anticoagulants, and analgesia

To prioritize the list of interventions, medicines dispensing data were reviewed using the Pharmaceutical Collection, a national database of all community pharmacy dispensed medicines. Possible interventions for medicines not dispensed in large numbers (<5000 dispensing’s a year nationally) were excluded. Possible interventions remaining were reviewed by the clinical advisory group to determine the appropriateness of the recommendations for the New Zealand context. The group assessed the importance of each medicine intervention using a list of references provided and their own clinical expertise. International specialist opinion was also sought. Descriptions of the interventions were modified following comments by the group, and the proposed medicine interventions with the greatest consensus of agreement were selected as the final intervention components.

These final medicine intervention components are expected to have the most impact on patient care and polypharmacy, and were used to form the clinical rules of the MORE decision support tool. See Figure 1 for an outline of the intervention development process.

Phase Two: Development of the MORE Decision Support Tool

Decision support programmers (from BPAC Inc) will use the medicine intervention components generated in phase one to create the clinical rules in the MORE decision support tool. This tool will automatically interact with the prescribing component of general practice electronic records systems and access individual patients’ demographic and clinical information to make recommendations to prescribers. The program developers have previously built the Best Practice Advocacy Centre (BPAC) decision support tools, ensuring that the look, feel, and function of the MORE tool are similar to that of the current BPAC decision support tools that are in widespread use in New Zealand. These tools are also currently in use in Australia and soon to be introduced into the United Kingdom.

The decision support architecture will be using open electronic health record (open EHR) for the patient object model, which is an open international standard. This allows for the inclusion of any coding system from any underlying patient management system (PMS) to be integrated into the patient object model. The patient object model also allows for integration with any PMS application program interface (API), allowing for rapid international roll out. Interoperability is also enabled through the mapping of drug codes, medical classification, laboratory codes, and measurement parameters to the systematized nomenclature of medicine clinical terminology (SNOMED CT) using an in-house ontology service. Furthermore, the clinical rules driving the prescribing advice reside in a rules engine, allowing additional rules and functionality to be added independently of any programming interfaces. Figure 2 shows the platform architecture of the decision support module. Not all components will be used for this study, such as the SMS text messaging (short message service, SMS) function, but will be available for future roll out. This platform architecture enables rapid scalability.

The MORE decision support tool will alert prescribers through the patient prompt: this analyses patient records at the time of consultation, notifying clinicians of any areas where action may be required. Recommendations will be based on individual patients’ data and therefore are specific for each patient.

Simple representation of targeted advice and prescribing alternatives is more effective than highlighting a medicine that may be inappropriate [17]. Therefore, this decision support tool will alert prescribers to medicines within patient records that are potentially inappropriate with suggested action based on each patient’s individual data. There will also be a link to electronic information supporting recommended actions, where relevant. The decision support tool will also allow linking to patient information and advice that can be printed.

Figure 1. Development of the clinical rules for the MORE decision support tool.
View this figure
Figure 2. Platform architecture of the decision support tool.
View this figure

Phase Three: Validation of the MORE Decision Support Tool

The MORE decision support tool will be validated by challenging it with data from a de-identified historical database. This will ensure that the clinical rule set is working correctly. The decision support tool will be applied to a variety of patients in the database (patients with ≥10 medicines and patients with only a few medicines) to confirm that patient data are autopopulated into the decision support tool correctly and whether an alert is triggered by the appropriate information. This phase will also allow technical checks such as the linking to external information and printing functionality. Personal practitioner progress reports will also be generated to ensure this functionality works correctly.

Phase Four: Application of the MORE Decision Support Tool

Phase four is the feasibility study of the MORE decision support tool using MORE care versus usual care to reduce unnecessary or inappropriate prescribing. The MORE decision support tool will be piloted for 6 months in two intervention general practices. A comparison in prescribing behavior and outcomes will be made between the practices and two matched control practices. This will establish whether the decision support tool stands up to real world conditions and causes a change in prescribing practices.

During the 6 months that the MORE decision support tool is active, prescribers will receive individual monthly progress reports that have been extracted remotely by BPAC analysts using the MORE decision support tool. The reports will detail the proportion of patients at their practice experiencing polypharmacy, and patients taking prescribed medicines identified for review.

This information will include patient demographics and the total number of times prescribers in the practice have engaged with the MORE decision support tool with resulting changes they may have implemented. They will also be informed of the clinical rules that, following the alert, have resulted in changes to patients’ therapy the most and the least number of times.

As the study progresses the reports will depict the changes in these variables over time. Using the decision support tool query builder, prescribers will also be able to access lists of patients experiencing polypharmacy and those taking prescribed medicines identified for review, any time during the project.

Study Setting

The MORE decision support tool will be provided to two intervention general practices recruited by invitation. In order for a practice to be considered for inclusion, it must be using the MedTech software, which is in use in over 80% of New Zealand practices. Practices will be excluded from the study if they are specialist practices (eg, a sexual health clinic, student health clinic), or if they have less than 1500 registered patients. Of the practices wishing to participate, two will be randomly assigned as the MORE decision support practices and two control practices will be matched according to practice patient demographics (mean age, gender, ethnicity, and geography (urban/rural)).

The MORE decision support tool will be added to the intervention and control practices’ suite of BPAC decision support tools, although in the control practices, the alerts will not be revealed at point-of-care. Instead, the type and frequency of the alerts that would have been shown will be recorded for comparison against those in the intervention group. The decision support tool can be added to the decision support platform remotely because it is a Web-based system that allows upgrades or revisions to existing decision support tools to be implemented through one centralized server. This will minimize disruption to the daily activities of study general practices.

Sample Size

Between 2013 and 2014, 11% of the New Zealand population used five or more medicines [2]. A sample size of 367 patients for each arm (the intervention and the control) will have 80% power to detect a difference in patients with polypharmacy of 20% at the 5% level of significance.

We intend to recruit four practices with 1500 or more registered patients. The average number of patients per practice in New Zealand is approximately 4000 [18], so our goal of practices with 1500 or more registered patients is achievable. Ethical approval has been obtained from the Health and Disability ethics committee (16/STH/7).

Measures

This study will test the hypothesis that there is no difference between the practices using the MORE decision support tool and usual care practices in the identification of patients with polypharmacy and the identification and review of patients who have inappropriately prescribed medicines.

Due to the nature of the study, it is not possible to blind the general practices to the intervention. However, the data retrieval from the general practices is through an automated Web-based system and a blinded analysis of the measures comparing intervention and control groups will be undertaken. This allows for unbiased assessment of these outcomes.

We plan to assess the use of the MORE decision support tool by seeing how often general practitioners use the tool and follow its recommendations. We will calculate how often the alert for review was raised and compare this with changes in prescribing. The number of patients experiencing inappropriate medicines (as defined by the clinical rules in the decision support tool) and polypharmacy before and after the study will also be reviewed, as a proxy measure of effectiveness of the intervention. See Table 1 for detailed descriptions of study measures and timeframes.

Table 1. Study Outcomes and How They Will be Measured.
OutcomesHow this will be measuredTimeframe will this be measured
Practitioners use the toolThe number of times prescribers engage with the MORE decision support tool in the intervention practices (how many times prescribers choose to look at possible interventions)Every month during the 6-month pilot phase

The total number of interventions suggested by the MORE decision support tool in the intervention practices (how many instances of unnecessary or inappropriate medicines)Every month during the 6-month pilot phase

Interview and questionnaire containing the standardized System Usability Scale and qualitative questions on usability and usefulnessAfter completion of pilot phase
Practitioners follow the recommendationsThe total number of times prescribers change medicines in response to the MORE decision support tool in the intervention practices (ie, remove unnecessary or inappropriate medicines)Every month during the 6-month pilot phase

The number of changes sustained or reverted to original prescribing methods within 6 months of the intervention in the intervention practicesSix months after completion of pilot phase

The difference in number of laboratory tests ordered between intervention and control practices in line with suggestions from the MORE decision support toolEvery month during the 6-month pilot phase
Reduction in inappropriate prescribing and polypharmacyThe change in the proportion of patients with inappropriately prescribed medicinesa before and after the introduction of the decision support tool (adjusted for age and comorbidities) between intervention and control practices. Specifically:
1. The number and proportion of patients with hyperpolypharmacy (more than nine medicines [2]) before and after intervention
2. The number and proportion of patients with polypharmacy (more than four medicines [ 2]) before and after intervention
3. The average number of medicines per patient before and after intervention
At baseline and at end of the 6-month pilot phase

The percentage of patients receiving appropriate treatment based on the developed clinical rules in each practice between intervention and control practicesAt baseline and at end of the 6-month pilot phase

Initiation rates of inappropriately prescribed medicines based on the developed clinical rules in each practice between intervention and control practicesEvery month during the 6-month pilot phase

Initiation rates of appropriately prescribed medicines according to the developed clinical rules in each practice between intervention and control practicesEvery month during the 6-month pilot phase
Improvement in patient healthChanged, maintained, or improved health status measured by number of patient visits to general practices and hospital or emergency department admissions between intervention and control practicesFrom 6 months before study enrolment to 6 months after completion of the pilot phase

Deaths in the study year between intervention and control practicesFrom 6 months before study enrolment to 6 months after completion of the pilot phase

The number of related referrals, emergency department visits or hospitalizations within 6 months of the intervention in the intervention practicesDuring the 6-month pilot phase and for 6 months after completion of the pilot phase

aThis assumes that on consideration of the MORE decision support tool guidance, only inappropriately prescribed medicines will be stopped (therefore, a decrease in inappropriate polypharmacy will occur).

Cost Analysis

Potential cost savings through analysis of reduced hospitalizations, emergency department visits, and visits to the general practitioner following implementation of the decision support tool will be calculated after 6 months. The costs associated with stopping or starting medicines as recommended by the MORE decision support tool during the study period will also be evaluated. This will be balanced against any consultation slow down’s due to using the tool. There will also be an estimation of the time saved using the MORE decision support tool compared with alternative methods of medicine review.

Interviews

The intervention group will be interviewed to elicit feedback on the usefulness of the decision support tool and its usability and presentation. This will provide the basis for future improvements for the decision support tool. During the interview a questionnaire will be completed containing the system usability scale, a validated tool for measuring a systems usability [19], to elicit participants’ satisfaction with the decision support tool. Reasons for and against following the decision support tool’s interventions and the differences in managing patients with multiple medicines before and during the use of the MORE decision support tool will also be explored. This will determine whether the decision support tool improves identification of patients with polypharmacy and those who have inappropriately prescribed medicines.

The control group will also be interviewed to understand how they regularly identify and manage patients with polypharmacy. Also, to see who has inappropriately prescribed medicines, what percentage of their patients they estimate belong to these groups, and how often they would undertake a medicine review in an average week.


Phase one has now been completed and the decision support tool is under development. See Textbox 1 for the clinical rules to be built into the decision support tool. Phases two and three are expected to be finalized early 2016 with implementation and analysis of the decision support tool by October 2016. Results will be reported in December 2016.


Reducing Inappropriate Polypharmacy

New Zealand general practice is facing increasing challenges in caring for a growing number of patients with long-term conditions. This increases the complexity of primary care interactions and makes high quality medicines review more difficult to achieve within available consultation time frames. It is well recognized that the quality use of medicines leads to decreased medicine interactions, reduced health care use (including hospitalizations), and improved quality of life [72,73]. Therefore, the individualized approach to medicine review undertaken in this study will promote a safe and effective means of practicing.

The premise of optimizing medicines and the reduction of inappropriate polypharmacy is about finding the best available medication at the right dosage and for the shortest possible duration on a case-by-case basis. This decision support tool will assist prescribers in achieving these goals by collating relevant patient information automatically using the prescribing patient management system and making individual recommendations for these complex patients. This will reduce the need for manual processing, the gold standard for reviewing medicines, which is a costly and timely enterprise.

The project partnership with BPAC Inc will allow the transfer of this study from evidence to practice as, if successful, the electronic decision support decision support tool could be directly rolled out to most general practices in New Zealand, through existing BPAC systems and networks. If the MORE module is successful in achieving its goals, the clinical rules could be applied across primary care settings, internationally.

Study Strengths and Limitations

Using a full range of clinical rules rather than focusing on one condition or drug class is a strength of this study. It allows for inclusion of rules applicable to the whole population and for a wide variety of therapeutic interventions. It also enables further development of the tool without restriction to a singular therapeutic condition. Using real-world locations ensures clinical relevance and applicability of the tool.

This is a fully automated and prepopulated tool. There is no manual data entry required by general practitioners and the patient management system will be one they are currently using in their practice. This will ensure minimal time for setup and the practitioners’ time can be focused on the clinical interventions rather than the technical aspect of inputting data. Moreover, training of practitioners will not need to be extensive and they will already have working knowledge of how this type of alert works.

An extra strength and uniqueness of the study is the content of the tool. It will provide suggested actions for prescribers rather than just highlighting a potentially inappropriate medicine. It is believed this will improve the use of the clinical rules by giving prescribers evidence-based guidance on how these rules should be applied to their patients. The tool will also contain links to patient information and options for printing patient information sheets.

One limitation is the inability to blind the clinicians to the intervention. However, because the analysis of the data will be blinded, objective assessment of results will be possible and is an additional strength.

A further limitation of the study is that the tool is being designed for use with a specific PMS in New Zealand. However, our patient object model allows integration with any PMS API, allowing for the solution to be rolled out internationally. If international roll out is an outcome of this pilot study, further usability studies will be necessary for users of different systems.

The measures of effectiveness are not able to capture the reasons why an alert may be dismissed by a practitioner at the time of consultation. It is hoped that the main reasons alerts are being ignored will be captured through feedback and the user survey. The initial results will help shape the questionnaire and interview of the practitioners so these issues can be explored.

It will be difficult to specify patient deaths and hospital admissions that are not related to the intervention. However, we can investigate the causes of deaths and admission to hospital for those with a medicine-related diagnosis for the purposes of the study.

There is a risk of alert fatigue, particularly if a patient with extensive polypharmacy is triggering many alerts. We have tried to minimize this by ensuring alerts are specific to patients’ characteristics. Furthermore, we will investigate complaints of this nature highlighted in the user feedback reports and work to reduce or eliminate them.

Conclusions

Optimizing medicines use is not necessarily about reducing the number of medicines below an agreed threshold, but about finding the best available medication at the right dosage and for the shortest possible duration on a case-by-case basis. Measuring reductions in the number of patients experiencing polypharmacy and the average number of medicines per patient will indicate removal of unnecessary medicines and the ‘inappropriate polypharmacy’ from their regimen. This study will establish whether the MORE decision support tool stands up to real world conditions and promotes changes in prescribing practice.

Clinical Rules to be Built Into the Decision Support Tool.

Duplicate therapy [8]

Stop if duplication of drug class or therapy:

  • H2-receptor antagonists with proton pump inhibitors
  • Duplication of benzodiazepines
  • Duplication of antipsychotics
  • Duplication of selective serotonin receptor antagonists

Gastroprotectants [7,8,20,21]

  • Stop proton pump inhibitors if they were prescribed for gastroprotection with nonsteroidal anti-inflammatory drugs, aspirin, or corticosteroid therapy, which has now been stopped
  • Consider stopping or reducing dose of proton pump inhibitors prescribed for uncomplicated peptic ulcer disease or erosive peptic esophagitis for >8 weeks

Chronic constipation [8,9,22]

In patients with chronic constipation, stop drugs likely to cause constipation if nonconstipating alternatives are appropriate

Antiplatelets and anticoagulants [8,9,23-28]

Stop vitamin K antagonist, direct thrombin inhibitor, or factor Xa inhibitors:

  • if prescribed for first deep venous thrombosis without continuing provoking risk factors for >6 months
  • if prescribed for first pulmonary embolus without continuing provoking risk factors for >12 months

Stop aspirin if taken for primary prevention of cardiovascular disease if risk is <20% and there is no personal history of cardiovascular disease (ie, angina, myocardial infarction, percutaneous coronary intervention, coronary artery bypass grafting, transient ischemic attack, ischemic stroke, or peripheral vascular disease)

Medicines in the elderly [8,9,29-37]

  • Stop long-term treatment with loop diuretics treating gravitational edema (unrelated to congestive heart failure) in the elderly. If pharmacological treatment necessary, prescribe as required
  • Reduce spironolactone if >25-mg daily in elderly patients with congestive heart failure or with creatinine clearance less than 30 mL/min
  • Review patients >85 years taking statins for >5 years for primary cardiovascular prevention
  • Stop benzodiazepines or zopiclone if taken for treatment of insomnia, agitation, or delirium in adults aged >65 years
  • Stop antipsychotics for behavioral problems of dementia unless nonpharmacological options have failed, and patient is a threat to themselves or others
  • Stop first-generation antihistamines in patients >75 years
  • Stop orphenadrine in patients >75 years

Antipsychotics, antidepressants, and hypnotics [8,9,27,31,38-40]

  • Stop all antipsychotics (except for quetiapine and clozapine), metoclopramide, prochlorperazine, and promethazine in patients with Parkinson's disease
  • Reduce citalopram doses ˃40-mg daily
  • Stop benzodiazepines or zopiclone if taken for ≥4 weeks (unless for seizure disorders, rapid eye movement sleep disorders, benzodiazepine withdrawal, alcohol withdrawal, severe generalized anxiety disorder, periprocedural anesthesia, end-of-life care)

Analgesics [31,41-47]

  • Review opioids if being prescribed long term (>3 months) for nonmalignant pain
  • Stop two different types of long-acting opioid
  • Stop codeine or tramadol if prescribed with a strong opioid
  • Stop combination paracetamol and codeine products and prescribe individual components based on the World Health Organization pain ladder

Nonsteroidal anti-inflammatory drugs [8,9,27,48,49]

  • Review stopping nonsteroidal anti-inflammatory drugs:
    • if used for ˃3 months in patients over 75 years
    • if used for ˃3 months in patients for symptom relief of osteoarthritis pain where paracetamol has not been tried
  • Stop nonsteroidal anti-inflammatory drugs:
    • if prescribed with concurrent antiplatelet or prescribe proton pump inhibitor prophylaxis
    • in patients with heart failure
    • in patients being treated with a diuretic or an Angiotensin Converting Enzyme inhibitor or Angiotensin Receptor Blocker
    • if patients already taking a nonsteroidal anti-inflammatory drug (ie, duplicate therapy)

Gout [8,50-52]

  • Prescribe allopurinol for patients with a history of recurrent episodes of gout (recurrent nonsteroidal anti-inflammatory drug or colchicine prescriptions), or a suitable alternative if allopurinol contraindicated
  • Alert if patient is on allopurinol and no uric acid levels or renal function tests for >1 year
  • Maximize dose of allopurinol or add additional therapy if uric acid not ˂0.36 mmol/L

Bisphosphonates [8,53,54]

  • Consider stopping bisphosphonate treatment or a ‘drug holiday’ after continuous use for >5 years (for treatment or prevention of osteoporosis) if bone mineral density stabilized
  • Consider initiating bisphosphonates in patients taking long-term systemic corticosteroid therapy

Metformin [31,55]

Adjust dose of metformin in renal impairment

Seasonal influenza vaccine [8,56-59]

Recommend an annual seasonal influenza vaccine to:

  • patients ≥65 years of age
  • patients with ischemic heart disease
  • patients with congestive heart failure
  • patients with rheumatic heart disease
  • patients with Transient Ischemic Attack/Stroke
  • patients with asthma, if on a regular preventative therapy
  • patients with chronic obstructive pulmonary disorder
  • patients with diabetes
  • patients with any cancer, excluding basal and squamous skin cancers if not invasive
  • patients with human immune deficiency virus
  • transplant recipients
  • pre and post splenectomy patients
  • pregnant patients

Interactions [8,27,31,60-64]

Alert for the following interactions:

  • Tricyclic antidepressants and selective serotonin receptor blockers
  • Beta-blocker with verapamil or diltiazem
  • Concomitant use of two or more antimuscarinics
  • Concomitant drugs that prolong the QT-interval

Monitoring [8,65-71]

  • Monitor potassium levels if not done >6 months in patients taking spironolactone with potassium sparing drugs
  • Stop potassium supplements if serum potassium >4.0 mmol/L and if cause of hypokalemia resolved. Consider follow up potassium levels after cessation
  • Alert if on lithium and no thyroid function tests, renal function tests, serum lithium levels, or sodium levels for >6 months, or no calcium levels or electrocardiogram undertaken for >1 year
  • Alert if on an Angiotensin Converting Enzyme inhibitor and no renal function or serum potassium for >1 year
  • Alert if on an atypical antipsychotic for schizophrenia and no CV assessment, full blood count, urea and electrolytes, liver function tests, lipid profile, weight measurement, fasting blood glucose, or prolactin levels for >1 year
Textbox 1. Clinical Rules to be Built Into the Decision Support Tool.

Acknowledgments

This work was supported by the Health Research Council of New Zealand grant number 14/730.

With thanks to the Clinical Advisory Group: Murray Tilyard, Sharon Leitch, Sharyn Willis, Hywel Lloyd, Lalitha Raman-Wilms, Howard Wilson, Tobias Dreischulte, Bruce Guthrie, Rupert Payne, Chris Leathart, and Marcus Hawkins.

Authors' Contributions

AY had primary responsibility for the manuscript. AS conceived the study, wrote the original protocol, and assisted with development of the manuscript. JT assisted in developing the protocol and read and revised the final manuscript. SD, HL, and MT gave feedback on various medical and technical issues, contributed to refinement of the study protocol, and revised the manuscript.

Conflicts of Interest

Professor Murray Tilyard is Chief Executive of BPAC Inc, the organization that will develop the MORE decision support software.

Multimedia Appendix 1

Grant agency peer-review reports.

PDF File (Adobe PDF File), 576KB

  1. Holmes HM, Hayley DC, Alexander GC, Sachs GA. Reconsidering medication appropriateness for patients late in life. Arch Intern Med 2006;166:605-609. [CrossRef] [Medline]
  2. BPAC. Polypharmacy Report. 2014.   URL: http://www.bpac.org.nz/Report/2014/November/polypharmacy.aspx [accessed 2016-01-05] [WebCite Cache]
  3. Patterson SM, Cadogan CA, Kerse N, Cardwell CR, Bradley MC, Ryan C, et al. Interventions to improve the appropriate use of polypharmacy for older people. Cochrane Database Syst Rev 2014;10:CD008165. [CrossRef] [Medline]
  4. Goldberg RM, Mabee J, Chan L, Wong S. Drug-drug and drug-disease interactions in the ED: analysis of a high-risk population. Am J Emerg Med 1996;14:447-450. [CrossRef] [Medline]
  5. BPAC. Best Practice Journal. 2010. A Practical Guide to Stopping Medicines in Older People   URL: http://www.bpac.org.nz/BPJ/2010/April/stopguide.aspx [accessed 2016-05-12] [WebCite Cache]
  6. Smith A, Dovey S, Lloyd H, Tilyard M. The prevalence of chronic conditions in New Zealand primary care patients. Surry Hills: NPS MedicineWise; 2014 Presented at: National Medicines Symposium; 21-23 May 2014; Brisbane, Australia.
  7. Garfinkel D, Mangin D. Feasibility study of a systematic approach for discontinuation of multiple medications in older adults: addressing polypharmacy. Arch Intern Med 2010;170:1648-1654. [CrossRef] [Medline]
  8. O'Mahony D, O'Sullivan D, Byrne S, O'Connor MN, Ryan C, Gallagher P. STOPP/START criteria for potentially inappropriate prescribing in older people: version 2. Age Ageing 2015;44:213-218 [FREE Full text] [CrossRef] [Medline]
  9. American Geriatrics Society 2012 Beers Criteria Update Expert Panel. American Geriatrics Society updated Beers Criteria for potentially inappropriate medication use in older adults. J Am Geriatr Soc 2012;60:616-631 [FREE Full text] [CrossRef] [Medline]
  10. BPAC. Best Practice Journal. 2013. Five Tips for Getting the Most out of Your Practice Management System   URL: http://www.bpac.org.nz/BPJ/2013/November/tips.aspx [accessed 2016-05-12] [WebCite Cache]
  11. de Wit HA, Mestres GC, Hurkens KP, Mulder WJ, Janknegt R, Verhey FR, et al. Development of a computer system to support medication reviews in nursing homes. Int J Clin Pharm 2013;35:668-672. [CrossRef] [Medline]
  12. Clyne B, Bradley MC, Hughes C, Fahey T, Lapane KL. Electronic prescribing and other forms of technology to reduce inappropriate medication use and polypharmacy in older people: a review of current evidence. Clin Geriatr Med 2012;28:301-322. [CrossRef] [Medline]
  13. Hecht T, Bundscherer AC, Lassen CL, Lindenberg N, Graf BM, Ittner K, et al. The expenditure of computer-related worktime using clinical decision support systems in chronic pain therapy. BMC Anesthesiol 2015;15:113 [FREE Full text] [CrossRef] [Medline]
  14. Meulendijk MC, Spruit MR, Drenth-van Maanen AC, Numans ME, Brinkkemper S, Jansen PA, et al. Computerized decision support improves medication review effectiveness: an experiment evaluating the STRIP assistant's usability. Drugs Aging 2015;32:495-503 [FREE Full text] [CrossRef] [Medline]
  15. Tamblyn R, Huang A, Perreault R, Jacques A, Roy D, Hanley J, et al. The medical office of the 21st century (MOXXI): effectiveness of computerized decision-making support in reducing inappropriate prescribing in primary care. CMAJ 2003;169(6):549-556. [Medline]
  16. Topinková E, Baeyens JP, Michel J, Lang P. Evidence-based strategies for the optimization of pharmacotherapy in older people. Drugs Aging 2012;29:477-494. [CrossRef] [Medline]
  17. Ranta A, Dovey S, Weatherall M, O'Dea D. Efficacy and safety of a TIA/stroke electronic support tool (FASTEST) trial: study protocol. Implement Sci 2012;7:107 [FREE Full text] [CrossRef] [Medline]
  18. Ministry Of Health. Primary Health Organisation Enrolment Collection. 2012.   URL: http:/​/www.​health.govt.nz/​nz-health-statistics/​national-collections-and-surveys/​collections/​primary-health-organisation-enrolment-collection [accessed 2016-01-05] [WebCite Cache]
  19. U.S. Department of Health & Human Services. System Usability Scale (SUS). 2016.   URL: http://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html [accessed 2016-02-29] [WebCite Cache]
  20. BPAC. Best Practice Journal. 2011. Managing Dyspepsia and Heartburn in General Practice - An Update   URL: http://www.bpac.org.nz/BPJ/2011/february/dyspepsia.aspx [accessed 2016-05-12] [WebCite Cache]
  21. Farrell BPK, Thompson W, Boghossian T, Pizzola L, Rashid J, Rojas-Fernandez C, et al. Evidence-Based Clinical Practice Guideline for Deprescribing Proton Pump Inhibitors. 2015.   URL: http://www.open-pharmacy-research.ca/wordpress/wp-content/uploads/ppi-deprescribing-algorithm-cc.pdf [accessed 2016-01-05] [WebCite Cache]
  22. Waitemata District Health Board (WDHB). SafeRx. 2014. Clozapine - Safe Prescribing - We are Counting on You   URL: http://www.saferx.co.nz/full%5Cclozapine.pdf [accessed 2016-01-05] [WebCite Cache]
  23. Keeling D, Baglin T, Tait C, Watson H, Perry D, Baglin C, British Committee for Standards in Haematology. Guidelines on oral anticoagulation with warfarin - fourth edition. Br J Haematol 2011;154:311-324. [CrossRef] [Medline]
  24. New Zealand Guidelines Group. New Zealand Primary Care handbook. 2012. Cardiovascular Disease Risk Assessment (Updated 2013)   URL: https:/​/www.​health.govt.nz/​system/​files/​documents/​publications/​cardiovascular-disease-risk-assessment-updated-2013-dec13.​pdf [accessed 2016-01-05] [WebCite Cache]
  25. Antithrombotic Trialists' (ATT) Collaboration, Baigent C, Blackwell L, Collins R, Emberson J, Godwin J, et al. Aspirin in the primary and secondary prevention of vascular disease: collaborative meta-analysis of individual participant data from randomised trials. Lancet 2009;373:1849-1860 [FREE Full text] [CrossRef] [Medline]
  26. Seshasai SR, Wijesuriya S, Sivakumaran R, Nethercott S, Erqou S, Sattar N, et al. Effect of aspirin on vascular and nonvascular outcomes: meta-analysis of randomized controlled trials. Arch Intern Med 2012;172:209-216. [CrossRef] [Medline]
  27. Avery AJ, Dex GM, Mulvaney C, Serumaga B, Spencer R, Lester HE, et al. Development of prescribing-safety indicators for GPs using the RAND Appropriateness Method. Br J Gen Pract 2011;61:e526 [FREE Full text] [CrossRef] [Medline]
  28. BPAC. Best Practice Journal. 2011. The Use of Antithrombotic Medicines in General Practice: A Consensus Statement   URL: http://www.bpac.org.nz/BPJ/2011/october/antithrombotic.aspx [accessed 2016-05-12] [WebCite Cache]
  29. Kolbach DN, Sandbrink MW, Hamulyak K, Neumann HA, Prins MH. Non-pharmaceutical measures for prevention of post-thrombotic syndrome. Cochrane Database Syst Rev 2004:CD004174. [CrossRef] [Medline]
  30. Simon C. Prescribing for the elderly. InnovAiT 2008;1:813-819. [CrossRef]
  31. New Zealand Formulary (NZF). New Zealand Formulary Release 43. 2016.   URL: http://nzf.org.nz/home/noaccess [accessed 2016-01-05] [WebCite Cache]
  32. Tidy C. Patient. 2014. Prescribing for the Older Patient   URL: http://m.patient.media/pdf/241.pdf?v=635737391117055516 [accessed 2016-01-05] [WebCite Cache]
  33. Holmes HM, Min LC, Yee M, Varadhan R, Basran J, Dale W, et al. Rationalizing prescribing for older patients with multimorbidity: considering time to benefit. Drugs Aging 2013;30:655-666 [FREE Full text] [CrossRef] [Medline]
  34. Waitemata District Health Board (WDHB). SafeRx. 2015. Simvastatin - Safe Prescribing -Time to Chew the Fat   URL: http://www.saferx.co.nz/full%5Csimvastatin.pdf [accessed 2016-01-05] [WebCite Cache]
  35. RANZCP Faculty of Psychiatry of Old Age (New Zealand). The Use of Antipsychotics in Residential Aged Care. Clinical Recommendations Developed by the RANZCP Faculty of Psychiatry of Old Age (New Zealand). 2008.   URL: http://www.bpac.org.nz/a4d/resources/docs/RANZCP_Clinical_recommendations.pdf [accessed 2016-01-05] [WebCite Cache]
  36. Ballard C, Lana MM, Theodoulou M, Douglas S, McShane R, Jacoby R, Investigators DART AD. A randomised, blinded, placebo-controlled trial in dementia patients continuing or stopping neuroleptics (the DART-AD trial). PLoS Med 2008;5:e76 [FREE Full text] [CrossRef] [Medline]
  37. Ballard CG, Thomas A, Fossey J, Lee L, Jacoby R, Lana MM, et al. A 3-month, randomized, placebo-controlled, neuroleptic discontinuation study in 100 people with dementia: the neuropsychiatric inventory median cutoff is a predictor of clinical outcome. J Clin Psychiatry 2004;65:114-119. [Medline]
  38. Waitemata District Health Board (WDHB). SafeRx. 2013. Citalopram and Escitalopram - Same but Different   URL: http://www.saferx.co.nz/full/citalopram_escitalopram.pdf [accessed 2016-01-05] [WebCite Cache]
  39. BPAC. Best Practice Journal. 2008. Managing Insomnia   URL: http://www.bpac.org.nz/BPJ/2008/June/insomnia.aspx [accessed 2016-05-12] [WebCite Cache]
  40. Waitemata District Health Board (WDHB). SafeRx. 2012. Hypnotics - Safe Prescribing - Dying for a Good Sleep?   URL: http://www.saferx.co.nz/full%5CHypnotics.pdf [accessed 2016-01-05] [WebCite Cache]
  41. BPAC. Best Practice Journal. 2014. Helping Patients Cope With Chronic Non-Malignant Pain: It's Not About the Opioids   URL: http://www.bpac.org.nz/BPJ/2014/September/chronicpain.aspx [accessed 2016-05-12] [WebCite Cache]
  42. BPAC. Best Practice Journal. 2014. Identifying and Managing Addiction to Opioids   URL: http://www.bpac.org.nz/BPJ/2014/October/opioid-addiction.aspx [accessed 2016-05-12] [WebCite Cache]
  43. Murnion BP. Australian Prescriber. 2010. Combination Analgesics in Adults   URL: http://www.australianprescriber.com/magazine/33/4/113/5 [accessed 2016-05-12] [WebCite Cache]
  44. de Craen J, Di GG, Lampe-Schoenmaeckers JE, Kessels AG, Kleijnen J. Analgesic efficacy and safety of paracetamol-codeine combinations versus paracetamol alone: a systematic review. BMJ 1996;313:321-325 [FREE Full text] [Medline]
  45. Arora S, Herbert ME. Myth: codeine is a powerful and effective analgesic. West J Med 2001;174:428 [FREE Full text] [Medline]
  46. Toms L, Derry S, Moore RA, McQuay HJ. Single dose oral paracetamol (acetaminophen) with codeine for postoperative pain in adults. Cochrane Database Syst Rev 2009(1):CD001547 [FREE Full text] [CrossRef] [Medline]
  47. Hunter Integrated Pain Service. NSW Agency for Clinical Innovation. 2014. Reconsidering Opioid Therapy - A Hunter New England perspective   URL: http:/​/www.​aci.health.nsw.gov.au/​__data/​assets/​pdf_file/​0015/​212226/​Reconsidering_Opioid_Therapy_1.​pdf [WebCite Cache]
  48. Dreischulte T, Grant A, Donnan P, McCowan C, Davey P, Petrie D, et al. A cluster randomised stepped wedge trial to evaluate the effectiveness of a multifaceted information technology-based intervention in reducing high-risk prescribing of non-steroidal anti-inflammatory drugs and antiplatelets in primary medical care: the DQIP study protocol. Implement Sci 2012;7:24 [FREE Full text] [CrossRef] [Medline]
  49. Waitemata District Health Board (WDHB). SafeRx. 2014. The Triple Whammy - Safe Prescribing - A Dangerous Trio   URL: http://www.saferx.co.nz/triplewhammy.pdf [accessed 2016-01-05] [WebCite Cache]
  50. BPAC. Best Practice Journal. 2013. An Update on the Management of Gout   URL: http://www.bpac.org.nz/BPJ/2013/March/managing-gout.aspx [accessed 2016-05-12] [WebCite Cache]
  51. BPAC. Best Practice Journal. 2011. The Medical Management of Gout Revisited   URL: http://www.bpac.org.nz/BPJ/2011/august/gout.aspx [accessed 2016-05-12] [WebCite Cache]
  52. Waitemata District Health Board (WDHB). SafeRx. 2015. Allopurinol - Safe Prescribing - Dose Up   URL: http://www.saferx.co.nz/full/allopurinol.pdf [accessed 2016-01-05] [WebCite Cache]
  53. Brown JP, Morin S, Leslie W, Papaioannou A, Cheung AM, Davison KS, et al. Bisphosphonates for treatment of osteoporosis: expected benefits, potential harms, and drug holidays. Can Fam Physician 2014;60:324-333 [FREE Full text] [Medline]
  54. Diab DL, Watts NB. Bisphosphonate drug holiday: who, when and how long. Ther Adv Musculoskelet Dis 2013;5:107-111 [FREE Full text] [CrossRef] [Medline]
  55. Waitemata District Health Board (WDHB). SafeRx. 2013. Metformin - Safe Prescribing - First-Line   URL: http://www.saferx.co.nz/full/Metformin.pdf [accessed 2016-01-05] [WebCite Cache]
  56. National Influenza Specialist Group. Fightflu. 2015. Can I Get a Free flu Vaccine?   URL: http://www.fightflu.co.nz/ [accessed 2016-01-05] [WebCite Cache]
  57. Gross PA, Hermogenes AW, Sacks HS, Lau J, Levandowski RA. The efficacy of influenza vaccine in elderly persons. A meta-analysis and review of the literature. Ann Intern Med 1995;123:518-527. [Medline]
  58. Nichol KL, Nordin J, Mullooly J, Lask R, Fillbrandt K, Iwane M. Influenza vaccination and reduction in hospitalizations for cardiac disease and stroke among the elderly. N Engl J Med 2003;348:1322-1332. [CrossRef] [Medline]
  59. Pharmaceutical Management Agency. PHARMAC. 2016. Online Pharmaceutical Schedule - Vaccinations   URL: http://www.pharmac.govt.nz/patients/PharmaceuticalSchedule/Schedule?osq=influenz [accessed 2016-01-05] [WebCite Cache]
  60. Baxter K, Preston CL, editors. Stockley's Drug Interactions. 10th edition. London: Pharmaceutical Press; 2013.
  61. Rohan J, Pramesh K. Australian Prescriber. 2002. Drugs and the QTc interval   URL: http://www.australianprescriber.com/magazine/25/3/63/5 [accessed 2016-05-12] [WebCite Cache]
  62. Fazio G, Vernuccio F, Grutta G, Re GL. Drugs to be avoided in patients with long QT syndrome: Focus on the anaesthesiological management. World J Cardiol 2013;5:87-93 [FREE Full text] [CrossRef] [Medline]
  63. Medsafe Pharmacovigilance Team. Prescriber Update Publications. 2010. Drug-Induced QT Prolongation and Torsades de Pointes - The Facts   URL: http://www.medsafe.govt.nz/profs/PUArticles/DrugInducedQTProlongation.htm [accessed 2016-01-05] [WebCite Cache]
  64. Medsafe Pharmacovigilance Team. Prescriber Update Publications. 2013. Update: QT Prolongation With Antidepressants   URL: http://www.medsafe.govt.nz/profs/PUArticles/December2013QTProlongationAndAntidepressants.htm [accessed 2016-01-05] [WebCite Cache]
  65. Garfinkel D, Zur-Gil S, Ben-Israel J. The war against polypharmacy: a new cost-effective geriatric-palliative approach for improving drug therapy in disabled elderly people. Isr Med Assoc J 2007;9:430-434 [FREE Full text] [Medline]
  66. BPAC. Best Tests. 2011. A Primary Care Approach to Sodium and Potassium Imbalance   URL: http://www.bpac.org.nz/BT/2011/September/imbalance.aspx [accessed 2016-05-12] [WebCite Cache]
  67. BPAC. Best Practice Journal. 2014. Bipolar Disorder: Identifying and Supporting Patients in Primary Care   URL: http://www.bpac.org.nz/BPJ/2014/July/bipolar.aspx [accessed 2016-05-12] [WebCite Cache]
  68. Malhi GS, Tanious M, Bargh D, Das P, Berk M. Australian Prescriber. 2013. Safe and Effective Use of Lithium   URL: http://www.australianprescriber.com/magazine/36/1/18/21 [accessed 2016-05-12] [WebCite Cache]
  69. Hunt SA, American College of Cardiology, American Heart Association Task Force on Practice Guidelines (Writing Committee to Update the 2001 Guidelines for the EvaluationManagement of Heart Failure). ACC/AHA 2005 guideline update for the diagnosis and management of chronic heart failure in the adult: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Writing Committee to Update the 2001 Guidelines for the Evaluation and Management of Heart Failure). J Am Coll Cardiol 2005;46:e1-e82 [FREE Full text] [CrossRef] [Medline]
  70. Douglas Pharmaceuticals Ltd. Medsafe. 2014. Lithium Carbonate 250 mg Capsule Data Sheet   URL: http://www.medsafe.govt.nz/profs/datasheet/l/Lithiumcarbonatecap.pdf [accessed 2016-01-05] [WebCite Cache]
  71. Waitemata District Health Board (WDHB). SafeRx. 2014. Atypical Antipsychotics - Safe Prescribing - Better, but not Perfect   URL: http://www.saferx.co.nz/full/atypical_antipsychotics.pdf [accessed 2016-01-05] [WebCite Cache]
  72. Roughead EE, Barratt JD, Ramsay E, Pratt N, Ryan P, Peck R, et al. The effectiveness of collaborative medicine reviews in delaying time to next hospitalization for patients with heart failure in the practice setting: results of a cohort study. Circ Heart Fail 2009;2:424-428 [FREE Full text] [CrossRef] [Medline]
  73. Roughead EE, Barratt JD, Ramsay E, Pratt N, Ryan P, Peck R, et al. Collaborative home medicines review delays time to next hospitalization for warfarin associated bleeding in Australian war veterans. J Clin Pharm Ther 2011;36:27-32. [CrossRef] [Medline]


API: application program interface
BPAC: Best Practice Advocacy Centre
EHR: electronic health record
MORE: medicines optimization/review and evaluation
PMS: patient management system
SNOMED CT: systematized nomenclature of medicine clinical terminology


Edited by G Eysenbach; submitted 19.01.16; peer-reviewed by M Numans, A Billis, T Irizarry; comments to author 18.02.16; revised version received 15.03.16; accepted 30.03.16; published 10.06.16

Copyright

©Amber Young, June Tordoff, Susan Dovey, David Reith, Hywel Lloyd, Murray Tilyard, Alesha Smith. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 10.06.2016.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.