This paper is in the following e-collection/theme issue:

RCTs - Protocols/Proposals (funded, already peer-reviewed, eHealth) (353) Clinical Information and Decision Making (1407) Adverse Drug Events Detection, Pharmacovigilance and Surveillance (156) Electronic Health Records (1053)

Published on in Vol 11, No 11 (2022): November

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/40456, first published .
Automatic Detection of Adverse Drug Events in Geriatric Care: Study Proposal

Automatic Detection of Adverse Drug Events in Geriatric Care: Study Proposal

Automatic Detection of Adverse Drug Events in Geriatric Care: Study Proposal

Authors of this article:

Frederic Gaspar1, 2, 3 Author Orcid Image ;   Monika Lutters4 Author Orcid Image ;   Patrick Emanuel Beeler5 Author Orcid Image ;   Pierre Olivier Lang6 Author Orcid Image ;   Bernard Burnand7 Author Orcid Image ;   Fabio Rinaldi8, 9, 10, 11 Author Orcid Image ;   Christian Lovis12 Author Orcid Image ;   Chantal Csajka1, 2, 3 Author Orcid Image ;   Marie-Annick Le Pogam7 Author Orcid Image ;   SwissMADE study13
Article Authors Cited by (3) Tweetations (4) Metrics

    Proposal

    1Center for Research and Innovation in Clinical Pharmaceutical Sciences, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland

    2School of Pharmaceutical Sciences, University of Geneva, Geneva, Switzerland

    3Institute of Pharmaceutical Sciences of Western Switzerland, University of Geneva, University of Lausanne, Geneva and Lausanne, Switzerland

    4Service of Clinical Pharmacy, Baden University Hospital, Baden, Switzerland

    5Division of Occupational and Environmental Medicine, Epidemiology, Biostatistics and Prevention Institute, University of Zurich and University Hospital Zurich, Zurich, Switzerland

    6Clinique de Montchoisi, Lausanne, Switzerland

    7Unisanté Center for Primary Care and Public Health, Department of Epidemiology and Health Systems, University of Lausanne, Lausanne, Switzerland

    8Dalle Molle Institute for Artificial Intelligence Research, Scuola Universitaria Professionale della Svizzera Italiana, Universita della Svizzera Italiana, Lugano, Switzerland

    9Department of Quantitative Biomedicine, University of Zurich, Zurich, Switzerland

    10Swiss Institute of Bioinformatics, Lausanne, Switzerland

    11Fondazione Bruno Kessler, Trento, Italy

    12Division of Medical Information Sciences, Geneva University Hospitals and University of Geneva, Geneva, Switzerland

    13See Acknowledgments

    *these authors contributed equally

    Corresponding Author:

    Chantal Csajka, PharmD, PhD

    Center for Research and Innovation in Clinical Pharmaceutical Sciences

    Lausanne University Hospital and University of Lausanne

    Rue du bugnon 19

    Lausanne, 1011

    Switzerland

    Phone: 41 21 314 28 55

    Email: chantal.csajka@chuv.ch


    Background: One-third of older inpatients experience adverse drug events (ADEs), which increase their mortality, morbidity, and health care use and costs. In particular, antithrombotic drugs are among the most at-risk medications for this population. Reporting systems have been implemented at the national, regional, and provider levels to monitor ADEs and design prevention strategies. Owing to their well-known limitations, automated detection technologies based on electronic medical records (EMRs) are being developed to routinely detect or predict ADEs.

    Objective: This study aims to develop and validate an automated detection tool for monitoring antithrombotic-related ADEs using EMRs from 4 large Swiss hospitals. We aim to assess cumulative incidences of hemorrhages and thromboses in older inpatients associated with the prescription of antithrombotic drugs, identify triggering factors, and propose improvements for clinical practice.

    Methods: This project is a multicenter, cross-sectional study based on 2015 to 2016 EMR data from 4 large hospitals in Switzerland: Lausanne, Geneva, and Zürich university hospitals, and Baden Cantonal Hospital. We have included inpatients aged ≥65 years who stayed at 1 of the 4 hospitals during 2015 or 2016, received at least one antithrombotic drug during their stay, and signed or were not opposed to a general consent for participation in research. First, clinical experts selected a list of relevant antithrombotic drugs along with their side effects, risks, and confounding factors. Second, administrative, clinical, prescription, and laboratory data available in the form of free text and structured data were extracted from study participants’ EMRs. Third, several automated rule-based and machine learning–based algorithms are being developed, allowing for the identification of hemorrhage and thromboembolic events and their triggering factors from the extracted information. Finally, we plan to validate the developed detection tools (one per ADE type) through manual medical record review. Performance metrics for assessing internal validity will comprise the area under the receiver operating characteristic curve, F1-score, sensitivity, specificity, and positive and negative predictive values.

    Results: After accounting for the inclusion and exclusion criteria, we will include 34,522 residents aged ≥65 years. The data will be analyzed in 2022, and the research project will run until the end of 2022 to mid-2023.

    Conclusions: This project will allow for the introduction of measures to improve safety in prescribing antithrombotic drugs, which today remain among the drugs most involved in ADEs. The findings will be implemented in clinical practice using indicators of adverse events for risk management and training for health care professionals; the tools and methodologies developed will be disseminated for new research in this field. The increased performance of natural language processing as an important complement to structured data will bring existing tools to another level of efficiency in the detection of ADEs. Currently, such systems are unavailable in Switzerland.

    International Registered Report Identifier (IRRID): DERR1-10.2196/40456

    JMIR Res Protoc 2022;11(11):e40456

    doi:10.2196/40456

    Keywords

    adverse drug events; adverse drug reactions; older inpatients; aged 65 and older; multimorbidity; polypharmacy; patient safety; inappropriate prescribing; medication errors; natural language processing; clinical decision support system; automated adverse drug event reporting system; electronic medical record; hospitals; multicenter study; interdisciplinary research; quality of hospital care; machine learning; antithrombotics; venous thromboembolism; hemorrhage 


    Adverse Drug Events in Older Inpatients: A Significant Health Issue

    Patient injury resulting from medication use [1,2], also known as adverse drug events (ADEs), is the second most frequent complication experienced by hospitalized patients, accounting for approximately one-third (10%-40%) of all inpatient care-related adverse events [3,4]. ADEs include nonpreventable ADEs subsequent to appropriate care and preventable ADEs (pADEs) resulting from suboptimal care [5]. Between 0.2% and 65% of hospitalized patients experience at least one ADE during their stay [5-9]. This prevalence depends on the selected definition of ADEs, the methods used for their detection, the study size, various risk factors related to the clinical setting (eg, medical or surgical), patient characteristics (eg, age of ≥65 years and polypharmacy), and prescribed drugs [6,8,10,11]. Apart from increasing morbidity and mortality, ADEs have a significant impact on hospital use (ie, increased length of stay and readmissions) and associated costs [2,4,12-15]. Moreover, pADEs, which account for 20% to 50% of all ADEs, are often more serious and associated with increased lengths of stay and costs compared with non-pADEs [2,4,8,13,15].

    Older inpatients (aged ≥65 years) and, in particular, oldest-old inpatients (aged ≥80 years) are especially at risk of ADEs and pADEs. Over 30% of these patients experience at least one ADE during their hospital stay, and up to 70% of these events are deemed preventable [8,16-18]. In addition, ADEs have more severe consequences in this population, such as inducing or worsening frailty; causing functional and cognitive disability; and leading to loss of autonomy, more frequent and prolonged hospitalizations, nursing home admissions, and even death [16,17,19]. Finally, ADEs affect patients’ hospital care experience and quality of life [19,20]. Given that older patients are hospitals’ most frequent users [21,22], ADEs represent an important clinical and economic burden to this population and to health systems [23,24]. Thus, limiting ADEs has become a major patient safety and public health concern worldwide [23].

    Antithrombotic Therapy

    Cardiovascular drugs, in particular antithrombotic and antihypertensive drugs, are frequently associated with ADEs in older patients. Although recommended and widely used in older patients who are at increased risk of cardiovascular events, antiplatelet and anticoagulant treatments are highly associated with bleeding complications in this population and are a major cause of emergency department admissions and death [25-27]. Thus, antithrombotic therapy is similar to the sword of Damocles, conferring protection against thrombosis while exposing patients to bleeding, with severe consequences in both cases [27]. A recent study indicated that bleeding events were the most common ADE (36%) in patients aged >65 years [28]. The drugs most frequently involved in serious ADEs were antithrombotic agents (31%). Disregarding drug interactions, contraindications, and precautions caused 20% of ADEs, and drug overdoses were present in 17% [28]. In addition, combinations of factors and inefficacy raise particular concerns from an individual and public health point of view [28].

    Epidemiology and Risk Management of ADEs in Swiss Inpatients

    The Swiss health system is taking an increasing interest in medication safety issues. Few studies have been conducted in Switzerland to assess the incidence of ADEs during hospital stays and ADEs as the cause of hospital admissions [29-32]. A cohort study (Stiftung für Arzneimittelsicherheit or Comprehensive Hospital Drug Monitoring) conducted in the internal medicine departments of Zürich and St. Gallen university hospitals found incidences of ADEs and pADEs of 11.2% and 0.4%, respectively [30,31]. The causative drugs were antithrombotic and cardiovascular drugs, antibiotics, hypnotics, and nonsteroidal anti-inflammatory drugs. The observed pADE-related mortality was 3 deaths per 100,000 persons annually [30,31]. Another study conducted in 10 hospitals in the canton of Vaud estimated that between 10% and 17% of hospitalized patients were exposed annually to ADEs [32]. In this study, a patient safety improvement program was deployed over 18 months with the aim of reducing ADEs by 20%. After implementing the patient safety program, which entailed patient identification, high-alert medication, and medication preparation in the ward, the annual rate of harmed patients decreased to 7% within 2 years [32]. Another study conducted at Lausanne University Hospital (LUH) found that 7% of all emergency department admissions were caused by ADEs, of which 32% were classified as avoidable. The most frequent ADEs were gastrointestinal bleeding (22.3%) and febrile neutropenia (14.4%) [29]. Similar results were reported in a prospective analysis of reasons for hospital admission in the internal medicine department of the Bellinzona Regional Hospital. The authors estimated that 6.4% of patients admitted over 1 year presented with ADEs at admission and that most of them were potentially preventable. Cardiovascular and cerebrovascular drugs accounted for 65% of ADEs, and the risks of occurrence increased with age, polypharmacy, multimorbidity, and length of stay. The authors finally estimated that, in Switzerland, 12,000 to 16,000 admissions per year were caused by inappropriate or unnecessary treatment, with additional direct annual costs of CHF 70 to CHF 100 million (US $70.8-101.1 million) [33].

    ADE Detection Tools

    Worldwide, numerous optimization strategies have been adopted to improve the quality and safety of medications prescribed to older patients [4,23,34-37]. Among these are evidence-based specific guidelines [38]; lists of potentially inappropriate medication criteria [37,39]; pharmacist-based interventions, including patient counseling [35]; medication reconciliation; clinical pharmacist rounding; and team-based interventions such as multidisciplinary geriatric teams [35,36,40,41].

    The availability of clinical decision support systems within computer provider order entry systems has raised hopes and expectations to improve the safety and efficiency of care [42]. Clinical decision support systems comprise a wide range of functionalities, including medication dosing support, point-of-care alerts or reminders (eg, for drug-drug interactions), or workflow support for medication reconciliation [43-45]. Many studies have reported the positive impacts of such systems on patient outcomes, such as fewer duplicate orders, dosage errors, drug interactions, or delayed actions using reminders [41].

    Clinical event monitors are a type of clinical information system with considerable potential to contribute to the detection and monitoring of medication-related problems, in particular ADEs [44]. Such systems provide feedback to clinicians through alerts and reminders when certain signals regarding pharmacy orders (eg, sudden stop orders, antidote ordering, and dose correction orders), laboratory test results, or patient characteristics are triggered [42]. Classen et al [43] developed a computerized method for detecting ADEs that uses signals identified from several types of patient medical record data. This computerized monitor increased >60-fold the detection and reporting of ADEs in hospitalized patients. Owing to growing evidence regarding the benefits of such clinical event monitors, several prominent national organizations have recommended their use to detect ADEs. Compared with voluntary reporting or manual methods of chart reviews, electronic clinical event monitors are faster, less expensive, and often identify ADEs that are not normally detected during the course of routine hospital care [6,44]. However, current clinical event monitors generate many false-positive alerts, target rather inappropriate prescriptions instead of clinically relevant ADEs, and do not consider the type of hospital or unit (eg, medical or surgical) or the patients’ characteristics [42,46,47]. In Switzerland, health care facilities have progressively introduced clinical decision support. However, current systems are mostly decision support tools targeting drug dosage or drug-drug interaction or incompatibilities or supporting information from the hospital drug formulary [40,46].

    Owing to the poor specificity and overalerting of existing ADE detection tools and the availability of large amounts of structured and unstructured information contained in computer-based patient records, new ADE detection and monitoring systems are currently being developed [47-51]. They are based on multiple sources of data and rely on new methodologies for data processing combining structured data mining (SDM) and natural language processing (NLP). SDM is defined as the process of finding and extracting useful information from semistructured data, whereas NLP is a domain at the crossroads of computer science and linguistics that aims at modeling language to extract meaningful information from free text. Typically, structured data include drug names, doses, treatment durations, administration routes, laboratory results, and diagnostic or procedure codes, whereas reasons for admission, patient histories and conditions, nursing and medical progress notes, inpatient reports, and discharge summaries are essentially available in the form of free text. The use of NLP improves the ability to detect ADEs because the available data sources are often in free text. As a result, structured data analytics have the advantage of being language-independent, though limited by poor specificity and overalerting, whereas free-text analytics have strong power to support ADE detection while strongly depending on language [51]. Tools have already been developed for the English language but are not directly applicable to other languages such as French, German, or Italian, which are 3 of the 4 official languages in Switzerland. As a result, no ADE detection and monitoring system based on electronic medical record (EMR) SDM and NLP is currently available in Switzerland.

    Aim and Research Questions

    This Swiss national initiative aims to develop and validate a multimodal, multisource, and multicentric approach for the automated detection of antithrombotic-related ADEs and their risk management in older inpatients. Our hypothesis is that the automated detection of ADEs from EMRs using SDM and NLP could significantly improve risk management and patient safety in hospitalized older inpatients with multimorbidity, frailty, and polypharmacy. This will provide reliable data regarding the incidence of ADEs for health care professionals, patient safety organizations, and policy makers. Thus, the project will comprise complementary steps aimed at (1) quantifying the cumulative incidence of ADEs associated with and caused by antithrombotic drugs; (2) assessing the causality, severity, and preventability of detected ADEs induced by antithrombotic drugs; and (3) developing strategies for the implementation of the project results to improve the risk management of antithrombotic drugs in the hospital setting.


    Overall Design

    We will conduct a multicenter cross-sectional study using retrospective medical data (years 2015 and 2016) from the EMRs of 4 large Swiss hospitals. Our project will include 2 hospitals in the French-speaking part of Switzerland (Lausanne and Geneva; LUH and Geneva University Hospital [GUH]) and 2 hospitals in the German-speaking part (Zürich and Baden; Zürich University Hospital [USZ] and Baden Cantonal Hospital [KSB]). Three are large university hospitals (GUH, LUH, and USZ), and one is a smaller cantonal hospital (KSB).

    Study Participants

    Study participants will consist of all Swiss residents aged ≥65 years who were admitted for >24 hours to 1 of the 4 hospitals between 2015 and 2016 (ie, the inclusion period) and received at least one antithrombotic drug during their stay. We will exclude any patients for whom an explicit refusal to be involved in research projects or give access to their personal health data is documented.

    Source Data

    This project will use health-related information that is routinely collected during daily practice. Relevant health-related data, as defined in this section, will be extracted by each participating hospital for all patient stays fulfilling the inclusion criteria.

    Health-related data include the following: (1) structured data, which encompass each included stay; patient administrative data (eg, age, gender, place of residence, and admission and discharge mode and date); admission unit, diagnosis, and procedure codes (International Classification of Diseases, 10th Revision [ICD-10], and Swiss classification of surgical interventions [CHOP] codes); drug administration orders (eg, drug names, administration dates and times, drug dosages, frequencies, durations, administration routes, and Anatomical Therapeutic Chemical codes); laboratory test orders (ie, test names, time stamps, and samples); laboratory test results (ie, test names, typical ranges, units, results, and time stamps); and imaging and endoscopic test orders (ie, test names and time stamps); and (2) free-text data, including discharge letters, medical and nursing progress notes, pharmacological consultation notes, ADE reports, and all existing local metadata. The extracted variables are presented in Table 1 for structured data. The documents that will be extracted as free text are presented in Textbox 1. Details of the extracted items are presented in Tables S1-S4 in Multimedia Appendix 1.

    Table 1. Structured data extracted for the project.
    Data type, extracted data, and subcategoryUnit
    General administrative data

    		Patient identification number (coded)Category

    		Case identification number (admission ID, hospitalization ID, or stay ID)Category

    		Insurance typeCategory

    		Region of residence (MedStata region)Category

    		Admission mode (eg, admission via emergency department, planned admission, or transfer)Category

    		NationalityCategory

    		Date of birth (coded)Date

    		GenderCategory

    		Date of death (if applicable)Date
    Clinical measurements

    		Blood pressureValue

    		WeightValue

    		HeightValue

    		Sum of alcohol withdrawal syndrome scoreValue
    Patient location or locations and transfers

    		Unit of hospitalizationCategory

    		Transfers (medicine, surgery, intermediate care, and intensive care)Category

    		Date and time of admissionDate and time

    		Date and time of dischargeDate and time
    Diagnoses and procedures

    		DRGb codesCategory

    		CHOPc codesCategory

    		ICD-10d codesCategory

    		Readmissions and reasons for readmissions (first, second, third, fourth, and subsequent readmissions)Category

    		Drugs coded for reimbursementCategory

    		Intensive care unit length of stay (in hours)Category

    		Duration of mechanical ventilation (in hours)Category

    		Disease severity and scoresCategory

    		NEMSeCategory
    Laboratory valuesf

    		Electrolytes and ions


    		Blood ionogram (sodium and potassium)mmol/L


    		Serum lactate and bicarbonate levelsmmol/L


    		Serum uric acid levelmmol/L


    		Serum urea levelmmol/L


    		Serum iron, transferrin saturation, and serum ferritin levelmmol/L and µg/L

    		Enzymes


    		Serum levels of aminotransferasesASTg and ALTh (UIi/L)


    		Serum levels of 5′-nucleotidaseUI/L


    		Serum level of CKjUI/L


    		Serum level of GGTkUI/L


    		Serum level of ALPlUI/L

    		CBCm


    		Red blood cell countAbsolute value/mm3


    		Hemoglobinmmol/L


    		HematocritPercentage of total blood volume


    		Mean corpuscular volumeµ3


    		White blood cell countAbsolute value/mm3


    		Platelet countAbsolute value/mm3


    		Reticulocyte countAbsolute value/mm3

    		Hemostasis


    		PTnTime


    		APTToTime


    		TTpTime


    		INRqN/Ar


    		Plasma fibrinogeng/L


    		Procoagulant balanceAntithrombin (g/L), protein C and S (nmol/L), anticardiolipin antibody (GPLs unit), and anti-beta-2-glycoprotein 1 antibody (GPL unit)


    		Individual coagulation factorsPercentage relative to a reference pool


    		FibrinolysisD-dimer (µg/L)


    		Anticoagulation monitoringAnti-Xa (percentage relative to a reference value) and anti-IIa (percentage relative to a reference value)


    		Markers of coagulationTATt (ng/mL) and fragment 1 and 2 of prothrombin (percentage relative to a reference value)

    		Other


    		Serum albuming/dL


    		Serum total proteing/dL


    		Oxygen saturation in arterial bloodPercentage


    		CRPumg/L


    		Serum myoglobinµg/L


    		Serum troponinµg/L


    		Serum creatinine and creatinine clearancemg/L and mL/minute


    		Serum total bilirubin and direct and indirect bilirubinmg/L


    		Serum glycated hemoglobinPercentage


    		Serum tumor markers availableN/A
    Prescription or medicationv

    		ATCw code (and product ID)Category

    		Information on dose and planned administration frequency, including unit (eg, milligrams)Value and category

    		Information on administration route (eg, intravenous administration vs oral)Category

    		PRNx orders (“as needed”: drugs available at patient’s request; eg, analgesics)Category

    		Administrations performed (signed by nurses)Category

    aMedStat: MedStat regions are geographic areas with a sufficiently large population to anonymously assign a residence to each person hospitalized in Switzerland.

    bDRG: Swiss Diagnosis Related Groups.

    cCHOP: Swiss classification of surgical interventions.

    dICD-10: International Classification of Diseases, 10th Revision.

    eNEMS: Nine Equivalents of Nursing Manpower Use Score.

    fLaboratory results that were ordered or received within the time frame of any of the recorded or extracted stays (if available).

    gAST: Aspartate aminotransferase.

    hALT: Alanine transaminase.

    iUI: international unit.

    jCK: creatine kinase.

    kGGT: gamma-glutamyltransferase.

    lALP: alkaline phosphatase.

    mCBC: complete blood count.

    nPT: prothrombin time.

    oAPTT: activated partial thromboplastin time.

    pTT: thrombin time.

    qINR: international normalized ratio.

    rN/A: not applicable.

    sGPL: immunoglobulin G [IgG] phospholipid unit.

    tTAT: thrombin-antithrombin III complex.

    uCRP: C-reactive protein.

    vAll medication orders that have (1) a planned start date ≤discharge date AND (2) a planned discontinuation date ≥admission date.

    wATC: Anatomical Therapeutic Chemical.

    xPRN: Pro re nata.

    Free texts and narratives extracted for the project.

    Extracted free-text data

    • Patient identification number (metadata)
    • Case identification number (admission ID, hospitalization ID, or stay ID, metadata)
    • Notes taken at admission
    • Discharge summaries and letters
    • Nurses’ progress notes
    • Imaging and radiology reports
    • Specialists’ (eg, hematologist, cardiologist, angiologist, and particularly endoscopy reports) consultation notes
    • Clinical pharmacology or pharmacy service consultation notes
    • Adverse drug event or pharmacovigilance reports
    • Critical incidence reporting system reports
    Textbox 1. Free texts and narratives extracted for the project.

    Data Analysis

    The overall strategy includes the development of automated detection tools that will provide quantitative measures and triggers of ADEs associated with antithrombotic drugs and the elaboration of measures for implementation, replication, and prevention (Figure 1). A total of 5 work packages (WPs) will provide answers to these objectives.

    ‎
    Figure 1. Schematic framework of the SwissMADE project. ATC: Anatomical Therapeutic Chemical; INN: international nonproprietary name; work package (WP) 1: drug selection and adverse drug event (ADE) marker and trigger definition and selection; WP 2: data extraction and management; WP 3: algorithm generation based on structured data; WP 4: resource and algorithm generation for free-text data; WP 5: ADE detection tool assessment.
    View this figure

    Drug Selection and ADE Marker or Trigger Definition and Selection From the Literature (WP 1)

    Five different classes of antithrombotic drugs (approved and licensed in Switzerland) will be considered: heparins (eg, unfractionated heparins and low–molecular-weight heparins), vitamin K antagonists (eg, coumarin derivatives), directly acting oral anticoagulants and fondaparinux (eg, direct thrombin and factor Xa inhibitors), and antiplatelet drugs. The list of selected drugs is provided in Table S4 in Multimedia Appendix 1. For each administration of an antithrombotic drug during each inpatient stay, we will retrieve the following information from the corresponding EMR: international drug name; active pharmaceutical ingredient; Anatomical Therapeutic Chemical code; approved therapeutic regimen (eg, dose, range, frequency, and duration); pharmacokinetics (eg, absorption, primary metabolism pathways, and elimination); and potential drug-drug or drug-disease interactions, indications, and contraindications.

    To identify thromboembolic and hemorrhagic ADEs caused by antithrombotics, we will first develop direct and indirect indicators of ADEs (Textbox 2). Potential clinical and biological antithrombotic-related ADEs will be assembled from clinical guidelines and standard pharmacological references, including Swiss medic product information, Lexicomp, Martindale, and Meyler’s Side Effects of Drugs encyclopedia [52]. We will focus on 3 types of ADEs in particular: hemorrhages and venous and arterial thromboembolisms, which will be defined according to international references [53-55].

    Similarly, potential confounding factors in the causal relationship between antithrombotic drugs and related ADEs will be identified from scientific literature. They will include concomitant drugs (eg, drugs that modulate up or down the coagulation process or affect the metabolism or elimination of anticoagulants), patient characteristics (eg, congenital deficit in coagulation factors), and concomitant health conditions (eg, cirrhosis and kidney failure).

    Finally, relevant ADEs and confounding factors will be selected using a modified Delphi method, which combines the summarized evidence with the collective judgment of an expert panel [56-58]. The panel will comprise geriatricians, pharmacologists, pharmacists, and internists from the research team. These experts will assess several statements on ADE characteristics during a 2-round process. These characteristics will comprise clinical significance, association with quality of care, and relevant confounding factors. The first round will consist of an individual remote rating with no interaction among the panelists. During the second round, the experts will meet and reach a consensus on ADE selection and relevant confounding factors. In each round, each panelist will receive an individualized document showing the distribution of all the experts’ ratings together with the panelist’s specific ratings.

    Direct and indirect adverse drug event (ADE) indicators.

    ADE indicators

    • ADE markers: clinical signs or symptoms, diagnostic or treatment procedures, prescription and imaging orders, and biological test results indicating that an antithrombotic-related ADE occurred (ie, event)
    • ADE triggers: diagnostic procedures, prescription and imaging orders, and biological test results that indicate that an antithrombotic-related ADE should have occurred but did not (ie, near miss)
    • ADE confounding conditions and risk factors: conditions or factors that increase the risk of antithrombotic-related ADEs or the occurrence of a spontaneous bleeding or thrombotic event; risk factors include patient characteristics, specific concomitant health conditions that may interact with antithrombotic drugs, and concurrent use of more than one antithrombotic drug
    Textbox 2. Direct and indirect adverse drug event (ADE) indicators.

    Data Extraction and Management (WP 2)

    Overview

    Structured data and free texts will be extracted from EMRs by the IT department of each hospital. Before being processed, structured data will be standardized in a unique common format (see the common data model in Tables S1-S4 in Multimedia Appendix 1), and unstructured data will be transformed into a machine-readable format when necessary.

    Owing to the multisite and bilingual (ie, French and German) nature of the project, we will use a centralized and decentralized data governance strategy. A 2-step approach will be undertaken for data processing.

    Step 1 (Decentralized Data Processing)

    Raw data from the study participants’ EMRs will be managed and processed by the IT team of each hospital according to established protocols. Nominative identifiers of the structured data will be coded, and a table of correspondence between the original and coded identifiers will be stored on a protected server of each IT team. There are 18 identifiers to delete as described in the Health Insurance Portability and Accountability Act Privacy Rule, Code of Federal Regulations [59] Title 45: Public Welfare, Subtitle A §164.514. Free-text data of the learning set will be deidentified and coded locally by an encryption software before being transferred to secured servers dedicated to the project within each hospital.

    Step 2 (Centralized Data Processing)

    Locally extracted coded data of the learning set from the German-speaking part of Switzerland (USZ and KSB) and the French-speaking part (LUH and GUH) will be transferred to a centralized common SwissMADE database (DB). After proof of deidentification, structured items from free texts and narratives will be transferred to the SwissMADE DB. One authorized person per site will be allowed access to personal data and will decide who can access which data linked to which analysis. Remote access to the local working DBs within each hospital and the SwissMADE DB to authorized investigators will be made possible through a virtual private network.

    Data Analysis (WP 3 and 4)

    Part of the structured and unstructured data will be used to develop ADE detection algorithms (working set), and another part will be used for validation (validation set). To test the algorithms’ accuracy, we will randomly select a validation data set to verify in the corresponding EMRs (gold standard) whether an ADE has truly occurred. As a result, the algorithms will be improved according to the results of the validation (ie, maximization of sensitivity and specificity as well as of positive predictive values [PPVs] and negative predictive values [NPVs]). Finally, validated algorithms will serve to identify ADEs accurately (ie, validated outcomes). For each 2015 and 2016 hospital stay of patients aged ≥65 years treated with antithrombotics, we will obtain a Case ID of positive ADE detected from the developed algorithms based on structured data (SDM), free-text data (NLP), and both types of data (SDM+NLP).

    Elaboration of Algorithms Based on Structured Data (WP 3)

    Computational algorithms based on logical rules applied to structured data will be developed to identify ADE markers, triggers, confounding conditions or risk factors, and causes. Detection algorithms for clinical markers of ADEs (ie, hemorrhagic events or thromboembolism) and confounding clinical conditions (eg, chronic liver or kidney disease, hypertension, diabetes, cancer, and multimorbidity) will target ICD-10-German Modification diagnostic codes in hospital discharge data [60-62]. Regarding ADE triggers, clinical conditions (eg, hypotension, shock, and acute kidney failure) and procedures (eg, postoperative control of hemorrhage, drainage of hematoma, or surgical treatment of venous or arterial thromboembolism) will be identified from hospital discharge data using algorithms based on ICD-10-German Modification diagnostic codes and CHOP codes, respectively. Biological triggers of ADEs (ie, abnormal laboratory values) will be detected by algorithms applied to laboratory test results. Similarly, some algorithms based on prescription orders will search for pharmacological triggers of ADEs, including sudden medication stop orders, antidote ordering, dose correction orders, underdosing and overdosing, misprescribing, insufficient monitoring, and drug-drug or drug-disease associations.

    In addition to the rule-based algorithms, data-driven algorithms will be created from the same data. Thus, we will perform predictive modeling of ADEs using penalized and nonpenalized logistic regression models with backward selection of predictors based on the Akaike information criterion and supervised and unsupervised machine learning approaches (eg, random forest, cluster analysis, and neural networks).

    Elaboration of Resources and Algorithms for Free-Text Data Processing (NLP; WP 4)

    Overview

    To develop models for classifying free-text discharge summaries as containing or not containing an ADE, multiple approaches will be considered. We divide these approaches into 2 categories: pure NLP approaches that require little to no medical or pharmaceutical expertise and hybrid approaches that are performed in close collaboration with medical and pharmaceutical experts. Both types of approaches require that we have access to a data set of discharge summaries that have been classified as containing or not containing an ADE. Pure NLP approaches include but are not limited to word and document embeddings, Bidirectional Encoder Representations from Transformers (BERT) for document classification, and other modern NLP methodologies. For hybrid approaches, we wish to build a named-entity recognition (NER) model that automatically annotates discharge summaries with various entities related to and relationships with ADEs. NER is a well-studied task in NLP in which a model learns to detect and label the mention of any predefined entity (ie, symptom, disease, and drug) in a span of unstructured text. Such a model will provide us with structured information from the free text that we could then exploit by designing rule-based algorithms or machine learning classification models using the entities as derived features. We note that more methods may be used to develop better classification models depending on the classification scores obtained using the aforementioned methods.

    Pure NLP Approaches

    These approaches require only a set of discharge summaries classified as containing or not containing an ADE. We can then, upon some preprocessing of the summaries, apply various NLP methods for classification. Popular methods include word and document embedding, BERT for document classification, and other modern approaches.

    Word embeddings (respectively document embeddings) are a class of functions that transform a high-dimensionality space of words (respectively documents) into information-rich vectors in a lower-dimension embedding space, which can be used for evaluating the word-to-word (respectively document-to-document) similarity or used as a feature for more complex models. Recently, there have been promising results for several document classification tasks such as patient classification [63] or legal document classifications [64]. BERT [65] is a language model published in 2019 that can be used for a variety of tasks such as translation [66], question answering [67], and document classification. Although the original BERT model is based on the English language, French and German variants have since been published [68-70].

    Hybrid Approaches

    In this approach, we wish to leverage medical and pharmaceutical expertise to complement NLP methods. This hybrid approach can be seen as approximating the behavior of an expert classifying a discharge summary as “containing” or “not containing” an ADE. As a first step, we build an NER model to highlight various entities and connections that are highly related to ADEs in the discharge summaries. We focus on the following entities: antithrombotic drugs, other drugs, dosages, risk factors, hemorrhagic events, and thrombotic events. We also identify the relationships between a drug and its corresponding dosage as well as hemorrhagic and thrombotic events that happened in the past and not during the hospital stay. Already, domain expertise is required to build dictionaries of drugs and risk factors for the events we wish to detect. These sets of concepts will be organized in a coherent and pertinent taxonomy validated by pharmacologists, pharmacists, and geriatricians from the research team. To build an NER model, we rely on data annotated with the various entities that we want to predict. To work with gold-standard annotated data, we follow a robust annotation protocol inspired by Gurulingappa et al [71], which guarantees consistency between different annotators. Indeed, highly technical annotations such as those we are performing in our context can lead to disagreements between annotators. We mitigate this risk through several rounds of protocol harmonization as well as annotation review—each letter is independently annotated by 2 annotators and then reviewed by a third annotator. To build the NER model, we can rely, for instance, on the 2018 National NLP Clinical Challenges shared task [72]. This shared task focused on the detection of ADEs and related entities from clinical records. It led to breakthrough results (best overall F1-score of 0.94 on similar entities), and we intend to replicate some of the methods on our French and German corpora. We will also leverage new language models that have arisen since, such as BERT (and, more specifically, its French and German counterparts). With a data set of discharge summaries that have been classified as “containing” or “not containing” an ADE, we can then use features extracted with our NER model to train a machine learning classification model. We can also design rule-based algorithms based on domain expertise.

    Eventually, NLP models for text annotation and ADE detection will be deployed as part of a pipeline that takes raw clinical text as input and outputs annotated text files with a list of detected ADEs, confounding conditions, and their probability scores.

    ADE Detection Tool Assessment (WP 5)

    To assess the performance (eg, sensitivity, specificity, PPVs, and NPVs) of the ADE detection tool, a validation will be performed on a random sample of 600 hospital stays. ADE occurrence, type, causality, severity, and preventability will be assessed by means of a patient medical record review and analyzed by a team of pharmacologists, pharmacists, and geriatricians from the research team.

    To ensure that the ADE assessment and data abstraction are structured and reliable, pharmacologists and pharmacists from the research team will develop a common ADE assessment form both in French and German based on existing good pharmacovigilance practice rules that will be disseminated in its original languages and English after study completion. Pairs of trained clinicians (eg, pharmacists, pharmacologists, and geriatricians) will then assess all selected medical records using this form. The causality between taking a drug and the advent of an ADE will be assessed using existing causality assessment scales [73,74]. The severity of the ADEs will be scored according to the Common Terminology Criteria for Adverse Events. Finally, an ADE will be deemed preventable if it was caused by a medication error that occurred during prescribing, transcribing, dispensing, administering, and monitoring or if it was due to a lack of medication adherence [75].

    To test the reliability of the ADE assessment by pharmacists and pharmacologists, we will calculate intra- and interrater agreements for overall ADE occurrence, causality, severity, and preventability. For each pair of trained pharmacists and pharmacologists, the interrater agreement will be tested by comparing the results of the ADE assessment between members of the pair. To test for intrarater agreement in each participating hospital, a random sample including 100 ADEs detected by SDM and NLP will be reassessed by the assigned pair 3 months after their first assessment. For both intra- and interrater agreements, the measure of agreement will be the Cohen or uniform κ statistic [76]. After excluding false-positive ADEs, we will measure the cumulative incidence of true positive pADEs and nonpreventable ADEs for each hospital and medical unit as well as overall.

    Ethics Approval

    The research project was approved by all cantonal ethics commissions involved in the project: Commission cantonale d'Ethique de la Recherche sur l'être humain Vaud" (CER-VD), Ethikkommission Nordwest- un Zentralscheiz (EKNZ), Commission cantonale d'éthique de la recherche Genève (CCER) and Kantonale Ethikkommission Zürich, under approval CER 2018-00272. Obtaining ethical permission is necessary for every study involving human participants. A method in which study participants can be satisfied that potential hazards have been evaluated, minimized, and declared acceptable is through the ethical review process.


    Health Data and Study Populations

    After accounting for the inclusion and exclusion criteria, we will include 34,522 residents aged ≥65 years in our study (n=5888, 17.06% from LUH; n=11,581, 33.55% from GUH; n=8986, 26.03% from KSB; and n=8067, 23.37% from USZ).

    Sample Size Calculation for Geriatric Patient Safety Indicators Criterion Validity Assessment

    The sample size was estimated to assess the performance of the SDM+NLP tool in detecting hemorrhagic adverse events. Indeed, the SDM+NLP tool is considered the critical outcome determining project feasibility, and hemorrhage is considered the most important adverse event related to antithrombotic drugs. We used a test result–based sampling method to minimize the number of medical records to be abstracted [77]. Given that CI is the cumulative incidence of ADEs detected from both structured and unstructured data, N is the number of 2016 hospital stays of patients at risk (ie, patients aged ≥65 years treated with antithrombotic drugs), p(ADE+) is the proportion of hospital stays with an ADE detected by the SDM+NLP tool among all at-risk hospital stays (calculated as the number of true positive and false-positive ADEs detected divided by N), Se is the expected sensitivity of the SDM+NLP tool, Sp is the expected specificity of the SDM+NLP tool, PPV, and NPV. We calculated the sample sizes for CI ranging from 3% to 24%, a desired Se of 80%, a 20% width for the 95% CI of Se, volumes of at-risk hospital stays ranging from N=2000 to N=20,000, and a balanced sample of ADE+ and ADE−hospital stays (ie, hospital stays with and without ADEs, respectively; Textbox 2). CI values were obtained from the literature (ie, a range of 30%-40% for ADE cumulative incidence and a range of 10%-40% for proportion of hemorrhagic ADEs among ADEs). The values of N were estimated from annual numbers of at-risk stays in the 4 participating hospitals. In particular, we considered selecting hospital units with a high prevalence of antithrombotic prescriptions (ie, acute geriatric unit, internal medicine, cardiology, angiology, orthopedic surgery, thoracic surgery, and cardiovascular surgery), which would increase the number of at-risk stays. The frequencies of 2015 at-risk stays in these selected units were 4711, 5130, 5564, and 5016 for LUH, GUH, USZ, and KSB, respectively (N=20,421). Therefore, we made the assumption that the number of at-risk stays for the 2015 to 2016 period would approximate 40,000. The sample size calculation was performed using Stata IC (version 14; StataCorp LLC). Thus, assuming CI equals 10% and p(ADE+) equals 12%, with 40,000 at-risk hospital stays and an expected Se equal to 80% with a 20% width for its 95% CI, a random sample of at least 523 medical records (51 ADE+ medical records and 472 ADE–medical records) will be necessary to assess the SDM+NLP tool. Considering that some medical records might not be available (perhaps 1%), the validation will finally require 530 medical records. Thus, we will abstract 15 medical records flagged as ADE+ and 120 medical records flagged as ADE–by the SDM+NLP tool during the 2015 to 2016 period in each of the 4 participating hospitals (135 medical records per hospital). Under these assumptions, the expected values and CIs for Se, Sp, PPV, and NPV should be as follows: Se=80% (95% CI 68%-88%), Sp=96% (95% CI 94%-97%), PPV=67% (95% CI 52%-79%), and NPV=98% (95% CI 96%-99%).

    Regarding the external validation of predictive models, we will compare predicted outcomes with true outcomes based on medical record screenings for a small sample of 20 to 30 hospital stays for which predicted and validated outcomes diverge.

    Project Timetable

    All structured and free-text data are now available to all research teams. However, the planned time frame for data extraction and the start of analyses was delayed by almost 1 year because of difficulties in obtaining approval from the 4 ethics committees. We also underestimated the challenges and time required to extract data from the 4 participating hospitals. Hospital information systems are not interoperable, so we had to provide additional resources to meet this challenge during 2021 and 2022. In addition, the barriers to data transfer or sharing from these hospitals despite ethics approval were highly unexpected. The data are being analyzed in 2022. We have completed ADE identification rules based on coded data (ICD-10 and CHOP codes) for all ADEs. Similar procedures will be applied to quantify the frequency of other ADEs related to antithrombotics and evaluate whether additional sources of information (laboratory values, drug prescriptions, and free text) will improve the detection algorithms. Tools for automatic annotation of free text (eg, exit letters) were developed in late 2021 and mid-2022 for the automated annotation of free texts (ie, discharge letters). They focus on drugs and symptoms for the German-language–based pipeline and on drugs, events, and risk factors for the French-language–based pipeline. These tools are in the validation phase in mid-2022, consisting of the “manual” revision and annotation of 600 (ZH) and 300 (CHUV) documents. Several machine learning models were trained and tested on a data set of 334 documents. Dictionaries specific to the medical languages encountered in these documents have been developed to explore the texts using both rule-based and machine learning algorithms. The research project will run until the end of 2022 to mid-2023.


    Overview

    This interdisciplinary and integrative project involves 4 hospitals and experts with a background in different disciplines. Apart from the core competencies in clinical research and pharmacology, extracting meaningful information from electronic health records requires competencies in data analysis and NLP. First, we ensured data availability at each participating hospital. Moreover, summary statistics provided by each hospital show a good representativeness of the target population and of the medical or surgical units. The choice of antithrombotic drugs in the geriatric population should guarantee sufficient data to build the SDM and NLP tools. Although some feasibility risk exists with the NLP part of the project (ie, suboptimal specificity and sensitivity), the information gathered during the NLP process and all other project components (pharmacology and SDM) will provide new and relevant data on the safety of antithrombotics in the geriatric population.

    This is the first study aiming at developing and validating a reliable automated tool to detect ADEs in older hospitalized patients using different data sources in Switzerland. Currently, only structured data are considered of value for the computerized patient record. However, the use of natural interfaces such as language analytics, which is being developed for the English language, has yet to expand to the German and French landscapes of health care systems. Assessing the quality and safety of antithrombotic therapy in older inpatients based on such data is innovative. This project will be able to leverage the importance of expressiveness in clinical free text and narratives by demonstrating that an analytical approach is applicable to such sources, thus fostering the possibility of using them for numerous other purposes such as within the Swiss Personalized Health Initiative.

    The greater understanding of the development of ADEs as a result of antithrombotic therapies, including the identification of important contributing factors, the early recognition of repeating events, and the setting up of preventative measures for sustained risk reduction, will have a major impact on the increasing population of older patients in hospitals and at particular risk of toxicity. Although antithrombotic therapy is highly recommended and commonly prescribed in the older adult population [27], antiplatelet and anticoagulant treatments have been shown to be highly associated with bleeding complications [25-27], which are a major cause of emergency department admissions and mortality in this population [27,78]. Vulnerable older inpatients experience more ADEs compared with younger adults. These adverse events result in considerable morbidity and mortality and frequent institutionalization. They also considerably affect patients’ quality of life and their confidence in the health care system and health care professionals. Some ADEs cannot be prevented, but most are associated with poor patient safety and quality of care [2,16,23,79,80]. Moreover, older patients are underrepresented in trials, and accurate information regarding the benefit-risk balance of most drugs in this population is limited [22,23,27,29,35,36]. Classen et al [6] wrote that “identification and measurement of adverse medical events is central to patient safety, forming a foundation for accountability, prioritizing problems to work on, generating ideas for safer care, and testing which interventions work.” We also claim that detecting and monitoring ADEs in older inpatients, in particular pADEs, is the most important step to reducing age-related disparities in patient safety and, therefore, health inequities.

    The technical and methodological aspects of the project (with multimodal, multisource, and multicentric data management) are the third strong point of the study. Owing to the multicomponent and multisite nature of the project, all organizational aspects to ensure effective scientific interactions between disciplines and hospitals throughout the research process will be disseminated for the Swiss research community. The algorithms used for SDM and NLP with all key information on data extraction and data mining will be made available (as open source) for further developments of medical NLP tools. Logical rules developed for SDM and the text-mining pipeline are electronic applications that can be directly implemented in hospital information systems. However, adapting automated detection tools to various hospital information systems and ADEs has proven to be difficult, but our ambitious and stepwise project will benefit from an interdisciplinary and experienced research team. We will focus on the 5 most clinically significant antithrombotic drug classes and plan to extend this project to other drug classes in the future. This tool could be implemented within EMRs and completed by e-alerts and reminders notifying providers of probable ADEs. It may also feature an automated causality assessment facilitating pharmacovigilance reports. This automated version of the Global Trigger Tool may help improve adverse drug effects and reaction reporting to local pharmacovigilance centers—and, therefore, to the Swiss authority for therapeutic products—while consuming fewer resources and being faster than the manual version currently in use.

    Finally, our project is consistent with the Swiss Federal Council “Health 2030” agenda that advocates for “better data” to inform health policy and patients’ choice and improve quality [81], safety, and efficiency in vulnerable patients, particularly in older inpatients at high risk of developing ADEs. Although ADE reporting by health care professionals has regularly improved between 2002 and 2019, Swissmedic suspects considerable underreporting. Frequent and common adverse drug reactions (ADRs), even severe ones, are less subject to professionals’ focus than rare and new events [82]. Therefore, ADRs only partially reflect the real ADE prevalence. Presuming that the nationwide pharmacovigilance DB collects ADRs that are subject to selection and underreporting and given the lack of systematic ADE reporting and monitoring in Swiss hospitals, new strategies encompassing more comprehensive and systematic detection of ADEs are needed.

    Conclusions and Perspectives

    This innovative study aiming to develop and validate an electronic application for the automated detection of ADEs related to antithrombotics will allow for the introduction of measures aimed at improving safety when prescribing antithrombotic medication. The increased performance of NLP as an important complement to structured data will bring existing tools to another level of efficiency in the detection of ADEs. Currently, such systems are not available in Switzerland, nor can “ready-made” systems from other countries be adapted as they are language-dependent. We hope that, in the near future, with these new types of tools developed for the French and German languages, the specificity of alerts will be improved, notifications will be prioritized, and clinical decision support will become more patient-centered.

    Acknowledgments

    The contributors associated with SwissMADE study are as follows: Angela Lisibach, Service of Clinical Pharmacy, Baden University Hospital, Baden, Switzerland; Arnaud Robert, Division of Medical Information Sciences, Geneva University Hospitals and University of Geneva, Geneva, Switzerland; Claire Coumau, Center for Research and Innovation in Clinical Pharmaceutical Sciences, Lausanne University Hospital and University of Lausanne, Lausanne, Switzerland; Christophe Gaudet-Blavignac, Division of Medical Information Sciences, Geneva University Hospitals and University of Geneva, Geneva, Switzerland; Denis Dériaz, Unisanté Center for Primary Care and Public Health, Department of Epidemiology and Health Systems, University of Lausanne, Lausanne, Switzerland; Elliott Bertrand, Effixis SA, Lausanne, Switzerland; Jean-Philippe Goldman, Division of Medical Information Sciences, Geneva University Hospitals and University of Geneva, Geneva, Switzerland; Jérôme Pasquier, Unisanté Center for Primary Care and Public Health, Department of Epidemiology and Health Systems, University of Lausanne, Lausanne, Switzerland; Nicola Colic, Institute of Computational Linguistics, University of Zurich, Zurich, Switzerland; Tapio Niemi, Unisanté Center for Primary Care and Public Health, Department of Epidemiology and Health Systems, University of Lausanne, Lausanne, Switzerland; Vasiliki Foufi, Division of Medical Information Sciences, Geneva University Hospitals and University of Geneva, Geneva, Switzerland.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Supplementary tables.

    DOCX File , 118 KB
    ‎
    Multimedia Appendix 2

    Evaluation of the protocol by the Swiss National Science Foundation.

    PDF File (Adobe PDF File), 92 KB
    ‎
    Multimedia Appendix 3

    Decision on the protocol submitted to the Swiss National Research Fund.

    PDF File (Adobe PDF File), 125 KB
    1. Nebeker JR, Barach P, Samore MH. Clarifying adverse drug events: a clinician's guide to terminology, documentation, and reporting. Ann Intern Med 2004 May 18;140(10):795-801. [CrossRef] [Medline]
    2. Krähenbühl-Melcher A, Schlienger R, Lampert M, Haschke M, Drewe J, Krähenbühl S. Drug-related problems in hospitals: a review of the recent literature. Drug Saf 2007;30(5):379-407. [CrossRef] [Medline]
    3. de Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA. The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care 2008 Jun;17(3):216-223 [FREE Full text] [CrossRef] [Medline]
    4. National action plan for adverse drug event prevention. US Department of Health Human Services. Washington, DC, USA: US Department of Health Human Services; 2014.   URL: https:/​/health.​gov/​our-work/​national-health-initiatives/​health-care -quality/​adverse-drug-events/​national-ade-action-plan [accessed 2021-09-01]
    5. Klopotowska JE, Wierenga PC, Smorenburg SM, Stuijt CC, Arisz L, Kuks PF, WINGS study group. Recognition of adverse drug events in older hospitalized medical patients. Eur J Clin Pharmacol 2013 Jan;69(1):75-85 [FREE Full text] [CrossRef] [Medline]
    6. Classen DC, Resar R, Griffin F, Federico F, Frankel T, Kimmel N, et al. 'Global trigger tool' shows that adverse events in hospitals may be ten times greater than previously measured. Health Aff (Millwood) 2011 Apr;30(4):581-589. [CrossRef] [Medline]
    7. Martins AC, Giordani F, Rozenfeld S. Adverse drug events among adult inpatients: a meta-analysis of observational studies. J Clin Pharm Ther 2014 Dec;39(6):609-620. [CrossRef] [Medline]
    8. Boeker EB, Ram K, Klopotowska JE, de Boer M, Creus MT, de Andrés AL, et al. An individual patient data meta-analysis on factors associated with adverse drug events in surgical and non-surgical inpatients. Br J Clin Pharmacol 2015 Apr;79(4):548-557 [FREE Full text] [CrossRef] [Medline]
    9. Bouvy JC, De Bruin ML, Koopmanschap MA. Epidemiology of adverse drug reactions in Europe: a review of recent observational studies. Drug Saf 2015 May;38(5):437-453 [FREE Full text] [CrossRef] [Medline]
    10. Alhawassi TM, Krass I, Bajorek BV, Pont LG. A systematic review of the prevalence and risk factors for adverse drug reactions in the elderly in the acute care setting. Clin Interv Aging 2014 Dec 1;9:2079-2086 [FREE Full text] [CrossRef] [Medline]
    11. Kaufmann CP, Stämpfli D, Hersberger KE, Lampert ML. Determination of risk factors for drug-related problems: a multidisciplinary triangulation process. BMJ Open 2015 Mar 20;5(3):e006376 [FREE Full text] [CrossRef] [Medline]
    12. Classen DC, Pestotnik SL, Evans RS, Lloyd JF, Burke JP. Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality. JAMA 1997 Jan 22;277(4):301-306. [Medline]
    13. Hug BL, Keohane C, Seger DL, Yoon C, Bates DW. The costs of adverse drug events in community hospitals. Jt Comm J Qual Patient Saf 2012 Mar;38(3):120-126. [CrossRef] [Medline]
    14. Rottenkolber D, Hasford J, Stausberg J. Costs of adverse drug events in German hospitals--a microcosting study. Value Health 2012;15(6):868-875 [FREE Full text] [CrossRef] [Medline]
    15. Formica D, Sultana J, Cutroneo PM, Lucchesi S, Angelica R, Crisafulli S, et al. The economic burden of preventable adverse drug reactions: a systematic review of observational studies. Expert Opin Drug Saf 2018 Jul;17(7):681-695. [CrossRef] [Medline]
    16. Klopotowska JE, Wierenga PC, Stuijt CC, Arisz L, Dijkgraaf MG, Kuks PF, WINGS Study Group. Adverse drug events in older hospitalized patients: results and reliability of a comprehensive and structured identification strategy. PLoS One 2013 Aug 5;8(8):e71045 [FREE Full text] [CrossRef] [Medline]
    17. Long SJ, Brown KF, Ames D, Vincent C. What is known about adverse events in older medical hospital inpatients? A systematic review of the literature. Int J Qual Health Care 2013 Oct;25(5):542-554. [CrossRef] [Medline]
    18. Morimoto T, Sakuma M, Matsui K, Kuramoto N, Toshiro J, Murakami J, et al. Incidence of adverse drug events and medication errors in Japan: the JADE study. J Gen Intern Med 2011 Feb;26(2):148-153 [FREE Full text] [CrossRef] [Medline]
    19. Fried LP, Ferrucci L, Darer J, Williamson JD, Anderson G. Untangling the concepts of disability, frailty, and comorbidity: implications for improved targeting and care. J Gerontol A Biol Sci Med Sci 2004 Mar;59(3):255-263. [CrossRef] [Medline]
    20. Hartgerink JM, Cramm JM, Bakker TJ, Mackenbach JP, Nieboer AP. The importance of older patients' experiences with care delivery for their quality of life after hospitalization. BMC Health Serv Res 2015 Aug 08;15:311 [FREE Full text] [CrossRef] [Medline]
    21. McKee M, Healy J, Edwards N, Harrison A. Pressures for change. In: McKee M, Healy J, editors. Hospitals in a Changing Europe. Buckingham, UK: Open University Press; 2002:36-58.
    22. Wier L, Pfuntner A, Steiner C. Hospital utilization among oldest adults, 2008: Statistical Brief #103. Healthcare Cost and Utilization Project (HCUP), Agency for Healthcare Research and Quality. 2010.   URL: https://www.hcup-us.ahrq.gov/re ports/statbriefs/sb103.pdf [accessed 2021-09-01]
    23. Spinewine A, Schmader KE, Barber N, Hughes C, Lapane KL, Swine C, et al. Appropriate prescribing in elderly people: how well can it be measured and optimised? Lancet 2007 Jul 14;370(9582):173-184. [CrossRef] [Medline]
    24. Jha AK, Larizgoitia I, Audera-Lopez C, Prasopa-Plaizier N, Waters H, Bates DW. The global burden of unsafe medical care: analytic modelling of observational studies. BMJ Qual Saf 2013 Oct;22(10):809-815. [CrossRef] [Medline]
    25. Dumbreck S, Flynn A, Nairn M, Wilson M, Treweek S, Mercer SW, et al. Drug-disease and drug-drug interactions: systematic examination of recommendations in 12 UK national clinical guidelines. BMJ 2015 Mar 11;350:h949 [FREE Full text] [CrossRef] [Medline]
    26. Schneider DJ, Sobel BE. Conundrums in the combined use of anticoagulants and antiplatelet drugs. Circulation 2007 Jul 17;116(3):305-315. [CrossRef] [Medline]
    27. Andreotti F, Rocca B, Husted S, Ajjan RA, ten Berg J, Cattaneo M, ESC Thrombosis Working Group. Antithrombotic therapy in the elderly: expert position paper of the European Society of Cardiology Working Group on Thrombosis. Eur Heart J 2015 Dec 07;36(46):3238-3249. [CrossRef] [Medline]
    28. Kanagaratnam L, Taam MA, Heng M, De Boissieu P, Roux MP, Trenque T. Les effets indésirables médicamenteux graves et leur évitabilité chez des sujets âgés de plus de 65 ans. Therapie 2015;70(5):477-484. [CrossRef] [Medline]
    29. Wasserfallen JB, Livio F, Buclin T, Tillet L, Yersin B, Biollaz J. Rate, type, and cost of adverse drug reactions in emergency department admissions. Eur J Intern Med 2001 Sep;12(5):442-447. [CrossRef] [Medline]
    30. Fattinger K, Roos M, Vergères P, Holenstein C, Kind B, Masche U, et al. Epidemiology of drug exposure and adverse drug reactions in two Swiss departments of internal medicine. Br J Clin Pharmacol 2000 Feb;49(2):158-167 [FREE Full text] [CrossRef] [Medline]
    31. Hardmeier B, Braunschweig S, Cavallaro M, Roos M, Pauli-Magnus C, Giger M, et al. Adverse drug events caused by medication errors in medical inpatients. Swiss Med Wkly 2004 Nov 13;134(45-46):664-670. [CrossRef] [Medline]
    32. Staines A, Mattia C, Schaad N, Lécureux E, Bonnabry P. Impact of a Swiss adverse drug event prevention collaborative. J Eval Clin Pract 2015 Aug;21(4):717-726. [CrossRef] [Medline]
    33. Lepori V, Perren A, Marone C. Unerwünschte internmedizinische Arzneimittelwirkungen bei Spitaleintritt. Schweiz Med Wochenschr 1999 Jun 19;129(24):915-922. [Medline]
    34. Institute of Medicine. In: Aspden P, Wolcott J, Bootman JL, editors. Preventing Medication Errors: Quality Chasm Series. Washington, DC, USA: The National Academies Press; 2007.
    35. Cresswell KM, Fernando B, McKinstry B, Sheikh A. Adverse drug events in the elderly. Br Med Bull 2007;83:259-274. [CrossRef] [Medline]
    36. Topinková E, Baeyens JP, Michel JP, Lang PO. Evidence-based strategies for the optimization of pharmacotherapy in older people. Drugs Aging 2012 Jun 01;29(6):477-494. [CrossRef] [Medline]
    37. Cooper JA, Cadogan CA, Patterson SM, Kerse N, Bradley MC, Ryan C, et al. Interventions to improve the appropriate use of polypharmacy in older people: a Cochrane systematic review. BMJ Open 2015 Dec 09;5(12):e009235 [FREE Full text] [CrossRef] [Medline]
    38. Onder G, Settanni S, Battaglia M. Authors' reply to Bahat: "Recommendations to prescribe in complex older adults: results of the CRIteria to assess appropriate Medication use among Elderly complex patients (CRIME) project". Drugs Aging 2014 Apr;31(4):329. [CrossRef] [Medline]
    39. Renom-Guiteras A, Meyer G, Thürmann PA. The EU(7)-PIM list: a list of potentially inappropriate medications for older people consented by experts from seven European countries. Eur J Clin Pharmacol 2015 Jul;71(7):861-875 [FREE Full text] [CrossRef] [Medline]
    40. Beeler PE, Eschmann E, Schumacher A, Studt JD, Amann-Vesti B, Blaser J. Impact of electronic reminders on venous thromboprophylaxis after admissions and transfers. J Am Med Inform Assoc 2014 Oct;21(e2):e297-e303 [FREE Full text] [CrossRef] [Medline]
    41. Beeler PE, Bates DW, Hug BL. Clinical decision support systems. Swiss Med Wkly 2014 Dec 23;144:w14073 [FREE Full text] [CrossRef] [Medline]
    42. Yourman L, Concato J, Agostini JV. Use of computer decision support interventions to improve medication prescribing in older adults: a systematic review. Am J Geriatr Pharmacother 2008 Jun;6(2):119-129. [CrossRef] [Medline]
    43. Classen DC, Pestotnik SL, Evans RS, Burke JP. Computerized surveillance of adverse drug events in hospital patients. 1991. Qual Saf Health Care 2005 Jun;14(3):221-226 [FREE Full text] [CrossRef] [Medline]
    44. Handler SM, Altman RL, Perera S, Hanlon JT, Studenski SA, Bost JE, et al. A systematic review of the performance characteristics of clinical event monitor signals used to detect adverse drug events in the hospital setting. J Am Med Inform Assoc 2007;14(4):451-458 [FREE Full text] [CrossRef] [Medline]
    45. Harinstein LM, Kane-Gill SL, Smithburger PL, Culley CM, Reddy VK, Seybert AL. Use of an abnormal laboratory value-drug combination alert to detect drug-induced thrombocytopenia in critically ill patients. J Crit Care 2012 Jun;27(3):242-249. [CrossRef] [Medline]
    46. Fritz D, Ceschi A, Curkovic I, Huber M, Egbring M, Kullak-Ublick GA, et al. Comparative evaluation of three clinical decision support systems: prospective screening for medication errors in 100 medical inpatients. Eur J Clin Pharmacol 2012 Aug;68(8):1209-1219. [CrossRef] [Medline]
    47. Chazard E, Preda C, Merlin B, Ficheur G, PSIP consortium, Beuscart R. Data-mining-based detection of adverse drug events. Stud Health Technol Inform 2009;150:552-556. [Medline]
    48. Ohno-Machado L, Nadkarni P, Johnson K. Natural language processing: algorithms and tools to extract computable information from EHRs and from the biomedical literature. J Am Med Inform Assoc 2013;20(5):805 [FREE Full text] [CrossRef] [Medline]
    49. Hazlehurst B, Mullooly J, Naleway A, Crane B. Detecting possible vaccination reactions in clinical notes. AMIA Annu Symp Proc 2005;2005:306-310 [FREE Full text] [Medline]
    50. Polepalli Ramesh B, Belknap SM, Li Z, Frid N, West DP, Yu H. Automatically recognizing medication and adverse event information from food and drug administration's adverse event reporting system narratives. JMIR Med Inform 2014 Jun 27;2(1):e10 [FREE Full text] [CrossRef] [Medline]
    51. Iqbal E, Mallah R, Jackson RG, Ball M, Ibrahim ZM, Broadbent M, et al. Identification of adverse drug events from free text electronic patient records and information in a large mental health case register. PLoS One 2015 Aug 14;10(8):e0134208 [FREE Full text] [CrossRef] [Medline]
    52. Aronson JK. Meyler's Side Effects of Drugs: The International Encyclopedia of Adverse Drug Reactions and Interactions. Amsterdam, The Netherlands: Elsevier; 2015.
    53. Schulman S, Kearon C, Subcommittee on Control of Anticoagulation of the Scientific and Standardization Committee of the International Society on Thrombosis and Haemostasis. Definition of major bleeding in clinical investigations of antihemostatic medicinal products in non-surgical patients. J Thromb Haemost 2005 Apr;3(4):692-694 [FREE Full text] [CrossRef] [Medline]
    54. Kearon C, Ageno W, Cannegieter SC, Cosmi B, Geersing GJ, Kyrle PA, Subcommittees on Control of Anticoagulation, and Predictive and Diagnostic Variables in Thrombotic Disease. Categorization of patients as having provoked or unprovoked venous thromboembolism: guidance from the SSC of ISTH. J Thromb Haemost 2016 Jul;14(7):1480-1483 [FREE Full text] [CrossRef] [Medline]
    55. Konstantinides SV, Torbicki A, Agnelli G, Danchin N, Fitzmaurice D, Galiè N, Task Force for the Diagnosis and Management of Acute Pulmonary Embolism of the European Society of Cardiology (ESC). 2014 ESC guidelines on the diagnosis and management of acute pulmonary embolism. Eur Heart J 2014 Nov 14;35(43):3033-369k. [CrossRef] [Medline]
    56. Fitch K, Bernstein SJ, Aguilar MD, Burnand B, LaCalle JR, Lázaro P, et al. The RAND/UCLA appropriateness method user's manual. RAND Europe. 2001.   URL: https://www.rand.org/content/dam/rand/pubs/monograph_reports/2011/MR1269.pdf [accessed 2021-09-01]
    57. Jones J, Hunter D. Consensus methods for medical and health services research. BMJ 1995 Aug 05;311(7001):376-380 [FREE Full text] [CrossRef] [Medline]
    58. Le Pogam MA, Quantin C, Reich O, Tuppin P, Fagot-Campagna A, Paccaud F, et al. Geriatric patient safety indicators based on linked administrative health data to assess anticoagulant-related thromboembolic and hemorrhagic adverse events in older inpatients: a study proposal. JMIR Res Protoc 2017 May 11;6(5):e82 [FREE Full text] [CrossRef] [Medline]
    59. 45 CFR Subpart E - Privacy of Individually Identifiable Health Information. Legal Information Institute, Cornell Law School.   URL: https://www.law.cornell.edu/cfr/text/45/part-164/subpart-E [accessed 2022-10-04]
    60. Quan H, Sundararajan V, Halfon P, Fong A, Burnand B, Luthi JC, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care 2005 Nov;43(11):1130-1139. [CrossRef] [Medline]
    61. Utter GH, Zrelak PA, Baron R, Tancredi DJ, Sadeghi B, Geppert JJ, et al. Detecting postoperative hemorrhage or hematoma from administrative data: the performance of the AHRQ Patient Safety Indicator. Surgery 2013 Nov;154(5):1117-1125. [CrossRef] [Medline]
    62. Hohl CM, Karpov A, Reddekopp L, Doyle-Waters M, Stausberg J. ICD-10 codes used to identify adverse drug events in administrative data: a systematic review. J Am Med Inform Assoc 2014;21(3):547-557 [FREE Full text] [CrossRef] [Medline]
    63. Li S, Hu J, Cui Y, Hu J. DeepPatent: patent classification with convolutional neural networks and word embedding. Scientometrics 2018 Sep 6;117(2):721-744. [CrossRef]
    64. Wan L, Papageorgiou G, Seddon M, Bernardoni M. Long-length legal document classification. arXiv 2019 Dec 14. [CrossRef]
    65. Devlin J, Chang MW, Lee K, Toutanova K. BERT: pre-training of deep bidirectional transformers for language understanding. arXiv 2018 Oct 11. [CrossRef]
    66. Zhu J, Xia Y, Wu L, He D, Qin T, Zhou W, et al. Incorporating BERT into neural machine translation. arXiv 2020 Feb 17. [CrossRef]
    67. Yang W, Xie Y, Lin A, Li X, Tan L, Xiong K, et al. End-to-end open-domain question answering with BERTserini. arXiv 2019 Feb 5. [CrossRef]
    68. Le H, Vial L, Frej J, Segonne V, Coavouox M, Lecouteux B, et al. FlauBERT: unsupervised language model pre-training for French. arXiv 2019 Dec 11. [CrossRef]
    69. Martin L, Muller B, Suárez PJ, Dupont Y, Romary L, de la Clergerie ÉV, et al. CamemBERT: a tasty French language model. arXiv 2019 Nov 10. [CrossRef]
    70. Scheible R, Thomczyk F, Tippmann P, Jaravine V, Boeker M. GottBERT: a pure German language model. arXiv 2020 Dec 3. [CrossRef]
    71. Gurulingappa H, Rajput AM, Roberts A, Fluck J, Hofmann-Apitius M, Toldo L. Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. J Biomed Inform 2012 Oct;45(5):885-892 [FREE Full text] [CrossRef] [Medline]
    72. Henry S, Buchan K, Filannino M, Stubbs A, Uzuner O. 2018 n2c2 shared task on adverse drug events and medication extraction in electronic health records. J Am Med Inform Assoc 2020 Jan 01;27(1):3-12 [FREE Full text] [CrossRef] [Medline]
    73. Naranjo CA, Busto U, Sellers EM, Sandor P, Ruiz I, Roberts EA, et al. A method for estimating the probability of adverse drug reactions. Clin Pharmacol Ther 1981 Aug;30(2):239-245. [CrossRef] [Medline]
    74. The use of the WHO-UMC system for standardized case causality assessment. World Health Organization. Uppsala, Sweden: The Uppsala Monitoring Centre; 2005.   URL: https://who-umc.org/media/164200/who-umc-causality-assessment_new-logo.pdf [accessed 2021-09-01]
    75. Grossmann N, Gratwohl F, Musy SN, Nielen NM, Donzé J, Simon M. Describing adverse events in medical inpatients using the Global Trigger Tool. Swiss Med Wkly 2019 Nov 04;149:w20149 [FREE Full text] [CrossRef] [Medline]
    76. Collet TH, Taffé P, Bordet J, Burnand B, Peytremann-Bridevaux I. Reproducibility of diabetes quality of care indicators as reported by patients and physicians. Eur J Public Health 2014 Dec;24(6):1004-1009. [CrossRef] [Medline]
    77. Taffé P, Halfon P, Ghali WA, Burnand B, International Methodology Consortium for Coded Health Information (IMECCHI). Test result-based sampling: an efficient design for estimating the accuracy of patient safety indicators. Med Decis Making 2012;32(1):E1-12. [CrossRef] [Medline]
    78. Sharma M, Cornelius VR, Patel JP, Davies JG, Molokhia M. Efficacy and harms of direct oral anticoagulants in the elderly for stroke prevention in atrial fibrillation and secondary prevention of venous thromboembolism: systematic review and meta-analysis. Circulation 2015 Jul 21;132(3):194-204 [FREE Full text] [CrossRef] [Medline]
    79. Anathhanam S, Powis RA, Cracknell AL, Robson J. Impact of prescribed medications on patient safety in older people. Ther Adv Drug Saf 2012 Aug;3(4):165-174 [FREE Full text] [CrossRef] [Medline]
    80. Takeda-Raguin C, Vogel T, Ferahta N, Smith C, Poloni B, Lang PO. Adherence to long-term drug regimen after hospital discharge: general practitioners' attitude. J Am Geriatr Soc 2016 Mar;64(3):657-659. [CrossRef] [Medline]
    81. De Geest S, Zúñiga F, Brunkert T, Deschodt M, Zullig LL, Wyss K, et al. Powering Swiss health care for the future: implementation science to bridge "the valley of death". Swiss Med Wkly 2020 Sep 07;150:w20323 [FREE Full text] [CrossRef] [Medline]
    82. Ageno W, Gallus AS, Wittkowsky A, Crowther M, Hylek EM, Palareti G. Oral anticoagulant therapy: Antithrombotic Therapy and Prevention of Thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest 2012 Feb;141(2 Suppl):e44S-e88S [FREE Full text] [CrossRef] [Medline]

    ‎
    ADE: adverse drug event
    ADR: adverse drug reaction
    BERT: Bidirectional Encoder Representations from Transformers
    CHOP: Swiss classification of surgical interventions
    DB: database
    EMR: electronic medical record
    GUH: Geneva University Hospital
    ICD-10: International Classification of Diseases, 10th Revision
    KSB: Baden Cantonal Hospital
    LUH: Lausanne University Hospital
    NER: named-entity recognition
    NLP: natural language processing
    NPV: negative predictive value
    pADE: preventable adverse drug event
    PPV: positive predictive value
    SDM: structured data mining
    USZ: Zürich University Hospital
    WP: work package

    Edited by T Leung; This paper was peer reviewed by the Swiss National Science Foundation. See the Multimedia Appendix for the peer-review report; submitted 22.06.22; accepted 07.07.22; published 15.11.22

    Copyright

    ©Frederic Gaspar, Monika Lutters, Patrick Emanuel Beeler, Pierre Olivier Lang, Bernard Burnand, Fabio Rinaldi, Christian Lovis, Chantal Csajka, Marie-Annick Le Pogam, SwissMADE study. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 15.11.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.