This paper is in the following e-collection/theme issue:

Formative Studies and eHealth/mHealth Development (524) Web-based and Mobile Health Interventions (3297) Innovations and Technology in Cancer Care (423) Patient Education for Cancer (147)

Published on in Vol 4, No 1 (2015): Jan-Mar

Development and Evaluation of an Educational E-Tool to Help Patients With Non-Hodgkin's Lymphoma Manage Their Personal Care Pathway

Development and Evaluation of an Educational E-Tool to Help Patients With Non-Hodgkin's Lymphoma Manage Their Personal Care Pathway

Development and Evaluation of an Educational E-Tool to Help Patients With Non-Hodgkin's Lymphoma Manage Their Personal Care Pathway

Authors of this article:

Jozette JC Stienen1 Author Orcid Image ;   Petronella B Ottevanger2 Author Orcid Image ;   Lianne Wennekes1 Author Orcid Image ;   Helena M Dekker3 Author Orcid Image ;   Richard WM van der Maazen4 Author Orcid Image ;   Caroline MPW Mandigers5 Author Orcid Image ;   Johan HJM van Krieken6 Author Orcid Image ;   Nicole MA Blijlevens7 Author Orcid Image ;   Rosella PMG Hermens1 Author Orcid Image
Article Authors Cited by (9) Tweetations (1) Metrics

    Original Paper

    1Radboud university medical center, Scientific Institute for Quality of Healthcare (IQ healthcare), Nijmegen, Netherlands

    2Radboud university medical center, Department of Medical Oncology, Nijmegen, Netherlands

    3Radboud university medical center, Department of Radiology, Nijmegen, Netherlands

    4Radboud university medical center, Department of Radiotherapy, Nijmegen, Netherlands

    5Canisius Wilhelmina Hospital, Department of Internal Medicine, Nijmegen, Netherlands

    6Radboud university medical center, Department of Pathology, Nijmegen, Netherlands

    7Radboud university medical center, Department of Hematology, Nijmegen, Netherlands

    Corresponding Author:

    Jozette JC Stienen, MSc

    Radboud university medical center

    Scientific Institute for Quality of Healthcare (IQ healthcare)

    PO Box 9101 (114)

    Nijmegen, 6500 HB

    Netherlands

    Phone: 31 24 36 67 310

    Fax:31 24 35 40 166

    Email: JozetteStienen@gmail.com


    Background: An overload of health-related information is available for patients on numerous websites, guidelines, and information leaflets. However, the increasing need for personalized health-related information is currently unmet.

    Objective: This study evaluates an educational e-tool for patients with non-Hodgkin’s lymphoma (NHL) designed to meet patient needs with respect to personalized and complete health-related information provision. The e-tool aims to help NHL patients manage and understand their personal care pathway, by providing them with insight into their own care pathway, the possibility to keep a diary, and structured health-related information.

    Methods: Together with a multidisciplinary NHL expert panel, we developed an e-tool consisting of two sections: (1) a personal section for patients’ own care pathway and their experiences, and (2) an informative section including information on NHL. We developed an ideal NHL care pathway based on the available (inter)national guidelines. The ideal care pathway, including date of first consultation, diagnosis, and therapy start, was used to set up the personal care pathway. The informative section was developed in collaboration with the patient association, Hematon. Regarding participants, 14 patients and 6 laymen were asked to evaluate the e-tool. The 24-item questionnaire used discussed issues concerning layout (6 questions), user convenience (3 questions), menu clarity (3 questions), information clarity (5 questions), and general impression (7 questions). In addition, the panel members were asked to give their feedback by email.

    Results: A comprehensive overview of diagnostics, treatments, and aftercare can be established by patients completing the questions from the personal section. The informative section consisted of NHL information regarding NHL in general, diagnostics, therapy, aftercare, and waiting times. Regarding participants, 6 patients and 6 laymen completed the questionnaire. Overall, the feedback was positive, with at least 75% satisfaction on each feedback item. Important strengths mentioned were the use of a low health-literacy level, the opportunity to document the personal care pathway and experiences, and the clear overview of the information provided. The added value of the e-tool in general was pointed out as very useful for preparing the consultation with one’s doctor and for providing all information on one website, including the opportunity for a personalized care pathway and diary. The majority of the revisions concerned wording and clarity. In addition, more explicit information on immunotherapy, experimental therapy, and psychosocial support was added.

    Conclusions: We have developed a personal care management e-tool for NHL patients. This tool contains a unique way to help patients manage their personal care pathway and give them insight into their NHL by providing health-related information and a personal diary. This evaluation showed that our e-tool meets patients’ needs concerning personalized health-related information, which might serve as a good example for other oncologic diseases. Future research should focus on the possible impact of the e-tool on doctor-patient communication during consultations.

    JMIR Res Protoc 2015;4(1):e6

    doi:10.2196/resprot.3407

    Keywords

    eHealth; personalized care; non-Hodgkin’s lymphoma; patient education; care pathway; consumer health information; empowerment; personal care management 


    Overview

    In the current digital era, patients are overloaded with health-related information. Many patient associations, health care institutes, hospitals, scientific societies, and guideline working groups provide their own information through websites and flyers. Unfortunately, the increasing call for personalized health-related information is still unmet [1-3]. On the one hand, this information need includes tools that provide support during interaction with caregivers, such as question sheets, decision aids, and option grids [4-6]. These tools aim at providing information about available (treatment) options and possible risks to make a well-informed decision. Decision aids, for example, are shown to be effective with regard to improvement of patients’ involvement and health-related knowledge [7].

    On the other hand, patients ask for more insight into their personal care pathway, including diagnostics, therapy, and aftercare [8], which makes it easier to act as managers of their own care. This also points to personalized care, which can be defined as the delivery of care that is tailored to an individual patient. Important elements are (1) the delivery of care that is responsive to individual preferences, needs, values, and possibilities, and (2) as much as possible, patients’ engagement in their own care and health. The latter point needs a well-informed patient, who has insight into his personal care pathway. In the literature, roadmaps or care pathways concerning patient care are mostly directed to professionals [9,10]. However, making these available for patients could help them in their personal care management.

    In addition, reliable health-related information is particularly important for patients, where the Internet is an information source of growing importance. A national survey in the United States showed that 59% of adults searched online for health-related information in 2012 [11]. In European countries, over 80% of adults used the Internet as the main source for health-related information in 2011 and 2012 [12,13]. Several quality criteria have been developed worldwide to monitor the quality of easily accessible health-related information on the Internet [14]. The best-known quality criteria are found in the Health On the Net Code of Conduct (HONcode) for websites [15]. Previous research showed that it remains difficult to accurately monitor all information posted on the Internet, for example, online information concerning cancer and other disease-related topics is still of poor quality [16-21].

    For non-Hodgkin’s lymphoma (NHL), a heterogeneous group of over 40 types of malignant lymphomas, an abundance of health-related information is available online. Previous research showed that patients diagnosed with NHL would like to have access to more complete and personalized information on diagnostics, therapy, and aftercare [2]. In response to this, we developed a unique e-tool for NHL patients. This study is, to our knowledge, the first description on the development and evaluation of a personalized care pathway for NHL patients that is also linked to the available health-related information concerning NHL.

    Aim and Objectives

    This paper describes the development and evaluation of a unique, educational e-tool for NHL patients, aiming to help patients manage and understand their personal NHL care pathway. This is done by providing insight into their personal care pathway based on the data patients enter and by providing essential information about NHL care. Additionally, patients are given the possibility to register personal experiences in their care pathway, as they would in a diary. We hypothesize that having access to all the information available in the e-tool, patients will have a better understanding of their disease and will be able to act as managers of their own care pathways during interaction with their caregivers.


    Methods

    Setting

    The e-tool described in this paper was developed in the context of the PEARL study (improvement of patients’ hospital care for non-Hodgkin’s lymphoma), aimed at improving hospital care for patients diagnosed with NHL [22]. In a previous study, insight into current NHL care was acquired by measuring quality indicators at the patient level [23]. Together with the barriers and facilitators found, as perceived by patients and physicians [2], a tailored improvement strategy was developed. Next to several physician-directed tools, an e-tool for patients was included in the improvement strategy. This paper describes the development and evaluation of an e-tool for NHL patients.

    E-Tool Development

    Overview

    We started with the development of a mock-up storyboard, and with the help of a system developer this was converted into a distinctive e-tool for NHL patients, tailored to address the patients’ barriers found in previous research [2]. The barriers included lack of insight into the patients’ personal care pathway, and lack of written information about diagnostics and therapy. This is why the e-tool developed consists of two sections: (1) a personal section for patients’ own NHL care pathway (roadmap) and their experiences with NHL care, and (2) an informative section including information on NHL. The complete e-tool aims to help patients manage the care they receive during their NHL care trajectory. The webmaster of the Dutch Lymphoma Patient Association (Hematon, known as LVN before 2014) and an expert panel, including a hematologist, radiologist, pathologist, radiation oncologist, epidemiologist, and a senior researcher, were closely involved in the e-tool development. Quality criteria from the HONcode were taken into account during the development process.

    Personal Section of the E-Tool

    In this section, users were able to document dates and experiences of first consultation, diagnostics, and therapy. The aim was to provide insight into the patients’ personal care pathway. We developed a general (ideal) NHL care pathway based on the national guidelines and recommendations available in the Netherlands. These included, among others, the Dutch NHL guidelines, recommendations of the Cooperative Trial Group for Hematology Oncology (HOVON) and the Dutch Society of Hematology (NVvH), general recommendations for acceptable waiting times, and an NHL guideline based on patients’ perspectives of Hematon. Internationally available guidelines were consulted when applicable. The ideal care pathway, including date of first consultation, date of diagnosis, and start date of therapy, was used to set up the format for the personal care pathway (roadmap).

    Informative Section of the E-Tool

    Users had access to reliable information on NHL and NHL care through this section of the e-tool. The aim was to cluster all reliable, online available information and make it understandable for all users. Too much or confusing information and a high health-literacy level were avoided as much as possible. The Dutch NHL guidelines, NVvH, HOVON, the Dutch Cancer Society (KWF), and several NHL-related websites (eg, Radiotherapie Nederland, Chemo and nu, Hematon, and Hematologie Groningen) were used as sources of NHL information for the content. These sources are frequently recommended and used by professionals, so the content can be considered as authorized by them. We cooperated with the webmaster of Hematon to make sure that the information included was complete and accurate. The format of the e-tool was based on the NHL care pathway as described in the guidelines and on user experiences with other websites.

    E-Tool Evaluation

    After evaluation of the concept of the text by the expert panel, an external hematologist, and the webmaster of Hematon, an assessment was set up to evaluate both sections of our e-tool concerning applicability. We recruited NHL patients via Hematon and a university hospital in the Netherlands. Patients were eligible when they had, or have had, a non-Hodgkin’s lymphoma, were over 18 years old, and had access to the Internet. Family and friends of 3 members of the project team were asked to participate as laymen. The e-tool was evaluated by patients and laymen using a 24-item questionnaire—19 yes/no questions, including space for reasoning, and 5 open-ended questions, including improvement suggestions. The questionnaire included questions on layout (6 questions), user convenience (3 questions), menu clarity (3 questions), information clarity (5 questions), and general impression (7 questions), for example, strengths and missing information. This method allowed us to obtain as much information as possible to improve our e-tool. The expert panel was invited by email to give general feedback on the e-tool.


    E-Tool Development

    Overview

    We developed an e-tool, consisting of two sections—a personal section and an informative section. The content of the sections will be described below, followed by the results of the evaluation of the e-tool applicability. The e-tool is available in Dutch [24] for participants of the PEARL study and will become publicly available in 2015. The e-tool started with a general introduction, which described the aim of the e-tool and gave a short overview of the personal and informative sections of the e-tool. Both sections were formatted in chronological order (diagnostics, therapy, aftercare), which is in line with the NHL care pathway as described in the guidelines and seen in clinical practice. Additionally, background information was available concerning the PEARL study, participating hospitals, and contact information.

    Personal Section of the E-Tool

    This section consisted of questions concerning a patient’s personal NHL care pathway, which allowed patients to develop a unique overview of all diagnostics, treatments, and aftercare/follow-up that they have had. Questions were divided into four items:

    1. Diagnostic examinations, including items such as type and date of the examination, executed by whom, and in which hospital.

    2. Diagnosis, including date of first consultation, date of diagnosis, type of NHL, stage of the disease, risk profile, date of treatment plan, hospital name, and physician name.

    3. Therapy, including type and date of therapy, executed by whom, and in which hospital.

    4. Aftercare, including items such as type and date of aftercare, executed by whom, and contact information.

    An easy link from the personal section to the informative section was provided next to each item. In this way, patients could look up the underlying information belonging to the different items. Additionally, it was possible to create a diary, to document their own experiences with the different diagnostic examinations or treatments. Information filled out in this diary was only visible to the patients themselves. Finally, an overview of the personal NHL care pathway was obtained using the answers to the questions. The personal diary—chronologically visualized experiences with NHL care—was provided when applicable. An example of the personal NHL care pathway (roadmap) is shown in Figure 1.

    ‎
    Figure 1. Example of a personal NHL care pathway (personal section of the e-tool). Original is in Dutch.
    View this figure
    Informative Section of the E-Tool

    This section consisted of NHL information from reliable online sources, divided into six information items:

    1. NHL in general, including a description of what NHL is, symptoms, and most occurring NHLs.

    2. Diagnostic examinations, including items such as blood sample, tissue biopsy, bone marrow biopsy/aspirate, PET/CT-scan, and additional examinations.

    3. Diagnosis, including stage, international prognostic index (IPI) score, treatment plan, and additional support.

    4. Therapy, including wait-and-see, radiotherapy, chemotherapy, immune therapy, stem cell transplantation, and clinical trials.

    5. Aftercare, including response evaluation and monitoring.

    6. Waiting times.

    External links for more extensive information were added as much as possible, together with short educational movies created by KWF or Hematon. An example of the informative section is shown in Figure 2.

    ‎
    Figure 2. Example of informative section of the e-tool. Original is in Dutch.
    View this figure

    E-Tool Evaluation

    We asked 20 people to participate in our test, including 14 NHL patients and 6 laymen. In total, 13 people logged in and 12 returned the questionnaire, which resulted in a 60% (12/20) response rate. Little feedback was given by 1 participant by email and he did not return the questionnaire. The mean age of the 12 participants (patients and laymen) that completed the questionnaire was 56 years (SD 11), with a range of 30-70 years. Of the participants, 42% (5/12) were female and 67% (8/12) had a high education level (college or university). Additionally, all 6 panel members and the external hematologist provided general feedback by email.

    Table 1 includes the results of our e-tool evaluation (applicability) using closed questions among NHL patients and laymen. Overall, the evaluation of the e-tool was highly positive. Table 1 shows at least 75% (9/12) satisfaction on each section. Concerning layout, all responders were satisfied with the clarity of the text, writing style, and clarity of the illustrations and short videos. When looking at the user convenience, 11 out of 12 participants (92%) were satisfied with the format and speed of the e-tool. In relation to menu clarity, all participants found the menu format intuitive and the information needed was easy to find. Information clarity, including understandability of the text, was evaluated positively as well. Only 2 out of 12 participants (17%) thought too much medical terminology was included.

    Table 1. Results of the e-tool evaluation among NHL patients and laymen (closed questions).
    Question
    		Positive response, n (%)


    		Patients (n=6)Laymen (n=6)
    Layout


    		Is the text clear?6 (100)6 (100)

    		Are you satisfied with letter type and size?3 (50)6 (100)

    		Are you satisfied with the writing style?6 (100)6 (100)

    		Is the use of colors attractive to you?4 (67)5 (83)

    		Are there enough illustrations and short videos to support the text?5 (83)5 (83)

    		Are the illustrations and short videos clear?6 (100)6 (100)
    User convenience


    		Is the e-tool easy to use?4 (67)5 (83)

    		Is the format of the e-tool easy to understand?5 (83)6 (100)

    		Are you satisfied with the speed of the e-tool?6 (100)6 (100)
    Menu clarity


    		Does the composition of the menu seem natural?6 (100)6 (100)

    		Are you satisfied with the navigation through the e-tool?5 (83)5 (83)

    		Is it easy to find the information you are looking for?6 (100)6 (100)
    Information clarity


    		Is the information provided on the e-tool understandable?6 (100)6 (100)

    		Does the e-tool make use of too much medical terminology?a4 (67)6 (100)

    		Do you understand what NHL is, after reading the information provided on the e-tool?6 (100)6 (100)

    		Do you understand what treatment options are available, after reading the e-tool?6 (100)6 (100)
    General impression


    		Are there any flaws/errors in this e-tool?a5 (83)5 (83)

    		Would you use this e-tool if made available to you?5 (83)6 (100)

    		Are there any redundant items in this e-tool?a6 (100)6 (100)

    aPercentage of negative responses presented, caused by negative questioning.

    Table 2 includes the results of our e-tool evaluation using open-ended questions, and shows mainly positive remarks about the e-tool and some improvement points, according to NHL patients and laymen. Valuable strengths mentioned by the participants included the use of a low health-literacy level, the opportunity to document their personal care pathways and experiences (diary), and the clear and helpful overview of the information provided. The added value of the e-tool in general was pointed out as very useful for preparing for their consultations with their doctors and for providing all information in one website, including the opportunity for a personalized care pathway and diary.

    In general, the e-tool is considered to be a good initiative to help patients manage their NHL care. For example, one participant wrote, “I think this tool is excellent and that it will be helpful in dealing with the fact that you are diagnosed with NHL. The tool can contribute to an optimal doctor-patient contact.” There were no distinct differences in feedback between patients and laymen, except that patients provided more comments concerning missing items and general remarks.

    Improvement points proposed by the expert panel included addition of a search function and contact button, adding items to the personal care pathway section, suggestions for links to general cancer websites, and some wording issues (data not shown).

    During the review of the e-tool, all feedback was taken into account. The majority of the revisions concerned wording and clarity. With respect to layout, the font size regarding explanatory text was enlarged and the color of the submenu was made less bright. Regarding user convenience and menu clarity, the navigation at the bottom of each page was made clearer. No revisions were made concerning information clarity. In addition, more explicit information was included about immune therapy, experimental therapy, and psychosocial support. As well, items for the patients’ risk profiles and stages of disease were added to the personal section.

    Table 2. Results of the e-tool evaluation among NHL patients and laymen (open-ended questions).
    QuestionResponse
    Information clarity

    		What does this e-tool add to the information provided by your doctor?Makes everything much clearer and understandable.
    Makes it possible to read everything at a quiet moment.
    Provides good preparation for consultation with your doctor.
    Provides a great overview of all necessary information.
    Provides a good overview of your own care pathway.
    The e-tool is very useful for the patients’ relatives.
    Makes it possible to compare with the ideal pathway.
    General impression

    		What are improvement points for this e-tool?More expressive color use.
    For patients that forget updating their care pathway, are there alerts?
    Better navigation through the care pathway.
    Clearer definition of first consult.

    		What are the strengths of this e-tool?Opportunity to document your personal care pathway.
    Opportunity to document your experiences in a diary.
    Possible to compare with quality criteria (ideal pathway).
    Low health-literacy level.
    Clear and calm colors.
    Clear overview of medical information and terminology.
    Clear layout and navigation.
    External links to additional information about NHL.
    Overview of information gives the opportunity to translate this to yourself.

    		What did you miss on this e-tool?Overview of lymph vessels.
    Information about immune therapy and clinical trials.
    Information about psychosocial support.
    Information about second opinion.

    		General remarks?Great overview of available NHL information.
    Other websites are fine, but this e-tool provides more specific information.
    I already have my own diary, which I will use in the future.
    The tool gives the opportunity to create your own care pathway and diary—this has a positive influence on dealing with the fact that you are diagnosed with NHL.
    This e-tool can contribute to an optimal doctor-patient contact during consultations.

    Principal Results

    In this study, we showed the development and evaluation of a unique e-tool for, and by, patients with non-Hodgkin’s lymphoma. Based on the evaluation, the e-tool is a feasible tool that gives patients insight into their personal care pathway and informs them about NHL. The overall positive feedback implies the e-tool’s added value of providing personalized health information, which is often an unmet need for patients. Taken as a whole, this is the first e-tool in the field of oncology that aims at helping patients in their personal care management and provides an additional overview of information about NHL.

    Comparison With Prior Work

    Involving patients in the development of health care improvement tools has been an upcoming phenomenon in the past several years. Nowadays, the patient’s voice is incorporated more often in clinical guidelines, quality indicators, and shared decision-making instruments [8,25,26]. It is important to support patients involved in developing health care improvement tools. For example, a wiki has been established for, and tested by, patients involved in the development of a guideline on infertility [27]. The shared decision-making instruments themselves also have a supporting role, as they try to help patients in making informed decisions [6,28-30]. However, not all decision aids are directed to patients [31,32] and they usually do not provide insight into the complete personal care pathway.

    Studies focusing on personalized care pathways and assistance with patients’ personal care management are sparsely described in the literature. Most studies concerning the development of patient care pathways or roadmaps concentrate on the education of physicians [33,34] or other health care professionals [10]. Ryhänen et al [35] and Dykes et al [36] developed tools to educate patients about their care pathway. Dykes et al aimed at providing education about the anticipated length of stay and treatment plan after acute myocardial infarction. Development and evaluation of a breast cancer pathway, an Internet-based patient education program, was described by Ryhänen et al. In line with our e-tool, these studies focused on patient education concerning the patient care pathway. However, these studies did not include information tailored to the individual patient, such as a chronological overview of the personal care pathway or a diary. Atack et al [37] developed an online patient education system including health-related information tailored to the individual patient. Physicians were able to “prescribe” information that met the patient’s need. However, this system was not focused on the total care pathway and was only pilot-tested in 8 patients. Our e-tool is the first that combines the unmet needs of patients, including personalized information about the patients’ care pathways and the possibility to create a diary, together with an overview of all necessary NHL information.

    Limitations

    Four limitations were identified with respect to this e-tool. First, this e-tool has been developed for NHL patients, a patient category with a more advanced age—incidence of NHL increases with age—that might not be active on the Internet. However, the mean age of NHL patients included in this study was 61 years (SD 7), and a previous study on possible barriers in NHL care also showed inclusion of patients of all ages [2]. Furthermore, health-related Internet use seems to increase over time in Europe among people of all ages [38]. Second, the questionnaire used to evaluate the e-tool was not validated, but based on evaluation factors derived from the human, organization, and technology-fit (HOT-fit) framework [39]. We believe that the questionnaire developed included all necessary items to extensively evaluate the e-tool.

    Third, the evaluation study only assessed feasibility of the e-tool on a small scale. Unfortunately, no information was available about the 7 nonresponders, and a relatively high percentage of the responders were highly educated, which could have introduced some bias. A next step in the implementation is to test the e-tool in daily NHL care and evaluate user experiences on a larger scale. This might give insight into the possibilities for making the e-tool appropriate on an international level and it might serve as a good example for other oncologic diseases.

    Finally, the e-tool is not yet certified by the HONcode. However, all eight items of the code of conduct for medical and health websites as described by HON were taken into account during the development of the e-tool [40]. After testing the e-tool at a larger scale we intend to apply for certification.

    Conclusions

    We have developed a personalized approach using an e-tool for NHL patients. This tool contains a unique way to help patients with their care management—it provides insight into the personal care pathway and offers general information about NHL. In this evaluation study, we report high satisfaction rates and some improvement points for future use. We expect that the e-tool will have a positive impact on both patient empowerment and doctor-patient communication, since patients are more informed in lay language about NHL and their personal care pathway. It is suggested that better-informed patients are able to ask more specific questions, which makes it possible to improve the management of their personal care. Finally, this e-tool may play an important role in dealing with NHL, particularly the personal diary option, which has the potential to provide psychosocial support for personal experiences in NHL care. The usage and effects of the e-tool should be tested and evaluated in future research, which is included as one of the purposes of the PEARL study [22].

    Acknowledgments

    The authors would like to thank everyone who contributed to the development of the e-tool, especially D Pasveer, J Liefers, F Brouwer (IQ healthcare, Radboudumc), and J Hefkens (Hematon). The authors also thank all participants of this e-tool evaluation. The authors would like to thank the Netherlands Organization for Health Research and Development (ZonMW), The Hague, for funding this study (grant No. 171103002) and the Dutch Organization for Scientific Research (NWO) for funding the open access publication. Funders did not have any role in the design and conduct of the study, the analysis of data, or in the preparation of the manuscript.

    Authors' Contributions

    RH, PO, and LW had the basic idea for this e-tool study. RH, PO, and JS were responsible for the research question of the study. NB and JvK commented on the design of the study, and all authors commented on the design of the e-tool. JS wrote the first draft of this manuscript and was responsible for the revisions. All authors critically reviewed the drafts and approved the final manuscript before submission.

    Conflicts of Interest

    None declared.

    1. Posma ER, van Weert JC, Jansen J, Bensing JM. Older cancer patients' information and support needs surrounding treatment: an evaluation through the eyes of patients, relatives and professionals. BMC Nurs 2009;8:1 [FREE Full text] [CrossRef] [Medline]
    2. Stienen JJ, Ottevanger PB, Wennekes L, van de Schans SA, Dekker HM, Blijlevens NM, et al. Delivering high-quality care to patients with a non-Hodgkin's lymphoma: barriers perceived by patients and physicians. Neth J Med 2014 Jan;72(1):41-48 [FREE Full text] [Medline]
    3. van den Boogaard NM, van den Boogaard E, Bokslag A, van Zwieten MC, Hompes PG, Bhattacharya S, et al. Patients' and professionals' barriers and facilitators of tailored expectant management in subfertile couples with a good prognosis of a natural conception. Hum Reprod 2011 Aug;26(8):2122-2128 [FREE Full text] [CrossRef] [Medline]
    4. Elwyn G, Lloyd A, Joseph-Williams N, Cording E, Thomson R, Durand MA, et al. Option Grids: shared decision making made easier. Patient Educ Couns 2013 Feb;90(2):207-212. [CrossRef] [Medline]
    5. Kinnersley P, Edwards A, Hood K, Ryan R, Prout H, Cadbury N, et al. Interventions before consultations to help patients address their information needs by encouraging question asking: systematic review. BMJ 2008;337:a485 [FREE Full text] [Medline]
    6. Schoorel EN, Vankan E, Scheepers HC, Augustijn BC, Dirksen CD, de Koning M, et al. Involving women in personalised decision-making on mode of delivery after caesarean section: the development and pilot testing of a patient decision aid. BJOG 2014 Jan;121(2):202-209. [CrossRef] [Medline]
    7. Stacey D, Légaré F, Col NF, Bennett CL, Barry MJ, Eden KB, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev 2014;1:CD001431. [CrossRef] [Medline]
    8. Ouwens M, Hermens R, Hulscher M, Vonk-Okhuijsen S, Tjan-Heijnen V, Termeer R, et al. Development of indicators for patient-centred cancer care. Support Care Cancer 2010 Jan;18(1):121-130 [FREE Full text] [CrossRef] [Medline]
    9. Maclennan SJ, Maclennan SJ, Imamura M, Omar MI, Vale L, Lam T, UCAN Care Pathway Development Group. Urological cancer care pathways: development and use in the context of systematic reviews and clinical practice guidelines. World J Urol 2011 Jun;29(3):291-301 [FREE Full text] [CrossRef] [Medline]
    10. Viklund P, Lagergren J. A care pathway for patients with oesophageal cancer. Eur J Cancer Care (Engl) 2007 Nov;16(6):533-538. [CrossRef] [Medline]
    11. Fox S, Duggan M. Pew Research Center’s Internet and American Life Project. 2013 Jan 15. Health Online 2013   URL: http://www.pewinternet.org/files/old-media/Files/Reports/PIP_HealthOnline.pdf [accessed 2013-11-25] [WebCite Cache]
    12. Bianco A, Zucco R, Nobile CG, Pileggi C, Pavia M. Parents seeking health-related information on the Internet: cross-sectional study. J Med Internet Res 2013;15(9):e204 [FREE Full text] [CrossRef] [Medline]
    13. Van de Belt TH, Engelen LJ, Berben SA, Teerenstra S, Samsom M, Schoonhoven L. Internet and social media for health-related information and communication in health care: preferences of the Dutch general population. J Med Internet Res 2013;15(10):e220 [FREE Full text] [CrossRef] [Medline]
    14. Bernstam EV, Shelton DM, Walji M, Meric-Bernstam F. Instruments to assess the quality of health information on the World Wide Web: what can our patients actually use? Int J Med Inform 2005 Jan;74(1):13-19. [CrossRef] [Medline]
    15. Boyer C, Selby M, Scherrer JR, Appel RD. The Health On the Net Code of Conduct for medical and health websites. Comput Biol Med 1998 Sep;28(5):603-610. [Medline]
    16. Chan DS, Willicombe A, Reid TD, Beaton C, Arnold D, Ward J, et al. Relative quality of Internet-derived gastrointestinal cancer information. J Cancer Educ 2012 Dec;27(4):676-679. [CrossRef] [Medline]
    17. Fast AM, Deibert CM, Boyer C, Hruby GW, McKiernan JM. Partial nephrectomy online: a preliminary evaluation of the quality of health information on the Internet. BJU Int 2012 Dec;110(11 Pt B):E765-E769. [CrossRef] [Medline]
    18. Killeen S, Hennessey A, El Hassan Y, Killeen K, Clarke N, Murray K, et al. Gastric cancer-related information on the Internet: incomplete, poorly accessible, and overly commercial. Am J Surg 2011 Feb;201(2):171-178. [CrossRef] [Medline]
    19. Quinn EM, Corrigan MA, McHugh SM, Murphy D, O'Mullane J, Hill AD, et al. Breast cancer information on the Internet: analysis of accessibility and accuracy. Breast 2012 Aug;21(4):514-517. [CrossRef] [Medline]
    20. Showghi NN, Williams AC. Information about male chronic pelvic and urogenital pain on the Internet: an evaluation of Internet resources. Pain Med 2012 Oct;13(10):1275-1283. [CrossRef] [Medline]
    21. Thompson AE, Graydon SL. Patient-oriented methotrexate information sites on the Internet: a review of completeness, accuracy, format, reliability, credibility, and readability. J Rheumatol 2009 Jan;36(1):41-49. [CrossRef] [Medline]
    22. Stienen JJ, Hermens RP, Wennekes L, van de Schans SA, Dekker HM, Blijlevens NM, et al. Improvement of hospital care for patients with non-Hodgkin's lymphoma: protocol for a cluster randomized controlled trial (PEARL study). Implement Sci 2013;8:77 [FREE Full text] [CrossRef] [Medline]
    23. Wennekes L, Ottevanger PB, Raemaekers JM, Schouten HC, de Kok MW, Punt CJ, et al. Development and measurement of guideline-based indicators for patients with non-Hodgkin's lymphoma. J Clin Oncol 2011 Apr 10;29(11):1436-1444 [FREE Full text] [CrossRef] [Medline]
    24. Non-Hodgkinlymfoom: het zorgtraject in het ziekenhuis. Nijmegen, Netherlands: PEARL study, Radboudumc   URL: https://nhl-info.nl/login.php [accessed 2014-03-17] [WebCite Cache]
    25. Archambault PM. WikiBuild: a new application to support patient and health care professional involvement in the development of patient support tools. J Med Internet Res 2011;13(4):e114 [FREE Full text] [CrossRef] [Medline]
    26. Pittens CA, Vonk Noordegraaf A, van Veen SC, Anema JR, Huirne JA, Broerse JE. The involvement of gynaecological patients in the development of a clinical guideline for resumption of (work) activities in the Netherlands. Health Expect 2013 Aug 29. [CrossRef] [Medline]
    27. den Breejen EM, Nelen WL, Knijnenburg JM, Burgers JS, Hermens RP, Kremer JA. Feasibility of a wiki as a participatory tool for patients in clinical guideline development. J Med Internet Res 2012;14(5):e138 [FREE Full text] [CrossRef] [Medline]
    28. Li LC, Adam PM, Townsend AF, Lacaille D, Yousefi C, Stacey D, et al. Usability testing of ANSWER: a Web-based methotrexate decision aid for patients with rheumatoid arthritis. BMC Med Inform Decis Mak 2013;13:131 [FREE Full text] [CrossRef] [Medline]
    29. Drugs and Therapeutics Bulletin Editorial Office. An introduction to patient decision aids. BMJ 2013 Jul 23;347:f4147. [Medline]
    30. Schonberg MA, Hamel MB, Davis RB, Griggs MC, Wee CC, Fagerlin A, et al. Development and evaluation of a decision aid on mammography screening for women 75 years and older. JAMA Intern Med 2014 Mar;174(3):417-424. [CrossRef] [Medline]
    31. Brodin NP, Maraldo MV, Aznar MC, Vogelius IR, Petersen PM, Bentzen SM, et al. Interactive decision-support tool for risk-based radiation therapy plan comparison for Hodgkin lymphoma. Int J Radiat Oncol Biol Phys 2014 Feb 1;88(2):433-445. [CrossRef] [Medline]
    32. Tartar A, Kiliç N, Akan A. A new method for pulmonary nodule detection using decision trees. In: Conf Proc IEEE Eng Med Biol Soc.: IEEE; 2013 Presented at: 35th Annual International Conference of the IEEE; July 3-7, 2013; Osaka, Japan p. 7355-7359. [CrossRef]
    33. Celano G, Costa A, Fichera S, Tringali G. Linking Six Sigma to simulation: a new roadmap to improve the quality of patient care. Int J Health Care Qual Assur 2012;25(4):254-273. [CrossRef] [Medline]
    34. NVA. Richtlijnendatabase. Richtlijn peroperatief traject, praktijkvoorbeelden   URL: http:/​/richtlijnendatabase.​nl/​gerelateerde_documenten/​f/​212/​Bijlage%207%20praktijkvoorbeelden%20pdf.​pdf [accessed 2014-03-12] [WebCite Cache]
    35. Ryhänen AM, Rankinen S, Tulus K, Korvenranta H, Leino-Kilpi H. Internet based patient pathway as an educational tool for breast cancer patients. Int J Med Inform 2012 Apr;81(4):270-278. [CrossRef] [Medline]
    36. Dykes PC, Currie L, Bakken S. Patient Education and Recovery Learning System (PEARLS) pathway: a tool to drive patient centered evidence-based practice. J Healthc Inf Manag 2004;18(4):67-73. [Medline]
    37. Atack L, Luke R, Chien E. Evaluation of patient satisfaction with tailored online patient education information. Comput Inform Nurs 2008;26(5):258-264. [CrossRef] [Medline]
    38. Andreassen HK, Bujnowska-Fedak MM, Chronaki CE, Dumitru RC, Pudule I, Santana S, et al. European citizens' use of E-health services: a study of seven countries. BMC Public Health 2007;7:53 [FREE Full text] [CrossRef] [Medline]
    39. Yusof MM, Kuljis J, Papazafeiropoulou A, Stergioulas LK. An evaluation framework for Health Information Systems: human, organization and technology-fit factors (HOT-fit). Int J Med Inform 2008 Jun;77(6):386-398. [CrossRef] [Medline]
    40. Health On the Net Foundation. The HON Code of Conduct for medical and health Web sites (HONcode)   URL: http://www.hon.ch/HONcode/Patients/Conduct.html [accessed 2014-03-12] [WebCite Cache]

    ‎
    HONcode: Health On the Net Code of Conduct
    HOT-fit: human, organization, and technology-fit
    HOVON: Cooperative Trial Group for Hematology Oncology
    IPI: international prognostic index
    KWF: Dutch Cancer Society
    NHL: non-Hodgkin’s lymphoma
    NVvH: Dutch Society of Hematology
    NWO: Dutch Organization for Scientific Research
    PEARL: improvement of patients’ hospital care for non-Hodgkin’s lymphoma
    ZonMW: Netherlands Organization for Health Research and Development

    Edited by G Eysenbach; submitted 18.03.14; peer-reviewed by L Ali, A Mehta, D Focosi; comments to author 24.07.14; revised version received 01.09.14; accepted 21.10.14; published 09.01.15

    Copyright

    ©Jozette JC Stienen, Petronella B Ottevanger, Lianne Wennekes, Helena M Dekker, Richard WM van der Maazen, Caroline MPW Mandigers, Johan HJM van Krieken, Nicole MA Blijlevens, Rosella PMG Hermens. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 09.01.2015.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.