This paper is in the following e-collection/theme issue:

Methods and Feasibility Studies (379) New Methods (944) Telehealth and Telemonitoring (1671)

Published on in Vol 5, No 2 (2016): Apr-Jun

Web Health Monitoring Survey: A New Approach to Enhance the Effectiveness of Telemedicine Systems

Web Health Monitoring Survey: A New Approach to Enhance the Effectiveness of Telemedicine Systems

Web Health Monitoring Survey: A New Approach to Enhance the Effectiveness of Telemedicine Systems

Authors of this article:

Maria Francesca Romano1 Author Orcid Image ;   Maria Vittoria Sardella1 Author Orcid Image ;   Fabrizio Alboni1 Author Orcid Image
Article Authors Cited by (2) Tweetations (4) Metrics

    Original Paper

    Institute of Economics, Scuola Superiore Sant'Anna, Pisa, Italy

    *all authors contributed equally

    Corresponding Author:

    Maria Francesca Romano, MSc

    Institute of Economics

    Scuola Superiore Sant'Anna

    piazza Martiri della Libertà 33

    Pisa, 56127

    Italy

    Phone: 39 50883388

    Fax:39 50883344

    Email: mariafrancesca.romano@sssup.it


    Background: Aging of the European population and interest in a healthy population in western countries have contributed to an increase in the number of health surveys, where the role of survey design, data collection, and data analysis methodology is clear and recognized by the whole scientific community. Survey methodology has had to couple with the challenges deriving from data collection through information and communications technology (ICT). Telemedicine systems have not used patients as a source of information, often limiting them to collecting only biometric data. A more effective telemonitoring system would be able to collect objective and subjective data (biometric parameters and symptoms reported by the patients themselves), and to control the quality of subjective data collected: this goal be achieved only by using and merging competencies from both survey methodology and health research.

    Objective: The objective of our study was to propose new metrics to control the quality of data, along with the well-known indicators of survey methodology. Web questionnaires administered daily to a group of patients for an extended length of time are a Web health monitoring survey (WHMS) in a telemedicine system.

    Methods: We calculated indicators based on paradata collected during a WHMS study involving 12 patients, who signed in to the website daily for 2 months.

    Results: The patients’ involvement was very high: the patients’ response rate ranged between 1.00 and 0.82, with an outlier of 0.65. Item nonresponse rate was very low, ranging between 0.0% and 7.4%. We propose adherence to the chosen time to connect to the website as a measure of involvement and cooperation by the patients: the difference from the median time ranged between 11 and 24 minutes, demonstrating very good cooperation and involvement from all patients. To measure habituation to the questionnaire, we also compared nonresponse rates to the items between the first and the second month of the study, and found no significant difference. We computed the time to complete the questionnaire both as a measure of possible burden for patient, and to detect the risk of automatic responses. Neither of these hypothesis was confirmed, and differences in time to completion seemed to depend on health conditions. Focus groups with patients confirmed their appreciation for this “new” active role in a telemonitoring system.

    Conclusions: The main and innovative aspect of our proposal is the use of a Web questionnaire to virtually recreate a checkup visit, integrating subjective (patient’s information) with objective data (biometric information). Our results, although preliminary and if need of further study, appear promising in proposing more effective telemedicine systems. Survey methodology could have an effective role in this growing field of research and applications.

    JMIR Res Protoc 2016;5(2):e101

    doi:10.2196/resprot.5187

    Keywords

    Web questionnaire; Web health monitoring survey; telemedicine; virtual checkup; survey quality; quality indicators; paradata 


    Survey methodology and medical research are becoming more connected: statisticians have always cooperated with clinicians in the analysis of collected data, and in recent years new research questions have arisen, along with new research fields (eg, health technology assessment and health economics). The aging European population and the interest in a healthy population in western countries have contributed to the increase in the number of health surveys, such as the English Longitudinal Study of Ageing (ELSA, since 2002) [1], the Survey of Health, Ageing and Retirement in Europe (SHARE, individuals from 20 European countries aged ≥50 years) [2], and the Health and Retirement Study (HRS) since 1992 in United States [3], to cite only the most well-known and relevant for the number of countries involved or for the systematic nature of the survey.

    In all these surveys, as well as in more traditional clinical trials, the role of methodology in survey design, data collection, and data analysis is clear and recognized by the whole scientific community. The role of statisticians in clinical trials is now complemented by survey methodology: patients are often surveyed at the beginning and at the end of the trial to assess their well-being or other characteristics connected to the topic under investigation. Furthermore, patients are often requested to give feedback on their satisfaction with health structures, and, last but not least, health data management and analysis are now mandatory in health services assessment. Therefore, the contact points between medical and statistical research are numerous. Due to the complexity of research topics, the requested skills and knowledge are multidisciplinary and interdisciplinary: the role of researchers has been changing, and researchers now need to interconnect with each other, in order to achieve their research goals.

    In the same years, survey methodology has had to couple with the challenges deriving from data collection through information and communications technology, the movement from landline telephones to mobiles and smartphones, and the increasing spread of Wi-Fi connections: modes and techniques to collect responses from people is always changing [4-8]. Guidelines and standards to follow are constantly being updated in order to be able to cover ever-changing and dynamic situations and data gathering modes [9,10], and new metrics derived from macro- and microparadata are being proposed and tested [11-16]. At the same time, the evolution of information and communications technology is giving a strong push to telemonitoring, telemedicine, and home rehabilitation systems [17-21]: new systems are being tested, and are quickly evolving from static to wireless devices, and from obtrusive to less-obtrusive equipment, also making it possible to follow patients and collect their information on their activities and reactions potentially everywhere and continuously.

    In our opinion, there are new and common challenges for medicine and survey methodology. Which data are useful to be collected and analyzed? How? And when? What is the data’s value for the well-being of the patient, which is the ultimate aim of a telemonitoring system? What about the quality and usefulness of the data collected?

    In the vast scientific literature on telemedicine systems, we have found only a few reports of systems that use data and information coming directly from patients [22-27], although several systems, as for example Philips’ eCareCompanion and eCareCoordinator [28], are now moving on to collecting this information through tablets or smartphones [29,30]. Surely researchers are exploring the implications of how and when these subjective data are collected, their utility for effective monitoring, and, more relevant from our point of view, whether there is a quality control process for this subjective information and how it works, but results have not reached the publication stage.

    Our vision is that an effective telemonitoring system would be able to 1) collect objective and subjective data at the same time (biometric parameters and symptoms reported by the patients themselves), 2) control the quality of data collected, and 3) analyze data both at the individual and at an aggregated level.

    This ambitious goal can be achieved only by using (and merging) all the technologies and skills available from both survey methodology and health research. In this paper we focus on the contribution of subjective data, that is, data reported by the patients themselves, about themselves, in way that simulates a standard medical examination: this, in our opinion, constitutes the real innovation in the scheme above.

    From Acquisition of Vital Signs to an Integrated Telemedicine System

    It is quite obvious that remote monitoring and telemedicine systems apply only to patients with chronic conditions. For them, a remote monitoring system is extremely useful, as it is designed to store data collected directly at the patient’s home, with the goal of detecting as soon as possible any worsening of the patient’s health status.

    Patient-doctor communication seems to be neglected in home monitoring, and yet it is always present in medical checkups (and in their telephone communications). Data and information are always the objects of the patient-doctor communication in each (first or follow-up) medical visit, and yet it seems that telemedicine systems do not include this relevant source of information. Generally, patient-doctor communication is face-to-face. In the first contact with the patient, the physician starts with a clinical interview: this allows the physician to know how the patient feels and the symptoms that he or she reports, and it precedes the objective physical examination. Thereafter, the patient undergoes further examinations (electrocardiography, echocardiography, chest x-ray, etc, and laboratory tests according to the physician’s request). Only after all the data and information have been collected and read is the physician able to confirm the diagnosis and prescribe suitable therapy.

    We underline that—as physicians themselves state—listening to the patient’s account of his or her symptoms and feelings is key to acquiring all of the information about the patient’s health status. To have only biometric or laboratory data is not sufficient for making a correct diagnosis and consequent decisions, for taking actions, and for giving prescriptions. For patients with a chronic disease, there are recognized guidelines (standardized by each disease) that have to be followed in a medical checkup: guidelines recommend not only which laboratory and other tests to take, but also that the symptoms reported by the patients themselves be recorded. Last but not least, patients with chronic diseases are increasingly able to recognize and communicate correct and useful information about their conditions. From these considerations, it follows that a remote monitoring system must have the aim of daily collecting vital signs and biometric data, as well as subjective data, from the patient.

    Information and communications technology tools and survey methodology allow for collection of these types of data according to the ideal standards: the questions (asked by a physician) and the responses (given by the patient) can be easily recognized as simulating a questionnaire, whose content and frequency of administration are determined by physicians.

    The Role of Survey Methodology in Health Monitoring

    The medical checkup can be transformed into a computer-assisted Web interview, and the Web-collected data can be used to generate a longitudinal survey. Subjective data can be analyzed together with objective data in order to have an assessment of the patient’s health status or to detect potential risks.

    In chronic diseases, the checkup visit can be more easily transformed into a structured questionnaire, because the clinical interview focuses mainly on presenting symptoms and their variations since the last visit. It is possible to acquire and store on a daily basis (or with a different periodicity; in any case, more frequently than the face-to-face checkup visits) all responses and information given by the patient. More important, it is possible to analyze both clinical and patient variables (measured at the same time) along appropriate longitudinal statistical models.

    The aging of the population in western countries calls for the unavoidable and necessary use of a system to automatically assess the health status of patients with chronic conditions, supporting and complementing face-to-face clinic activities, with the aim of focusing the efforts of medical staff on the most serious cases. In order to reliably automate prediction of the state of health of a chronically ill patient, it is also necessary to ensure that data collection is at the highest possible level of reliability and that the data are validated according to metrics established and accepted by the scientific community.

    To sum up, in a telemedicine system, questionnaires can be administered (almost) daily for a group of patients for an extended length of time. Are the questionnaires collected in a telemedicine system similar to a longitudinal survey? If we consider that a survey “is a systematic method for gathering information from (a sample of) entities for the purposes of constructing quantitative descriptors of the attributes of the larger population of which the entities are members” [5], then the answer is indeed positive, and we name it a Web health monitoring survey (WHMS).

    Quality Measures of a WHMS

    If “the quality of a survey is best judged not by its size, scope, or prominence, but by how much attention is given to [preventing, measuring, and] dealing with the many important problems that can arise” [31], then it is relevant to compare the American Association of Public Opinion Research’s (AAPOR) criteria to “produce a quality survey” with the particular situation of a WHMS (Table 1) [31].

    Table 1. Comparison between the American Association of Public Opinion Research’s (AAPOR) criteria [31] for a general survey and the features of a Web health monitoring survey (WHMS).

    		AAPOR survey criteriaFeatures of a WHMS
    Points better addressed by a WHMSHave specific goals.Have specific goals.

    		Consider alternatives.Consider alternatives.

    		Take great care in matching question wording to the concepts being measured and the population studied.Easier than in other fields of research. More attention to question wording in different languages or contexts.
     Maximize cooperation or response rates within the limits of ethical treatment of human subjects.Patients are strictly involved and their active role in the monitoring system can stimulate a high participation rate, a high response rate, and a low item nonresponse rate. The patient AND the patient’s relatives or caregivers can have more positive feelings in participating in the survey.
    Points easily respected by a WHMSSelect samples that well represent the population to be studiedThe sample is not probabilistic, rather it resembles an opt-in one.

    		Use designs that balance costs with errors.Use designs that balance costs with errors.

    		Pretest questionnaires and procedures.Pretest questionnaires and procedures.

    		Train interviewers carefully on interviewing techniques and the subject matter of the survey.Train and supervise patients (and doctors).

    		Use appropriate statistical analytic and reporting techniques.Use appropriate statistical analytic and reporting techniques.

    		Develop and fulfill pledges of confidentiality given to respondents.Develop and fulfill pledges of confidentiality given to respondents.
     Disclose all methods of the survey to allow for evaluation and replication.Disclose all methods of the survey to allow for evaluation and replication.
    Point to be studied for a WHMSCheck quality at each stage.New indicators and metrics are required.

    The main contribution that survey methodologists can make to WHMS is related to the quality of the survey at each stage.

    As a first contribution, we offer some indicators, calculated on data collected during the trial of an integrated telemedicine system (Assistenza domiciliare allo SCOmpenso cardiaco attraverso Le Tecniche Avanzate di comunicazione digitale, or ASCOLTA, the Italian word for listen) [32] conducted in 2011 in Pisa, Italy, involving 12 patients with a diagnosis of heart failure. We collected both biometric and subjective data for a period of 2 months for 11 patients and for <1 month for 1 patient, who was the only one hospitalized for noncardiac problems and was thus not included in the study. Patients had to connect daily to a website, at the time most convenient for the patients themselves. A Web questionnaire—dynamically constructed with a different number of questions (and answers) according to the patient’s clinical condition—collected the patients’ health data, including additional physical data (weight and arterial pressures) measured and reported by the patients themselves. At the end of the questionnaire, the system asked the patient to wear wireless electrocardiograph, pulse oximetry oxygen saturation, and respiratory rate recorders for 5 minutes. The patients were free to ask for a new connection at their discretion.

    We have previously described and discussed the positive contribution of subjective data to detecting potential risks [32]: variables obtained during a virtual visit substantially contributed to assessment of clinical status (69% of correct classifications), similar to the traditional biometric variables (70%), as assessed by a random forest classification algorithm. The combined use of both variables led to a more correct classification of the patient’s health status (84%).

    The questionnaire (see Multimedia Appendix 1 for the English translation) was designed under cardiologists’ supervision, tested, and transformed into a Web questionnaire. The wording of questions and answers and their sequence were in accordance with European Society of Cardiology guidelines [33] for heart failure patients and with the cardiologists’ clinical experience.

    We also collected paradata during the study period, along with instrumental and subjective data.

    During the 2-month trial, we collected a total of 478 records from 11 patients, 243 for the first month and 235 for the second month.

    As described before, a WHMS is a longitudinal survey that is quite different from a panel survey in terms of duration and contents; therefore, we needed more specific indicators than the well-known and consolidated indicators available in the literature [12,34,35].

    We therefore propose the use of indicators that measure the response and attrition rates at the patient level. The item nonresponse rate is calculated as a proxy of the patient’s cooperation. To measure whether there was any habituation to a daily questionnaire, we compared the time span to complete the questionnaire between the two halves of the entire study period. Finally, we measured the consistency of the time chosen to complete the questionnaire as a proxy of the patient’s compliance and satisfaction. Focus groups with patients were organized and conducted both at the beginning and at the end of the trial.


    Patient Adherence and Response Rate

    As expected, patients were very cooperative due to an effective strategy to encourage patient involvement. The number of missing questionnaires was very low. Figure 1 illustrates the response rate by each patient.

    ‎
    Figure 1. Response rate, by patient, during the 2-month ASCOLTA study.
    View this figure

    Item Nonresponse

    We measured the rate of nonresponse to each item, obtaining good results (Table 2). The 3 biometric variables that the patients were asked to report in the questionnaire all had a nonresponse rate of 0%, which we interpret as denoting a cooperative attitude among all of the patients: during the final focus group the patients expressed positive reactions for their active role.

    Table 2. Rate of nonresponse to questionnaire items (N=514).
    ItemsNonresponses
    n%
    Overall feeling20.39
    Weight relative to previous day316.03
    Swollen legs50.97
    Shortness of breath yesterday40.78
    Shortness of breath when combing285.45
    Shortness of breath when washing346.61
    Shortness of breath when getting dressed387.39
    Shortness of breath when tying shoes316.03
    Shortness of breath when climbing steps336.42
    Palpitations20.39
    Chest tightness20.39
    Tiredness20.39
    Maximum blood pressure00
    Minimum blood pressure00
    Weight00

    To measure habituation to the questionnaire, we compared the item nonresponses between the first and the second month. We found no significant differences (Figure 2).

    ‎
    Figure 2. Comparison of nonresponses to questionnaire items by period of the study (period 1: first month; period 2: second month) and by question.
    View this figure

    Time to Complete the Questionnaire

    From paradata collected, we calculated the time spent to complete the questionnaires both for the daily and the discretionary questionnaires (Table 3). The time required seemed to depend on the patients’ health conditions; in general, time required was very short and acceptable, and at our advice was not perceived as a burden by the patient.

    Table 3. Time spent to complete the daily and discretionary questionnaires, by patient.
    PatientDailyDiscretionaryTotal
    nMean time (min.s)nMean time (min.s)nMean time (min.s)
    CLMG731.0718.00741.12
    CLTS551.0800551.08
    CNCV201.3614.00211.43
    CNID851.4611.00861.45
    CNNV241.0311.00251.02
    DNTC223.4932.20253.38
    FRSG271.2711.00281.26
    LCUL461.4600461.46
    MCLN571.0911.00581.09
    PCCL501.3451.24551.33
    PSQN191.1621.30211.17
    Total4781.31162.044941.32

    We also tested for difference in completion time between the first and second period (Figure 3 and Table 4). Although the difference was statistically significant for 3 patients, the overall difference was not.

    Table 4. Time (min.s) spent to complete the daily questionnaires: comparison between first and second period of the study, by patient.
    PatientPeriodt testP valuedf

    		First halfSecond half


    CLMG1.091.050.71.4871
    CLTS1.161.022.10.04a53
    CNCV1.002.20–2.47.02a18
    CNID1.371.56–0.51.6183
    CNNV1.051.000.92.3722
    DNTC2.046.53–1.71.1020
    FRSG1.471.051.18.2525
    LCUL1.372.10–1.10.2844
    MCLN1.081.14–0.49.6355
    PCCL1.301.41–0.49.6348
    PSQN1.301.052.13.05a17
    Total1.251.37–1.06.29476

    aSignificant difference (P ≤.05).

    ‎
    Figure 3. Time spent to complete the questionnaires, by patient and by period of the study (period 1: first month; period 2: second month).
    View this figure

    Patients’ Chosen Time of Day to Complete the Questionnaire

    To accommodate the patient’s lifestyle, each one could choose the hour of the day when they would complete the questionnaire, with an interval of ±1 hour (Table 5).

    During the final focus group, the patients expressed their appreciation for this support, and therefore their behavior is a proxy of a cooperative attitude.

    Table 5. Time of day chosen to complete the questionnaire, by patient.
    PatientTotal no. of daily questionnaires completedMedian time at start of questionnaire
    (h.min.s)    		Mean time at start of questionnaire
    (h.min.s)    		Mean difference from medianMean difference from mean
    CLMG738.05.008.13.3918.2717.54
    CLTS5514.42.0014.42.1116.1315.46
    CNCV2015.23.0015.33.1413.5315.51
    CNID8517.48.3017.45.5314.2814.42
    CNNV2419.24.3019.14.2824.3326.3
    DNTC229.45.009.55.5522.5525.3
    FRSG279.29.309.25.5618.1318.11
    LCUL468.20.008.26.2922.2622.12
    MCLN579.20.009.30.4717.0219.25
    PCCL507.56.308.36.2811.5131.42
    PSQN1920.07.0019.58.1120.0919.32

    The mean difference was about 20 minutes, both from the median and from the mean.


    It is worth noting that our WHMS is part of a telemedicine system, making its goal different from that of other health surveys such as a Web-based daily questionnaire for health [26]. Telemedicine data are collected and must be analyzed at the patient level, because the main goal is to assess the health status of the single patient. On the contrary, in epidemiological studies the focus is on measuring health parameters for the entire population.

    Moreover, a WHMS is designed for patients with chronic conditions, and therefore they may already be greatly involved in the survey or their cooperative attitude can be reinforced: no incentive is needed, because the implicit incentive for each patient is to get better attention from the medical staff and to contribute actively to the management of their chronic disease. As a consequence, all indicators based on response rate (although—or perhaps because—they have values never obtained in a general survey) seem less relevant to measure the quality of the WMHS. Nonetheless, their trend over time for longer study is worth computing and controlling.

    WHMS is a longitudinal survey with very strict periodicity, even daily, which could be a burden for the respondent [36], resulting in a possible high attrition rate. The attrition rate at the patient level is therefore relevant in monitoring the quality of data collection. Our results demonstrate that this risk is very low, taking into account that only a few minutes are required to complete the questionnaire.

    Even with a diligent patient, there is the risk that patients’ responses will become automatic. The results we obtained led us to reject the hypothesis of the presence of automatic responses; therefore, the patient’s responses are deemed to be reliable.

    Signing in to the daily questionnaire and the patients’ choice of when during the day to sign in seem a good measure of their cooperation.

    Focus groups with patients confirmed the efficacy of the questionnaire and the positive reactions of patients to the new mode of collecting this information. Patients appreciated their new active role in the ASCOLTA system.

    Study Limitations

    We acknowledge that our study has some critical limitations, such as the small number of patients studied and the short duration of the longitudinal survey. Furthermore, the study involved only patients with heart failure. Other paradata can be collected, such as those needed to measure response latency or time-to-click [16,37].

    In more general terms, the digital skills of potential respondents to a WHMS is a challenge: a sample of patients with chronic conditions would be older, and the greater digital divide in this group could be a barrier to their participation in such a survey [38-40].

    Many aspects of a WHMS deserve to be developed, by applying the same approach to other kinds of chronic pathologies (eg, chronic obstructive pulmonary disease, diabetes, and hypertension) [41,42]. Moreover, the expansion of wireless technologies on platforms such as smartphones and tablets will enable the collection of biometric data to extend even to people who are not yet ill (eg, people at risk) or to healthy people who participate in amateur sports. In addition, although this is an open field, the reactions of medical staff to the virtual visit are worth investigating, as patient satisfaction is further explored [43-47].

    Information and communications technology solutions can be applied to facilitate and enable patient-doctor communication [48-53]. Many virtual medical visit applications are being developed [54-56] and the increasing number of websites testifies to patients’ willingness to play a more active role in their own health management [57,58].

    The main and innovative aspect of our proposal is the use of a Web questionnaire to virtually recreate a checkup visit, integrating subjective (patient information) with objective data (biometric information) in a unique database that can be analyzed with appropriate statistical methods. We suggest here some indicators to control the process of data collection, along with criteria already established by survey methodology, but we are conscious that other specific metrics need to be studied.

    Our results, although preliminary, appear promising and, in our opinion, could be of significance to the ongoing debate about the most appropriate type of telemonitoring and remote care of patients. Further studies, in a larger population, could be useful to ultimately confirm our preliminary results and to provide a more efficient cost-benefit ratio, according to health technology assessment.

    Survey methodology could have an effective role in this growing field of research and applications.

    In agreement with the Couper’s statement that “We constantly need to hone our skills, update our knowledge, and expose ourselves to new developments in other disciplines and fields of research and application.” [59], we think that monitoring of health data is a challenge for survey methodologists. Skills and competencies of survey methodologists and statisticians must guide data collection and data analysis in health monitoring surveys.

    Acknowledgments

    ASCOLTA Project was funded by the Regione Toscana (BANDO UNICO R&S, anno 2008 POR CREO Attività 1.6 Linea A) to a pool of firms (CAEN SpA - leader, Evidence Srl, EMAC Srl, Develer Srl, Sinergest Srl, Laboratori ARCHA Srl). Sant’Anna School of Advanced Studies (Institute of Economics, Institute of Life Sciences, and Institute of Communication, Information, and Perception Technologies) and the Department Cardiac Thoracic and Vascular Department (University of Pisa) gave scientific support. The project was presented in April 2009, approved in April 2010, and concluded in March 2012.

    The study protocol was approved by the Ethical Committee of the Hospital of Pisa (Study No. 3461/2011, presented November 29, 2011, approved February 9, 2012).

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Questionnaires used (English translation).

    PDF File (Adobe PDF File), 21KB
    1. ELSA English Longitudinal Study of Ageing: Insight Into a Maturing Population. London, UK: The Institute for Fiscal Studies; 2011.   URL: http://www.elsa-project.ac.uk/ [accessed 2015-09-16] [WebCite Cache]
    2. Survey of Health Ageing Retirement. Munich, Germany: SHARE-ERIC; 2013.   URL: http://www.share-project.org/ [accessed 2015-09-16] [WebCite Cache]
    3. Health and Retirement Study. Ann Arbor, MI: Institute for Social Research, University of Michigan; 2015.   URL: http://hrsonline.isr.umich.edu/; [accessed 2015-09-16] [WebCite Cache]
    4. Conrad FG, Schober MF, editors. Envisioning the Survey Interview of the Future. Hoboken, NJ: Wiley-Interscience; 2008.
    5. Groves R, Fowler FJ, Couper M, Lepkowski J, Singer E, Tourangeau R. Survey Methodology. New York, NY: Wiley; 2009.
    6. Couper MP. The future of modes of data collection. Public Opinion Q 2011 Dec 15;75(5):889-908. [CrossRef]
    7. Callegaro M, Baker R, Bethlehem J, Goritz A, Krosnick J, Lavrakas P, editors. Online Panel Research: A Data Quality Perspective. Chichester, UK: Wiley; 2014:A.
    8. Callegaro M, Lozar MK, Vehovar V, editors. Web Survey Methodology. Los Angeles, CA: Sage; 2015.
    9. Link MW, Murphy J, Schober MF, Buskirk TD, Hunter Childs J, Langer Tesfaye C. Mobile technologies for conducting, augmenting and potentially replacing surveys: executive summary of the AAPOR Task Force on Emerging Technologies in Public Opinion Research. Public Opinion Q 2014 Nov 25;78(4):779-787. [CrossRef]
    10. AAPOR. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys 8th edition.   URL: http://www.aapor.org/AAPORKentico/Communications/AAPOR-Journals/Standard-Definitions.aspx [accessed 2015-09-30] [WebCite Cache]
    11. Heerwegh D. Explaining response latencies and changing answers using client-side paradata from a Web survey. Soc Sci Comput Rev 2003 Aug 01;21(3):360-373. [CrossRef]
    12. Callegaro M, DiSogra C. Computing response metrics for online panels. Public Opinion Q 2009 Jan 29;72(5):1008-1032. [CrossRef]
    13. Kreuter F, editor. Improving Surveys With Paradata: Analytic Uses of Process Information. Hoboken, NJ: Wiley & Sons; 2013.
    14. Schouten B, Shlomo N, Skinner C. Indicators for monitoring and improving representativeness of responses. J Off Statistics. 2011. (2) p. 231-253   URL: http://www.jos.nu/Articles/abstract.asp?article=272231 [WebCite Cache]
    15. Sirkis R, Jans M. Using statistical process control to understand variation in computer assisted personal interviewing data. 2011 Presented at: Joint Statistical Meetings; July 30-August 4, 2011; Miami Beach, FL p. 477-489   URL: http://www.amstat.org/sections/srms/proceedings/ [WebCite Cache]
    16. Couper MP, Kreuter F. Using paradata to explore item level response times in surveys. J Royal Stat Soc Ser A (Stat Soc) 2013;176:271-286. [CrossRef]
    17. Nangalia V, Prytherch DR, Smith GB. Health technology assessment review: remote monitoring of vital signs--current status and future challenges. Crit Care 2010;14(5):233 [FREE Full text] [CrossRef] [Medline]
    18. Anker D, Koehler F, Abraham WT. Telemedicine and remote management of patients with heart failure. Lancet 2011 Aug 20;378(9792):731-739. [CrossRef] [Medline]
    19. Chaudhry SI, Mattera JA, Curtis JP, Spertus JA, Herrin J, Lin Z, et al. Telemonitoring in patients with heart failure. N Engl J Med 2010;363(24):2301-2309. [CrossRef]
    20. Desai AS. Home monitoring heart failure care does not improve patient outcomes: looking beyond telephone-based disease management. Circulation 2012 Feb 14;125(6):828-836 [FREE Full text] [CrossRef] [Medline]
    21. Bashshur RL, Shannon GW, Smith BR, Alverson DC, Antoniotti N, Barsan WG, et al. The empirical foundations of telemedicine interventions for chronic disease management. Telemed J E Health 2014 Sep;20(9):769-800 [FREE Full text] [CrossRef] [Medline]
    22. Little M, Wicks P, Vaughan T, Pentland A. Quantifying short-term dynamics of Parkinson's disease using self-reported symptom data from an Internet social network. J Med Internet Res 2013;15(1):e20 [FREE Full text] [CrossRef] [Medline]
    23. Bellos C, Papadopoulos A, Rosso R, Fotiadis D. Identification of COPD patients' health status using an intelligent system in the CHRONIOUS wearable platform. IEEE J Biomed Health Inform 2014 May;18(3):731-738. [CrossRef] [Medline]
    24. Bellos C, Papadopoulos A, Rosso R, Fotiadis D. CHRONIOUS: clinical validation of the CHRONIOUS wearable system in patients with chronic disease. 2013 Presented at: 35th Annual International Conference of the IEEE EMBS; July 3-7, 2013; Osaka, Japan p. 7084-7087.
    25. Steele GC, Khan AI, Kuluski K, McKillop I, Sharpe S, Bierman AS, et al. Improving patient experience and primary care quality for patients with complex chronic disease using the electronic patient-reported outcomes tool: adopting qualitative methods into a user-centered design approach. JMIR Res Protoc 2016 Feb;5(1):e28 [FREE Full text] [CrossRef] [Medline]
    26. Sugiura H, Ohkusa Y, Akahane M, Sano T, Okabe N, Imamura T. Development of a web-based survey for monitoring daily health and its application in an epidemiological survey. J Med Internet Res 2011;13(3):e66 [FREE Full text] [CrossRef] [Medline]
    27. Dorsch MP, Farris KB, Bleske BE, Koelling TM. A web application for self-monitoring improves symptoms in chronic systolic heart failure. Telemed J E Health 2015 Apr;21(4):267-270. [CrossRef] [Medline]
    28. Open A World of Cloud-Based Collaborative Care. Amsterdam, Netherlands: Koninklijke Philips N.V; 2015.   URL: http://www.usa.philips.com/healthcare-innovation/about-health-suite/applications-health-suite [accessed 2015-09-16] [WebCite Cache]
    29. Lygidakis C, Wallace P, Tersar C, Marcatto F, Ferrante D, Della VR, et al. Download your doctor: implementation of a digitally mediated personal physician presence to enhance patient engagement with a health-promoting Internet application. JMIR Res Protoc 2016 Mar;5(1):e36 [FREE Full text] [CrossRef] [Medline]
    30. McLean G, Murray E, Band R, Saunderson K, Hanlon P, Little P, et al. Digital interventions to promote self-management in adults with hypertension: protocol for systematic review and meta-analysis. JMIR Res Protoc 2015 Nov;4(4):e133 [FREE Full text] [CrossRef] [Medline]
    31. American Association for Public Opinion Research. Best Practices for Research. Oakbrook Terrace, IL: AAPOR; 2016.   URL: http://www.aapor.org/Standards-Ethics/Best-Practices.aspx [accessed 2016-05-18] [WebCite Cache]
    32. Romano MF, Sardella MV, Alboni F, L'Abbate A, Mariotti R, Di Bello V. The informative contribution of the “virtual medical visit” in a new heart failure telemedicine integrated system. Telemed J E Health 2014 Jun;20(6):508-521. [CrossRef] [Medline]
    33. McMurray JJ, Adamopoulos S, Anker SD, Auricchio A, Böhm M, Dickstein K, ESC Committee for Practice Guidelines. ESC Guidelines for the diagnosis and treatment of acute and chronic heart failure 2012: The Task Force for the Diagnosis and Treatment of Acute and Chronic Heart Failure 2012 of the European Society of Cardiology. Developed in collaboration with the Heart Failure Association (HFA) of the ESC. Eur Heart J 2012 Jul;33(14):1787-1847 [FREE Full text] [CrossRef] [Medline]
    34. Eysenbach G. Improving the quality of Web surveys: the Checklist for Reporting Results of Internet E-Surveys (CHERRIES). J Med Internet Res 2004 Sep 29;6(3):e34 [FREE Full text] [CrossRef] [Medline]
    35. Eysenbach G. The law of attrition. J Med Internet Res 2005;7(1):e11 [FREE Full text] [CrossRef] [Medline]
    36. Axinn WG, Jennings EA, Couper MP. Response of sensitive behaviors to frequent measurement. Soc Sci Res 2015 Jan;49:1-15 [FREE Full text] [CrossRef] [Medline]
    37. Callegaro M. Using paradata to improve the quality of web surveys: some examples and applications. London, UK: Royal Statistical Society; 2014 Presented at: Conference on Paradata From surveys research to practice; 2014; London, UK   URL: http:/​/www.​southampton.ac.uk/​s3ri/​news/​events/​2014/​06/​26_paradata_from_survey_research_to_practice.​page [WebCite Cache]
    38. Gracia E, Herrero J. Internet use and self-rated health among older people: a national survey. J Med Internet Res 2009 Dec;11(4):e49 [FREE Full text] [CrossRef] [Medline]
    39. Romano MF, Sardella MV, Alboni F, Russo L, Mariotti R, Nicastro I, et al. Is the digital divide an obstacle to e-health? An analysis of the situation in Europe and in Italy. Telemed J E Health 2015 Jan;21(1):24-35. [CrossRef] [Medline]
    40. Jones J, Cassie S, Thompson M, Atherton I, Leslie SJ. Delivering healthcare information via the internet: cardiac patients' access, usage, perceptions of usefulness, and web site content preferences. Telemed J E Health 2014;20(3):223-228. [CrossRef] [Medline]
    41. Wang J, Wang Y, Wei C, Yao NA, Yuan A, Shan Y, et al. Smartphone interventions for long-term health management of chronic diseases: an integrative review. Telemed J E Health 2014 Jun;20(6):570-583. [CrossRef] [Medline]
    42. Bellos C, Papadopoulos A, Rosso R, Fotiadis D. CHRONIOUS: A wearable platform for monitoringmanagement of patients with chronic disease. 2011 Presented at: 33rd Annual International Conference of the IEEE EMBS; August 30-September 3, 2011; Boston, MA p. 864-867.
    43. Bostock Y, Hanley J, McGown D, Pinnock H, Padfield P, McKinstry B. The acceptability to patients and professionals of remote blood pressure monitoring using mobile phones. Primary Health Care 2009 Jul 28;10(4):299-308. [CrossRef]
    44. Cafazzo JA, Leonard K, Easty AC, Rossos PG, Chan CT. Patient perceptions of remote monitoring for nocturnal home hemodialysis. Hemodial Int 2010 Oct;14(4):471-477. [CrossRef] [Medline]
    45. Matlock D. Big Brother is watching you: what do patients think about ICD home monitoring? Circulation 2010;122(4):319-321. [CrossRef]
    46. Or CKL, Karsh B, Severtson DJ, Burke LJ, Brown RL, Brennan PF. Factors affecting home care patients' acceptance of a web-based interactive self-management technology. J Am Med Inform Assoc 2011 Jan;18(1):51-59 [FREE Full text] [CrossRef] [Medline]
    47. McBride DL, Levasseur SA, Li D. Development and validation of a web-based survey on the use of personal communication devices by hospital registered nurses: pilot study. JMIR Res Protoc 2013;2(2):e50 [FREE Full text] [CrossRef] [Medline]
    48. Alkhaldi G, Hamilton FL, Lau R, Webster R, Michie S, Murray E. The effectiveness of technology-based strategies to promote engagement with digital interventions: a systematic review protocol. JMIR Res Protoc 2015;4(2):e47 [FREE Full text] [CrossRef] [Medline]
    49. Antonelli D, Bellomo D, Bruno G, Villa A. Evaluating collaboration effectiveness of patient-to-doctor interaction in a healthcare territorial network. In: Camarinha-Matos LM, Xu L, Afsarmanesh H, editors. Collaborative Networks in the Internet of Services. Berlin, Germany: Springer; 2012:128-136.
    50. Boulos MNK, Wheeler S, Tavares C, Jones R. How smartphones are changing the face of mobile and participatory healthcare: an overview, with example from eCAALYX. Biomed Eng Online 2011;10:24 [FREE Full text] [CrossRef] [Medline]
    51. Lindberg B, Nilsson C, Zotterman D, Söderberg S, Skär L. Using information and communication technology in home care for communication between patients, family members, and healthcare professionals: a systematic review. Int J Telemed Appl 2013;2013:461829 [FREE Full text] [CrossRef] [Medline]
    52. Bert F, Giacometti M, Gualano MR, Siliquini R. Smartphones and health promotion: a review of the evidence. J Med Syst 2014 Jan;38(1):9995. [CrossRef] [Medline]
    53. Mattsson S, Olsson EMG, Alfonsson S, Johansson B, Carlsson M. Measuring use of health-related support on the Internet: development of the Health Online Support Questionnaire (HOSQ). J Med Internet Res 2015 Nov;17(11):e266 [FREE Full text] [CrossRef] [Medline]
    54. Mahler SA, Burke GL, Duncan PW, Case LD, Herrington DM, Riley RF, et al. HEART pathway accelerated diagnostic protocol implementation: prospective pre-post interrupted time series design and methods. JMIR Res Protoc 2016 Jan;5(1):e10 [FREE Full text] [CrossRef] [Medline]
    55. Eissens van der Laan MR, van Offenbeek MAG, Broekhuis H, Slaets JPJ. A person-centred segmentation study in elderly care: towards efficient demand-driven care. Soc Sci Med 2014 Jul;113:68-76. [CrossRef] [Medline]
    56. Dixon RF, Rao L. Asynchronous virtual visits for the follow-up of chronic conditions. Telemed J E Health 2014 Jul;20(7):669-672. [CrossRef] [Medline]
    57. Society for Participatory Medicine. e-patients.net. 2015.   URL: http://e-patients.net/ [accessed 2015-09-28] [WebCite Cache]
    58. PatientsLikeMe. 2015.   URL: https://www.patientslikeme.com/ [accessed 2016-05-18] [WebCite Cache]
    59. Couper M. Is the sky falling? New technology, changing media, and the future of surveys. Survey Res Methods 2013;7(3):145-156. [CrossRef]

    ‎
    AAPOR: American Association of Public Opinion Research
    ASCOLTA: Assistenza domiciliare allo SCOmpenso cardiaco attraverso Le Tecniche Avanzate di comunicazione digitale
    ELSA: English Longitudinal Study of Ageing
    HRS: Health and Retirement Study
    SHARE: Survey of Health, Ageing and Retirement in Europe
    WHMS: Web health monitoring survey

    Edited by G Eysenbach; submitted 05.10.15; peer-reviewed by E Afari-kumah, P Giacomelli, W Glinkowski; comments to author 29.03.16; revised version received 25.04.16; accepted 13.05.16; published 06.06.16

    Copyright

    ©Maria Francesca Romano, Maria Vittoria Sardella, Fabrizio Alboni. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 06.06.2016.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.