This paper is in the following e-collection/theme issue:

Early Reports (28) e-Mental Health and Cyberpsychology (1386) mHealth for Data Collection and Research (907) Depression and Mood Disorders; Suicide Prevention (1271) Ecological Momentary Assessment (EMA) (294)

Published on in Vol 5, No 3 (2016): Jul-Sept

Predicting Negative Emotions Based on Mobile Phone Usage Patterns: An Exploratory Study

Predicting Negative Emotions Based on Mobile Phone Usage Patterns: An Exploratory Study

Predicting Negative Emotions Based on Mobile Phone Usage Patterns: An Exploratory Study

Authors of this article:

Galen Chin-Lun Hung1, 2 Author Orcid Image ;   Pei-Ching Yang3 Author Orcid Image ;   Chia-Chi Chang3 Author Orcid Image ;   Jung-Hsien Chiang3 Author Orcid Image ;   Ying-Yeh Chen1, 4 Author Orcid Image
Article Authors Cited by (32) Tweetations (7) Metrics

    Original Paper

    1Department of General Psychiatry, Taipei City Psychiatric Center, Taipei City Hospital, Taipei, Taiwan

    2Department of Public Health, School of Medicine, National Yang-Ming University, Taipei, Taiwan

    3Department of Computer Science and Information Engineering, National Cheng Kung University, Tainan, Taiwan

    4Institute of Public Health and Department of Public Health, National Yang-Ming University, Taipei, Taiwan

    Corresponding Author:

    Galen Chin-Lun Hung, MD, ScM

    Department of General Psychiatry

    Taipei City Psychiatric Center

    Taipei City Hospital

    No. 309,

    Songde Rd., Xinyi Dst.,

    Taipei, 106

    Taiwan

    Phone: 886 2 27263141 ext 1270

    Fax:886 2 27267246

    Email: galenhung@tpech.gov.tw


    Background: Prompt recognition and intervention of negative emotions is crucial for patients with depression. Mobile phones and mobile apps are suitable technologies that can be used to recognize negative emotions and intervene if necessary.

    Objective: Mobile phone usage patterns can be associated with concurrent emotional states. The objective of this study is to adapt machine-learning methods to analyze such patterns for the prediction of negative emotion.

    Methods: We developed an Android-based app to capture emotional states and mobile phone usage patterns, which included call logs (and use of apps). Visual analog scales (VASs) were used to report negative emotions in dimensions of depression, anxiety, and stress. In the system-training phase, participants were requested to tag their emotions for 14 consecutive days. Five feature-selection methods were used to determine individual usage patterns and four machine-learning methods were tested. Finally, rank product scoring was used to select the best combination to construct the prediction model. In the system evaluation phase, participants were then requested to verify the predicted negative emotions for at least 5 days.

    Results: Out of 40 enrolled healthy participants, we analyzed data from 28 participants, including 30% (9/28) women with a mean (SD) age of 29.2 (5.1) years with sufficient emotion tags. The combination of time slots of 2 hours, greedy forward selection, and Naïve Bayes method was chosen for the prediction model. We further validated the personalized models in 18 participants who performed at least 5 days of model evaluation. Overall, the predictive accuracy for negative emotions was 86.17%.

    Conclusion: We developed a system capable of predicting negative emotions based on mobile phone usage patterns. This system has potential for ecological momentary intervention (EMI) for depressive disorders by automatically recognizing negative emotions and providing people with preventive treatments before it escalates to clinical depression.

    JMIR Res Protoc 2016;5(3):e160

    doi:10.2196/resprot.5551

    Keywords

    mobile phone usage; depression; emotion; machine learning; affective computing 


    According to the World Health Organization, an estimated 350 million people are affected by depression worldwide. Depression is the leading cause of disability, and a major contributor to the global burden of disease, as measured by disability-adjusted life years [1]. Apart from pharmacological treatment and psychotherapy, self-management of negative emotions is of paramount importance because unprocessed negative emotions may escalate to clinical depression, and relapse of depression often results from an inadequate response to negative emotions [2]. In addition to those with clinical depression, the general public has developed an interest in recognizing their negative emotions, a major determinant of quality of life and adoption of health risk behaviors [3,4].

    Awareness of negative emotion is crucial because it allows individuals to promptly respond with cognitive and behavioral strategies, avoiding escalation to clinical depression [5]. The concepts of ecological momentary assessment (EMA) and ecological momentary intervention (EMI) are the basis for real-time monitoring and management of context-specific emotional states [6,7]. A common and effective way to improve self-awareness is via regular self-tracking [8]. Several questionnaires pertaining to symptoms of depression have been implemented to mobile devices to help individuals track their emotional states [9-12].

    In recent years, a novel and promising way for tracking emotion has emerged: via the mobile phone. The number of mobile phone users worldwide will surpass 2 billion in 2016 [13]. Mobile phone users spend almost an hour per day on their phones and check their phones on an average of 150 times daily [14]. Thus, mobile phones and mobile apps represent an opportunity to monitor and possibly intervene in mental health conditions [15,16]. In previous studies, mobile phones have shown potential for self-tracking of negative emotions [15,17,18].

    Automatic Detection of Emotion with Mobile Phone Sensors

    Affective computing aims to automatically detect emotions when they arise [19,20]. Prior studies have attempted to infer emotions with various sensors. For example, the collection of voice and speech signals [21-26] has been attempted, but it requires the use of a camera or microphone, which is more invasive and power-consuming. Tracing physiological correlates of emotions, such as heart rate variability [27] and electro-dermal activity [28] via portable sensors is another viable strategy, though these parameters are often interfered by determinants other than emotion itself, including physical exertion and environmental conditions (eg, temperature, humidity [29]). Information retrieved from Global Positioning System (GPS) signals (geospatial activity), multi-axial accelerometers (kinesthetic activity), and light/sound sensors (ambient features, sleep) is frequently used to differentiate emotional statuses [16,18,23-26,30]. Samsung Technology also conducted studies related to emotion detection [31,32], which focused on users’ physical signals including hand gestures performed on the touch screen of a mobile phone.

    Using Mobile Phone Usage Patterns to Predict Emotional State

    Behavioral patterns can either be an antecedent or a consequence of human emotion [33]. Mobile phone usage patterns are a traceable behavioral characteristic potentially associated with concordant emotional states [34]. Apps tracing mobile phone usage are generally lightweight and power efficient, without a need for computationally intensive or power-consuming data processing of video, audio or physiological signals [35]. Faurholt-Jepsen et al [24] demonstrated in patients with bipolar disorder possible correlations between number of calls and depressive symptoms. Saeb et al indicated that high phone usage was associated with depressive scores reported at baseline [30]. Burns et al [36] used mobile phone data, such as call logs, SMS (short message service) text messaging, and GPS to predict mood states, however, results were poor. Microsoft has developed the MoodScope system [37], which predicted users’ emotional state based on text messaging, emails, phone calls, application usage, Web browsing, and location and found that a personalized model with sufficient training period (ie, 2 months) would achieve a high predictive accuracy. However, few attempts have been made to compare the predictability of mobile phone usage data collected in different time frames (eg, 1 hour vs 2 hours prior to the reported emotion). In addition, a more sophisticated machine-learning algorithm was seldom applied.

    The present study intends to capture user’s self-reported negative emotions and mobile phone usage data, and analyze their association in order to predict negative emotion. The specific aims of this study are (1) to develop a mobile phone application for data collection; (2) to use machine-learning methods for system building; and (3) to prospectively evaluate the system predictability.


    Design

    We developed a mobile app (iHOPE) to capture self-reported negative emotions and automatically collect mobile phone usage patterns in the background.

    The study procedure was divided into the following three phases (Figure 1): (1) data collection, which involves collecting negative emotion reports and raw mobile phone usage data, (2) classifier training, which involves quantifying negative emotions, defining usage patterns, feature extraction and selection, and selecting adequate time slots, with each combination being defined as a classifier, and (3) detection classifier selection where we developed a detection classifier selection method based on rank product scoring to reduce the number of combinations. We then evaluated the performance of each classifier and chose the best-performing one.

    ‎
    Figure 1. Study procedures.
    View this figure

    Recruitment

    Participants were graduate students recruited from the Institute of Medical Informatics, National Cheng Kung University, and medical professionals at Taipei City Psychiatric Center. In this exploratory study, we intended to recruit medical professionals with regular mobile phone use, better emotional awareness, and protocol adherence to properly train and validate our system. An invitation letter for participating in the study was sent to potential participants via email. After enrollment, they were requested to download and use the app on their primary mobile phone and tag the momentary state of depression, stress, and anxiety 4 times daily, with an interval of at least 3 hours between 2 consecutive tags, for 14 consecutive days. A notification was sent to the participants each time they were requested to tag emotions. All data were uploaded to a cloud server to train their personalized classifier. At the evaluation phase, we sent the personalized classifier via email for installation and requested the participants to validate the predicted negative emotions for at least 5 days. The study procedure is depicted in Figure 2.

    ‎
    Figure 2. Study flow from recruitment to analysis.
    View this figure

    Data Collection

    Negative Emotions

    In collaboration with a psychiatrist at Taipei City Psychiatric Center (GCH), we built the app with visual analogue scales (VASs) to measure negative emotions in three dimensions: depression, stress, and anxiety (Figure 3). Each VAS contained a slider for the user to drag to a specific point indicating their current level of emotion. For depression, the lowest level indicated normal mood, whereas the highest level reflected the most depressed state. For anxiety and stress, the lowest level represented the usual state, whereas the highest level indicated the most anxious and stressful condition.

    ‎
    Figure 3. Visual analogue scale for anxiety.
    View this figure
    Mobile Phone Usage Patterns

    Our raw data contains call states and the package name of the app currently on the screen. We define the following three calling states: (1) idle state, the call function is not activated, (2) ringing state, which is a call waiting for the user to answer, with the phone either ringing or vibrating, and (3) off-hook state, that is, at least one call was dialing, active, or on hold. Our application monitored these calling states on the mobile phone every 3 seconds.

    The package name of the app is recorded. To avoid duplicate app names, we record the package name of the app, which is unique in the installed-app list on the mobile phone. If there are several apps in the operating system hash, we only consider the app that currently occupies the screen. App categories, top ten apps, and screen usage are then inferred from the app package name records. Call-in, call-out, and missed calls are inferred from the call states.

    Timeslot Selection

    We set a timeslot for every emotion tag, ranging from 0.5 hour, 1 hour, 1.5 hours, to 2 hours. In a given timeslot, the following mobile phone usage features were extracted: (1) count (the number of occurrences of the usage type within the timeslot), (2) total duration (the sum of the execution times), (3) average duration (the sum of the execution time divided by occurrence of usage), and (4) average interval (the average of the intervals between the start time of a usage and end time of previous usage of the same type). For example, if a time slot of 1 hour is selected, and we are determining the features of phone calls, we would extract the information regarding (during the hour preceding the emotional tag) the number of phone calls, the average and total durations of the phone calls, and the average duration of intervals between the two phone calls.

    Machine-Learning Methods Training

    We used four conventional classifiers for analyzing mobile phone usage patterns (1) the Naïve Bayes classifier (NB); (2) the C4.5 decision tree (C4.5); (3) the Naïve Bayes decision tree (NBT); and (4) the Support Vector Machine (SVM) [38,39]. The classifiers are widely used because they are fast, easy to implement, and explain. The SVM generally performs the best, but it requires more complex processing and computing. The feature sets used to train the classifiers and the feature selection are personalized for the distinct features of negative emotions expressed by each participant.

    Two types of 2-level t tests are applied for feature selection. The manner in which we grouped the sets for each level is illustrated in Figure 4. For the first 2-level t test selection, the features with P values less than .05 were chosen either in the first or second level. If less than five features were chosen, features with the smallest P values were selected. The t test was replaced by the homoscedastic t test for the other 2-level t test selection. This was different from the initial t test such that a smaller value represented greater similarity in the two testing sets. With the homoscedastic t test, all features were chosen in the beginning, and the features with P values less than .05 in the first or second levels of the test were deleted from the chosen set [40].

    The best first feature selections used greedy algorithms to find the best feature set for classification instead of choosing each feature independently. These selection procedures were thus more likely to find multi-feature usage than the 2-level t test. We applied sequential forward selection, sequential backward selection, and bi-directional selection to do the best first selection. A detection classifier selection method was developed based on rank product scoring, which considered the rank order rather than the accuracy of the classification result, so that the influence of outliers was minimized. The process for detection classifier selection is depicted in Figure 5. After the combination for each feature selection method was determined, we selected the best classifier from the four classifiers based on the average predictive accuracy for the three VASs.

    ‎
    Figure 4. Two-level feature selection for scales of negative emotions.
    View this figure
    ‎
    Figure 5. Detection classifier selection.
    View this figure

    Evaluation

    After the personalized classifiers were trained using data collected from the 2-week period, it was provided to the participants for an evaluation for at least 5 days. Using the identical VAS for reporting negative emotions, the classifier provided predictions for current state of depression, anxiety, and stress every 2 hours from 11:00 AM to 11:00 PM. A notification was sent to the participants about the prediction, and they were allowed to correct it by dragging the slider to the exact level of current emotion. We divided the range of each negative emotion to three categories (low, medium, high). The prediction failed if any of the corrected ratings were not in the same category. We then compared the performance of our classifier with that of two benchmark models: multiple linear regression and general guess method.


    Study Sample

    We recruited 40 healthy participants (30 graduate students and 10 medical professionals) who had contacted us to participate in the study. In the system-training phase, we used the data from 28 participants who had a minimum of 40 logs of negative emotions. Of them, 9 (32%, 9/28) were woman with a mean (SD) age of 29.2 (5.1) years. In the system-evaluation phase, we analyzed the data from 18 participants who had performed the evaluation for at least 5 days.

    Machine-Learning Methods Selection

    To select the appropriate classifier, we tested four classifiers. The combination of the NB classifier with 2-hour timeslots and greedy best forward feature selection had the highest accuracy for the depression and stress scales. Thus, this combination was chosen as the best classifier to detect negative emotion and was applied to evaluate the individual classifier (refer to Multimedia Appendix 1 for a complete list of the performance achieved by every combination).

    Evaluation Outcomes

    Among the 18 participants selected, they performed the evaluation for a mean (SD) of 10 (3) days. On average, an individual responded to 56 (20) predictions of their negative emotions. The results of the individual classifier evaluation are detailed in Table 1. In total, 1008 predictions were made and 995 of them (98.71 %, 995/1008) received feedback. Of those, 857 (86.1%, 857/995) indicated successful predictions, thus, the average rate of successful detections was 86.17%. Comparing with the predictive accuracy of multiple linear regressions (63%) and general guess method (77%), our classifier was substantially better in detecting negative emotions.

    Table 1. Result of the personalized classifier evaluation.
    User IDDuration of system
    evaluation, days    		Numbers of predictions
    respondeda    		Numbers of successful
    predictions    		Predictive accuracyb, %
    Subject 113756992.00
    Subject 210564783.93
    Subject 313776888.31
    Subject 49513874.51
    Subject 510585594.83
    Subject 65302893.33
    Subject 75302893.33
    Subject 85302376.67
    Subject 98484185.42
    Subject 108484593.75
    Subject 118484185.42
    Subject 128484287.50
    Subject 138484185.42
    Subject 148484083.33
    Subject 158483981.25
    Subject 1610605083.33
    Subject 1716967982.29
    Subject 1816968386.46
    Total10 (mean)99585786.17

    aThe personalized classifier routinely made 7 predictions daily, but participants may respond only to some but not all of them.

    bSuccessful predictions/responded predictions.


    Principal Findings

    We developed a mobile phone app capable of reporting negative emotions and collecting mobile phone usage patterns. The present study provides preliminary evidence that by adapting sophisticated machine-learning methods, it is possible to predict concurrent negative emotions via mobile phone usage patterns with substantial accuracy.

    Limitations

    The following limitations should be considered when interpreting the results of our pilot study. First, our participants were healthy professionals with moderate mobile phone usage and possibly better capability to define the degrees of negative emotions. A more extensive approach of validation would be recruiting a group of generally healthy controls, and another group with clinical depression. Second, mobile phone usage patterns is one but not the only indicator of underlying emotions. Other factors (eg, sleep, movement) are involved in manifesting the current emotional state and should be incorporated in future studies. Third, some participants either did not carry their mobile phones all the time or did not respond to emotion tags and predictions regularly, which interrupted data collection and prediction verification. Moreover, there were participants who used more than one mobile device. Therefore, data collected from a single mobile phone may not reflect the actual usage patterns. Lastly, users with higher suggestibility may be inclined to agree with the emotional state inferred. An alternative approach would be to ask users to report their momentary emotion before providing them with the prediction. Moreover, our system provided an individualized model for emotional prediction (ie, each participant had unique features of mobile phone usage patterns associated with her emotion). Due to the limited sample size, we were unable to identify any general patterns among the personalized features.

    Comparison with Previous Studies

    The predictive accuracy of our system (86.17%) is comparative to previous research using mobile phone usage patterns to infer human emotions, however, the training time required (2 weeks) is substantially shorter than other studies (3 weeks to 3 months) [11,20,37]. We have observed that participants' usage behaviors changed substantially over time. Data currently collected for emotion inference may be out of the range of data previously used to obtain the personalized classifier, leading to failed prediction. Consequently, the personal classifier may need to be retrained periodically to improve the predictive accuracy. Aimed at preventing clinical depression, our predictions involve three negative emotions (depression, anxiety, and stress), which frequently precede a full-blown depressive episode, while prior studies [35,37] often adapted the circumplex model of normal emotion (ie, pleasure and activeness) [41]. While many studies used simple regression to predict emotion, we applied a sophisticated machine-learning process in combination with timeslot and feature selections. Moreover, when performing system evaluations, instead of adapting the same dataset used to train the personalized classifier, we request the users to prospectively validate the predicted emotion.

    Implications and Future Work

    Though depressive disorders are becoming a global public health challenge, current treatment of depression has encountered considerable obstacles. A substantial portion of patients with depression either does not have access to standard treatments [1,42], primarily consisting of antidepressants and psychotherapy, or does not respond well to those treatments [43]. Outside of North America and Europe, there is a shortage of mental health professionals globally, and the stigma of mental disorders often prevents patients from actively seeking treatment [1]. Therefore, improving self-awareness and management of depression, or its antecedent negative emotion, can be a viable strategy for primary or secondary prevention. In addition, adopting the mobile phone interface can make it both accessible and scalable.

    Analyzing mobile phone usage behavior to infer negative emotions is a pragmatic approach because it is possibly the single most prevalent, continuous, and traceable behavioral characteristic in the modern era. Future work should combine mobile phone usage patterns and other indices available for continuous monitoring via mobile phone(eg, location, weather) to collectively infer the underlying emotion. Moreover, emotion is a temporary state, which follows a dynamic pattern over time. A time-series analysis should be employed to account for the temporal alterations of negative emotions. With automatic detection of negative emotions, EMI becomes possible so that cognitive or behavioral suggestions may be provided seamlessly, and therefore bypassing the necessity of users having to acknowledge their negative emotions before receiving support. We are currently collaborating with the Taipei City Psychiatric Center, and are recruiting patients with depressive disorder to examine the feasibility and usefulness of this system in the clinical setting. The study protocol has been approved by the institutional review board (TCHIRB-1030206).

    Conclusions

    In the present study, we developed a system capable of predicting negative emotions based on mobile phone usage patterns. The performance of our system appears to be superior to that of predictive models used in prior studies. The combination of timeslot selection, machine-learning process, and multiple feature selections may have substantially improved predictability. This system has a potential for ecological momentary intervention for health individuals or patients with depressive disorders by promptly recognizing negative emotion and providing them with preventive treatments before it escalates to clinical depression.

    Acknowledgments

    This study was supported by grants from Ministry of Science and Technology, R.O.C. (MOST 105-2314-B-532-001) and Department of Health, Taipei City Government (104-52, 53, 54). We are grateful to the dedication of staff at Taipei City Psychiatric Center, Taipei City Hospital, and to all the participants. We also would like to thank Delta Electronics, Inc. for sponsoring this study.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Comparison of performance achieved by different combinations of machine-learning methods.

    PDF File (Adobe PDF File), 63KB
    1. World Health Organization. Depression Fact Sheet. 2012.   URL: http://www.who.int/mediacentre/factsheets/fs369/en/ [accessed 2016-07-27] [WebCite Cache]
    2. Levens SM, Gotlib IH. Updating emotional content in recovered depressed individuals: evaluating deficits in emotion processing following a depressive episode. J Behav Ther Exp Psy 2015 Sep;48:156-163. [CrossRef] [Medline]
    3. Papakostas GI, Petersen T, Mahal Y, Mischoulon D, Nierenberg AA, Fava M. Quality of life assessments in major depressive disorder: a review of the literature. Gen Hosp Psychiatry 2004;26(1):13-17. [Medline]
    4. Verger P, Lions C, Ventelou B. Is depression associated with health risk-related behaviour clusters in adults? Eur J Pub Health 2009 Dec;19(6):618-624 [FREE Full text] [CrossRef] [Medline]
    5. Antoni MH, Lehman JM, Kilbourn KM, Boyers AE, Culver JL, Alferi SM, et al. Cognitive-behavioral stress management intervention decreases the prevalence of depression and enhances benefit finding among women under treatment for early-stage breast cancer. Health Psychol 2001 Jan;20(1):20-32. [Medline]
    6. Ebner-Priemer UW, Trull TJ. Ecological momentary assessment of mood disorders and mood dysregulation. Psychol Assess 2009 Dec;21(4):463-475. [CrossRef] [Medline]
    7. Heron KE, Smyth JM. Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. Br J Health Psychol 2010 Feb;15(Pt 1):1-39 [FREE Full text] [CrossRef] [Medline]
    8. Morris ME, Kathawala Q, Leen TK, Gorenstein EE, Guilak F, Labhard M, et al. Mobile therapy: case study evaluations of a cell phone application for emotional self-awareness. J Med Internet Res 2010;12(2):e10 [FREE Full text] [CrossRef] [Medline]
    9. Kramer I, Simons CJ, Hartmann JA, Menne-Lothmann C, Viechtbauer W, Peeters F, et al. A therapeutic application of the experience sampling method in the treatment of depression: a randomized controlled trial. World Psychiatry 2014 Feb;13(1):68-77 [FREE Full text] [CrossRef] [Medline]
    10. Depp CA, Mausbach B, Granholm E, Cardenas V, Ben-Zeev D, Patterson TL, et al. Mobile interventions for severe mental illness: design and preliminary data from three approaches. J Nerv Ment Dis 2010 Oct;198(10):715-721 [FREE Full text] [CrossRef] [Medline]
    11. Ben-Zeev D, Davis KE, Kaiser S, Krzsos I, Drake RE. Mobile technologies among people with serious mental illness: opportunities for future services. Adm Policy Ment Health 2013 Jul;40(4):340-343 [FREE Full text] [CrossRef] [Medline]
    12. Drake G, Csipke E, Wykes T. Assessing your mood online: acceptability and use of Moodscope. Psychol Med 2013 Jul;43(7):1455-1464 [FREE Full text] [CrossRef] [Medline]
    13. eMarketer. 2 Billion Consumers Worldwide to Get Smart(phones) by 2016. 2016.   URL: http://www.emarketer.com/Article/2-Billion-Consumers-Worldwide-Smartphones-by-2016/1011694 [accessed 2016-07-27] [WebCite Cache]
    14. Business Insider. How Much Time Do We Really Spend On Our Smartphones Every Day?. 2013.   URL: http://www.businessinsider.com.au/how-much-time-do-we-spend-on-smartphones-2013-6 [accessed 2016-07-27] [WebCite Cache]
    15. Torous J, Friedman R, Keshavan M. Smartphone ownership and interest in mobile applications to monitor symptoms of mental health conditions. JMIR Mhealth Uhealth 2014;2(1):e2 [FREE Full text] [CrossRef] [Medline]
    16. Osmani V, Gruenerbl A, Bahle G, Haring C, Lukowicz P, Mayora O. Smartphones in mental health: detecting depressive and manic episodes. IEEE Pervasive Comput 2015 Jul;14(3):10-13. [CrossRef]
    17. Church K, Hoggan E, Oliver N. A study of mobile mood awareness communication through MobiMood. 2010 Presented at: Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries; 16-20 October 2010; Reykjavik, Iceland.
    18. Saeb S, Zhang M, Kwasny MM, Karr CJ, Kording K, Mohr DC. The relationship between clinical, momentary, and sensor-based assessment of depression. Int Conf Pervasive Comput Technol Healthc 2015 Aug;2015 [FREE Full text] [CrossRef] [Medline]
    19. Picard RW. Affective medicine: technology with emotional intelligence. Stud Health Technol Inform 2002;80:69-83. [Medline]
    20. Nielek R, Wierzbicki A. Emotion aware mobile application. In: Computational Collective Intelligence. Technologies and Applications. Berlin: Springer Berlin-Heidelberg; 2010:122-131.
    21. Yang N, Samuel A. Context-rich detection of user’s emotions using a smartphone. Microsoft Research Internship Report 2011.
    22. Chang KH, Fisher D, Canny JF, Hartmann B. Context-rich detection of user’s emotions using asmartphone. 2011 Presented at: Proceedings of the 6th International Conference on Body Area Networks; 7-10 November 2011; Beijng, China. [CrossRef]
    23. Wang R, Chen F, Chen Z, Li T, Harari G, Tignor S, et al. StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones. 2014 Presented at: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing; 13-17 September 2014; Seattle, WA. [CrossRef]
    24. Faurholt-Jepsen M, Frost M, Vinberg M, Christensen EM, Bardram JE, Kessing LV. Smartphone data as objective measures of bipolar disorder symptoms. Psychiatry Res 2014 Jun 30;217(1-2):124-127. [CrossRef] [Medline]
    25. Ben-Zeev D, Scherer EA, Wang R, Xie H, Campbell AT. Next-generation psychiatric assessment: using smartphone sensors to monitor behavior and mental health. Psychiatr Rehabil J 2015 Sep;38(3):218-226 [FREE Full text] [CrossRef] [Medline]
    26. Grünerbl A, Muaremi A, Osmani V, Bahle G, Ohler S, Tröster G, et al. Smartphone-based recognition of states and state changes in bipolar disorder patients. IEEE J Biomed Health Inform 2015 Jan;19(1):140-148. [CrossRef] [Medline]
    27. Kemp AH, Quintana DS, Gray MA, Felmingham KL, Brown K, Gatt JM. Impact of depression and antidepressant treatment on heart rate variability: a review and meta-analysis. Biol Psychiatry 2010 Jun 1;67(11):1067-1074. [CrossRef] [Medline]
    28. Greco A, Valenza G, Lanata A, Rota G, Scilingo EP. Electrodermal activity in bipolar patients during affective elicitation. IEEE J Biomed Health Inform 2014 Nov;18(6):1865-1873. [CrossRef] [Medline]
    29. Ayzenberg Y, Picard RW. FEEL: a system for frequent event and electrodermal activity labeling. IEEE J Biomed Health Inform 2014 Jan;18(1):266-277. [CrossRef] [Medline]
    30. Saeb S, Zhang M, Karr CJ, Schueller SM, Corden ME, Kording KP, et al. Mobile phone sensor correlates of depressive symptom severity in daily-life behavior: an exploratory study. J Med Internet Res 2015;17(7):e175 [FREE Full text] [CrossRef] [Medline]
    31. Kim HJ, Choi YS. Exploring emotional preference for smartphone applications. 2012 Presented at: IEEE Consumer Communications and Networking Conference; 14-17 January 2012; Las Vegas, NV. [CrossRef]
    32. Kim M, Kim HJ, Lee SJ, Choi YS. A touch based affective user interface for smartphone. 2013 Presented at: IEEE International Conference on Consumer Electronics; 11-14 January 2013; Las Vegas, NV.
    33. Dolan RJ. Emotion, cognition, and behavior. Science 2002 Nov 8;298(5596):1191-1194. [CrossRef] [Medline]
    34. Wiese J, Min J, Hong J, Zimmerman J. “You never call, you never write”: call and SMS logs do not always indicate tie strength. 2015 Presented at: Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing; 14-18 March 2015; Vancouver, BC. [CrossRef]
    35. LiKamWa R, Liu Y, Lane N, Zhong L. Can your smartphone infer your mood? PhoneSense Workshop 2011.
    36. Burns MN, Begale M, Duffecy J, Gergle D, Karr CJ, Giangrande E, et al. Harnessing context sensing to develop a mobile intervention for depression. J Med Internet Res 2011;13(3):e55 [FREE Full text] [CrossRef] [Medline]
    37. LiKamWa R, Liu Y, Lane N, Zhong L. Moodscope: building a mood sensor from smartphone usage patterns. 2013 Presented at: Proceeding of the 11th Annual International Conference on Mobile Systems, Applications, and Services; 25-28 June 2013; Taipei, Taiwan. [CrossRef]
    38. Tan PN, Steinbach M, Kumar V. Introduction to Data Mining. Boston, MA: Pearson Addison Wesley; 2006.
    39. Larose DT. Discovering Knowledge in Data: An Introduction to Data Mining. Hoboken, NJ: Wiley-Interscience; 2005.
    40. Xu L, Yan P, Chang T. Best first strategy for feature selection. 1988 Presented at: Proceedings of the 9th International Conference on Pattern Recognition; 14-17 November 1988; Rome, Italy p. 706-708. [CrossRef]
    41. Russell JA. A circumplex model of affect. J Pers Soc Psychol 1980;39(6):1161-1178. [CrossRef]
    42. Olfson M, Marcus SC, Druss B, Pincus HA. National trends in the use of outpatient psychotherapy. Am J Psychiatry 2002 Nov;159(11):1914-1920. [CrossRef] [Medline]
    43. Sinyor M, Schaffer A, Levitt A. The sequenced treatment alternatives to relieve depression (STAR*D) trial: a review. Can J Psychiatry 2010 Mar;55(3):126-135. [Medline]

    ‎
    EMI: ecological momentary intervention
    GPS: Global Positioning System
    NB: Naïve Bayes classifier
    SMS: short message service
    SVM: Support Vector Machine
    VAS: visual analogue scale

    Edited by G Eysenbach; submitted 22.01.16; peer-reviewed by S Saeb, S Schueller; comments to author 28.03.16; accepted 28.06.16; published 10.08.16

    Copyright

    ©Galen Chin-Lun Hung, Pei-Ching Yang, Chia-Chi Chang, Jung-Hsien Chiang, Ying-Yeh Chen. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 10.08.2016.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.