The Undertaker Reunites With Kane To Promote His Mayoral CampaignStart your the right way by following some great people on Glen jacobs twitter and adding their knowledge to your daily feed. Daily links to unmissable recruitment-related blog content, live tweets from important industry events and invaluable nuggets of wisdom from an industry pro. Not only does Jerome share links to his own extremely interesting and thought-provoking blog posts on a daily basis all of which are well worth a readbut he also uses his Twitter account to live tweet glen jacobs twitter some of the biggest events and conferences the recruitment industry has to offer, keep the world updated with SmartRecruiters news, and give his expert dianabol price in south africa on the most pressing recruitment industry news stories. But our favourite thing that Jerome uses his Twitter for glen jacobs twitter the bite-sized nuggets of wisdom that he likes to put out from time to time, to get us thinking, inspired and motivated. Tweet of the day!
Glen Jacobs | St. George's University
With a lifetime prevalence of The aim of this study, building on previous work qualitatively analyzing depression-related Twitter data, was to describe the development of a comprehensive annotation scheme ie, coding scheme for manually annotating Twitter data with Diagnostic and Statistical Manual of Mental Disorders, Edition 5 DSM 5 major depressive symptoms eg, depressed mood, weight change, psychomotor agitation, or retardation and Diagnostic and Statistical Manual of Mental Disorders, Edition IV DSM-IV psychosocial stressors eg, educational problems, problems with primary support group, housing problems.
Using this annotation scheme, we developed an annotated corpus, Depressive Symptom and Psychosocial Stressors Acquired Depression, the SAD corpus, consisting of tweets randomly sampled from the Twitter application programming interface API using depression-related keywords eg, depressed, gloomy, grief. An analysis of our annotated corpus yielded several key results. Second, the most prevalent symptoms in our dataset were depressed mood and fatigue or loss of energy.
Finally, we found very high positive correlations between some depression-related symptoms in our annotated dataset eg, fatigue or loss of energy and educational problems; educational problems and diminished ability to think.
We successfully developed an annotation scheme and an annotated corpus, the SAD corpus, consisting of tweets randomly-selected from the Twitter application programming interface using depression-related keywords.
Our analyses suggest that keyword queries alone might not be suitable for public health monitoring because context can change the meaning of keyword in a statement. However, postprocessing approaches could be useful for reducing the noise and improving the signal needed to detect depression symptoms using social media. Several national face-to-face and telephonic interview-based surveys in the United States aim to better understand the prevalence of depressive symptoms in the community.
However, these surveys are both episodic and expensive to conduct. As part of our goal of developing language technologies capable of accurately identifying depressive symptoms, we have developed a large manually annotated coded corpus or collection of Twitter posts tweets coded according to depressive symptoms and psychosocial stressors derived primarily from Diagnostic and Statistical Manual of Mental Disorders, Edition 5 DSM 5; depressive symptoms [ 3 ] and DSM-IV: This annotated corpus allows us to better understand the language used to express depressive symptoms and psychosocial stressors associated with depression, to identify relationships between depressive symptoms and psychosocial stressors expressed in tweets, and ultimately, to facilitate the development of a natural language processing system capable of automatically identifying depressive symptoms and psychosocial stressors from Twitter data.
The use of social media for health applications, particularly in the public health domain, is a rapidly growing area of research [ 5 , 6 ]. For example, social media has been leveraged to monitor infectious disease outbreaks [ 7 , 8 ] and understand prescription drug and smoking behaviors [ 9 - 11 ]. The value of social media for understanding mental health is particularly marked, given that it provides—in the case of Twitter—access to public, first person accounts of user behaviors, activities, thoughts, and feelings that may be indicative of emotional well-being [ 12 ].
Twitter in particular has several advantages as a resource for data. Third, acquiring Twitter data via the free public application programming interface API or commercial data resellers eg, gnip [ 14 ] is a relatively straightforward process. However, the use of Twitter data does present a number of challenges. Finally, Twitter users may selectively discuss topics of interest with researchers; for example, some individuals may not feel comfortable discussing disease-related symptoms on social media due to concerns of privacy and stigmatization [ 16 ].
The American Psychiatric Association defines major depressive disorder as continuously experiencing depressed mood and anhedonia for 2 weeks or more as well as one or more of the following symptoms: For individuals living with depression, the disorder can substantially reduce quality of life in several areas, including interactions with others, productivity at work, and quality of sleep and nutrition [ 20 ].
Depression has also been correlated with other high-risk behaviors and chronic diseases, including smoking [ 21 ], alcohol consumption [ 22 ], physical inactivity [ 23 ], and sleep disturbance [ 20 , 24 ]. Given the range and extent that depression affects a given population, several surveys, programs, and diagnostic tools have been developed to better understand or diagnose depressive disorder.
For example, in the United States, the National Survey on Drug Use and Health NSDUH provides national, state, and local data related to alcohol, tobacco, illegal drug use and abuse, and mental disorders, including nonincarcerated citizens of age 12 and older [ 25 ]. The Youth Risk Behavior Surveillance System YRBSS monitors behaviors such as alcohol and other drug use, tobacco use, and unhealthy dietary behaviors, and so on, and their correspondence with death and disability among youth and adults [ 26 ].
The Behavioral Risk Factors Surveillance System BRFSS is a telephone survey that collects data from across the United States, including health-related risk behaviors, chronic health conditions, and use of preventive services [ 27 ].
The BRFSS - Anxiety and Depression Optional Module specifically collects information at the state level to assess the prevalence of anxiety and depressive disorders with questions that closely mirror the DSM 5 major depression criteria. Recent work at the intersection of computer science, public health, and psychology suggests that social media can be leveraged to better understand, identify, and characterize depression [ 12 ]. For example, De Choudhury et al used a crowdsourcing data generation method in conjunction with machine learning to identify depression-indicative tweets at scale [ 28 ], whilst a follow-up study investigated the characteristics of Twitter users prior to the onset of depression, discovering that decrease in social activity , raised negative affect , highly clustered ego networks , heightened relational and medical concerns , and greater expression of religious involvement were all characteristic of the onset of depression [ 29 ].
In a study using Facebook, Schwartz et al used status updates and personality survey results as features in a regression model to classify the degree of depression of 28, Facebook users [ 30 ]. A temporal analysis of these posts demonstrated that mood worsens in the transition from summer to winter for users.
In this study, we build on these existing efforts by developing an annotation scheme for encoding depressive symptoms and psychosocial stressors associated with major depressive disorder in Twitter tweets and conducting analyses to provide insights into how users express these symptoms on Twitter. From these analyses, specifically, we aim to 1 validate the annotation scheme, 2 learn the predictive value of depression-related keywords with respect to identifying depressive symptoms and psychosocial stressors, 3 determine the frequency of depressive symptoms and psychosocial stressors expressed, 4 learn new predictive words for each depressive symptom and psychosocial stressor, and 5 assess whether particular depressive symptoms and psychosocial stressors are correlated with one another.
In order to understand the various ways indicators of major depression disorder could be expressed in tweets and address our goal of building a dataset that can be used to train and test machine learning algorithms, we developed an annotation scheme coding scheme based on 6 resources:. Suicide risk factors derived from the Columbia Suicide Severity Scale [ 38 ].
Finally, we enriched the annotation scheme with additional depression-related categories observed frequently in the data weather and media. The resulting scheme contains depression symptom categories 9 parent categories and psychosocial stressor categories 12 parent categories; Figure 1 [ 39 ]. Before finalizing the annotation scheme, both a psychiatrist and a counseling psychologist provided feedback on the annotation categories chosen and annotation instructions.
Data for our depression-related Twitter corpus were collected in two distinct ways. Both corpora are described in detail below. In addition to the SAD corpus, we sampled tweets from a large corpus of Twitter data developed for the CLPsych shared task [ 40 ].
In order to validate our annotation scheme, 3 annotators—2 psychology graduate researchers and a postdoctoral biomedical informatics researcher—annotated tweets from the SAD corpus in 3 phases.
In phase 1, all 3 annotators annotated tweets and reached agreement with consensus review. In phase 2, for the remaining tweets and for all annotator pair combinations, 2 annotators independently annotated tweets, and the remaining third annotator adjudicated any disagreements. For example, if annotators A1 and A2 annotated tweets, annotator A3 would adjudicate those tweets where A1 and A2 disagreed regarding the appropriate label.
We compared the annotations between each pair of annotators to determine the asserted categorical matches and mismatches. For example, a match occurs when both annotators eg, A1 and A2 annotated the same category for the same tweet. There are 2 types of mismatches: F score is computed from the matches and mismatches and given as a percentage from the following equation:.
In phase 3, each annotator independently annotated tweets tweets total from 3 annotators and to further ensure reliability, tweets were annotated by all 3 annotators. The resulting SAD corpus consists of tweets. A summary of this annotation workflow can be found in Figure 2. The CLPsych corpus was annotated by 1 annotator resulting in tweets which are not included in the SAD tweets. SAD corpus annotation in phases. For both the SAD and CLPsych corpora, in order to assess how accurately these depression-related keywords could identify depression-related tweets, we computed the precision of each depression-related keyword, defined as the count of tweets identified by the depression-related keyword and associated with a depression-related category divided by the total count of tweets identified by the depression-related keyword tweet hits.
We classified the resulting precision using 5 equally sized categories:. For each corpus and each precision category, we report the count of tweets identified by the count of depression-related keywords tweet hits. In order to estimate the proportion of said depressive symptoms and psychosocial stressors in our corpus, we characterized our total corpus of tweets by the proportion of tweets representing no evidence of clinical depression and evidence of clinical depression.
Of the tweets representing evidence of clinical depression, we report the proportion of tweets representing depressive symptoms and psychosocial stressors. Finally, we provide example subtypes of depressive symptoms and psychosocial stressors. We compared the distributions of annotation categories between the SAD and CLPsych corpora in order to identify salient characteristics of Twitter users with a publicly stated diagnosis of depression.
For both the SAD and CLPsych corpora, in order to identify words and phrases most characteristic of each category of depressive symptoms and psychosocial stressors eg, the words most characteristic of, say, occupational problem , we used a technique referred to as feature selection [ 42 ] keyword extraction in the corpus linguistics literature [ 43 ].
The 10 most characteristic words—identified by information gain—are reported for each category. Specifically, we used Weka version 3. For the tweet SAD corpus only, in order to determine whether a correlation exists between 2 specific depressive symptoms and psychosocial stressors, we computed Pearson correlation coefficients for each pairwise combination of the 21 parent categories of depressive symptoms and psychosocial stressors from the annotation scheme. Given that each symptom or stressor category has only 2 states annotated or not annotated , this correlation coefficient is sometimes called the phi coefficient, although the phi and Pearson correlation coefficients are algebraically identical.
A higher correlation coefficient indicates that when the psychosocial stressor is annotated, the depressive symptom is more likely to also be annotated. We used the r value to interpret magnitude because P values are affected by sample size, whereas r values are not. We classified the correlation magnitude using Cohen effect size criteria into 4 categories [ 45 ]: Our depression disorder scheme is comprised of 9 depressive symptoms and 12 psychosocial stressor categories that were applied to the SAD and CLPsych Twitter corpora.
We observed an average number of words with a standard deviation between 7 and 8 words Table 2. We observed high overall interannotator agreement F scores between annotator pairs: Overall F scores dropped slightly when comparing matches for all 3 annotators.
F scores varied widely across all annotated categories. High F scores were observed across annotator pairs for the depression symptom fatigue or loss of energy and psychosocial stressors recurrent thoughts of death and suicidal ideation.
For the SAD corpus, interannotator agreement F scores between annotators according to depressive symptoms and psychosocial stressors. For the SAD corpus, of the unique depression-related keywords, keywords were found corresponding to nonmutually exclusive tweet hits. We observed a range of precision across depression-related keyword hits: For the CLPsych corpus, the 35 unique depression-related keywords found correspond to nonmutually exclusive tweet hits.
The SAD corpus consists of tweets. Of these tweets, were annotated with one or more categories from our scheme: Overall, we observed a total of category annotations with the following distribution of categories annotated per tweets.
A total of The CLPsych corpus consists of tweets. All tweets were annotated with only 1 category from our scheme. Prevalence of categories by corpus. For the SAD corpus, 31 words were identified as the most informative features for classifying tweets for 11 depressive symptoms and psychosocial stressor categories Figure 5.
About 19 of these terms are also covered by the original LIWC keyword list. Most informative terms classified with associated depressive symptoms and psychosocial stressors. Shared terms occur at the intersect of the circled lists. In terms of depressive symptoms and psychosocial stressors, we observed 5 pairs with higher than large correlations, 3 pairs with medium to large correlations, and 13 with small correlations Figure 6.
Specifically, fatigue or loss of energy demonstrated large effect with disturbed sleep and educational problems. Depressed mood had large effect with feelings of worthlessness or excessive inappropriate guilt. Educational problems had large effect with fatigue or loss of energy and diminished ability to think or concentrate and indecisiveness. Housing problems and economic problems also demonstrated a large effect. SAD heat map of tweet-level, depressive symptom, and psychosocial stressor cooccurrences.
Darker means larger measure of Cohen effect size; lighter means smaller measure of Cohen effect size. The number that indexes the category on the y-axis also corresponds to the category for the x-axis.