Suicide Ideation Detection via Social and Temporal User Representations using Hyperbolic Learning

Recent psychological studies indicate that individuals exhibiting suicidal ideation increasingly turn to social media rather than mental health practitioners. Personally contextualizing the buildup of such ideation is critical for accurate identification of users at risk. In this work, we propose a framework jointly leveraging a user’s emotional history and social information from a user’s neighborhood in a network to contextualize the interpretation of the latest tweet of a user on Twitter. Reflecting upon the scale-free nature of social network relationships, we propose the use of Hyperbolic Graph Convolution Networks, in combination with the Hawkes process to learn the historical emotional spectrum of a user in a time-sensitive manner. Our system significantly outperforms state-of-the-art methods on this task, showing the benefits of both socially and personally contextualized representations.


Introduction
Every 40 seconds, a person dies by suicide (Roth et al., 2018). Despite the success of psychoclinical methods, such as the Suicide Probability Scale (Bagge and Osman, 1998) and Suicide Ideation Questionnaire (wa Fu et al., 2007), the suicide rate in the U.S. has risen by 35% in the last 20 years (Hedegaard et al., 2020). While these methods are professional (Pestian et al., 2017), they have limited efficacy and may even impact participants negatively (Harris and Goh, 2017). Their limitations include barriers such as social stigma (Crisp et al., 2000), low literacy (Batterham et al., 2013), low motivation to seek hel (Essau, 2005), and finances (Czyz et al., 2013). Tragically, 80% of patients do not undergo clinical treatment, and 60% of those who died by suicide denied having suicidal thoughts to practitioners (McHugh et al., 2019). * Both authors contributed equally Contrarily, people turn to social media to express suicidal thoughts (Luxton et al., 2012;Coppersmith et al., 2014;Robinson et al., 2016), with 8 of 10 people disclosing their suicidal plans (Golden et al., 2009). Consequently, a growing body of work has shown that natural language processing can complement social media analysis to identify risk markers in online user behavior to aid suicide risk assessment (McCarthy, 2010;De Choudhury et al., 2016;Reger et al., 2020;Shing et al., 2018). However, analyzing individual user posts is not always sufficient to infer user's mental state and the associated suicide risk (Harris, 2010;Sisask et al., 2008).
Studies suggest that suicide can be influenced by social factors (Masuda et al., 2013;Gvion and Apter, 2012), and is a contagious phenomenon (Mann, 2002). If a user is inclined to suicide ideation, a neighbor in the social network also often exhibits suicidal behavior (Wray et al., 2011). Further, social media cultivates safe spaces that encourage users to share thoughts with those who appear similar to themselves (Bak et al., 2012;McPherson et al., 2001;Franklin et al., 2017). Analyzing such social context along with historical activity, as in Figure 1, can help further ascertain suicidal risk (Van Heeringen and Marušic, 2003).
According to psychosocial research, there exists an uneven distribution of power and influence on social media (Avin et al., 2018). People exhibiting suicidal ideation form social clusters (Robertson et al., 2012) and preferentially copy the behavior of popular users, manifesting social learning of suicide-related behavior such as the "copycat suicide" (Mesoudi, 2009;Henrich and Gil-White, 2001) (Figure 1). These social networks present a hierarchical structure of ideation propagation, characteristic for Scale-free networks ( Barabási and Bonabeau, 2003). In a scale-free network, most nodes have very few links, whereas a handful of in-so do i, will the pain ever end??
Ive good feeling about today's match, a win surely??? Figure 1: Illustration of social influence and context, specifically copycat suicidal ideation, in a scale-free network setting. Such social and temporal context can contextualize a user's state for a more accurate suicide risk assessment. We paraphrase all examples in this paper as per the moderate disguise scheme (Bruckman, 2002) to protect user privacy (Chancellor et al., 2019b). fluential nodes have a large number of connections, creating social hubs, further amplifying phenomena such as the "Werther effect" (Fahey et al., 2018).
Social networks with scale-free structure are subjects to major distortions when embedded into the Euclidean representation space (Chen et al., 2013;Aparicio et al., 2015) by ordinary graph neural networks. To overcome this limitation, we propose to model the social relations using graph convolutions over hyperbolic space (Chami et al., 2019).
Our key contributions are as follows: (i) We present the first deep graph neural framework to identify suicide ideation on social media by explicitly modeling users' social and temporal emotional context jointly ( §3).
(ii) Motivated by psychological studies and the scale-free nature of social networks, we propose the use of Hyperbolic Graph Convolutions ( §3.4).
(iii) We propose a mechanism leveraging Hawkes process to learn the historic emotional spectrum of a user in a time-sensitive manner from their historical posts ( §3.3).
(iv) Through a series of experiments ( §5), we show that our framework significantly outperforms existing methods ( §6.1) on this task, as well as standard Graph Neural Networks ( §6.2).
Aware of the sensitive nature of this work, we dedicate a standalone section ( §7) to the ethical considerations and applicability of this work.
2 Related Work

Suicide Ideation Detection
Early efforts in leveraging NLP for suicide ideation detection on social media (De Choudhury et al., 2013Shing et al., 2018;Sawhney et al., 2018) combine general features such as n-grams and POS tags with lexicons like LIWC (Pennebaker et al., 2001). Deep learning models like CNNs (Naderi et al., 2019) and LSTMs (Coppersmith et al., 2018) have improved suicide ideation detection (Ji et al., 2020) thanks to a more robust semantic context to interpret the tweet in question, however, lacking user-level context, are often unable to ascertain suicide risk (Sisask et al., 2008). The best performing models (Matero et al., 2019;Naderi et al., 2019) at the CLPsych (Zirikly et al., 2019) and CLEF e-Risk (Losada et al., 2019) exemplify the promising yet underexplored direction of user context modeling (Flek, 2020) for suicide ideation detection. Although recent studies (Shing et al., 2020;Sawhney et al., 2020) explore the personal historical context of users, community-based social context has rarely been explored for this task. One of the few attempts includes SNAPBAT-NET (Sinha et al., 2019), a shallow embedding model to extract network structural features.

Graph Neural Networks
While graph neural networks (GNNs) have made advances in enhancing NLP models for various tasks (Mishra et al., 2019a;Del Tredici et al., 2019;Lu and Li, 2020), two broad shortcomings limit their effectiveness for suicide ideation detection. First, these methods do not capture the personal historical and social network context together, both of which are strongly correlated to risk assessment on social media (Yang and Eisenstein, 2017). Second, studies have shown that users exhibiting suicide ideation tend to form social networks with scale-free characteristics (Jonas, 1992;Mesoudi, 2009), which regular GNNs are unable to accurately capture (Chami et al., 2019) in learnt social representations.We build on these limitations by combining historical and social contexts in the hy- Figure 2: An overview of Hyper-SOS: We first extract the emotional representation of the tweet to be assessed and the historic emotional spectrum of a user via the HEAT mechanism to initialize tweet nodes and user nodes in the heterogeneous social graph, respectively. A Hyperbolic GCN is then used to aggregate features from neighboring nodes to learn social and historic representation, which we use to assess the presence of suicidal intent. perbolic space to further contextualize and improve suicide ideation detection on social media.

Hyper-SOS: Formulation and Design
In this section we present the architecture of the Hyper-SOS framework (Hyperbolic Graph Convolutional Network for Suicide assessment On Social media) shown in Figure 3, designed to identify suicide ideation on social media by explicitly modeling user's social and temporal emotional context.

Problem Formulation
We formulate suicidal intent (SI) detection as a binary classification task to predict the presence of suicidal intent y i for a tweet t i , where, y i ∈ {SI present, SI absent}. We denote the tweet to be assessed for the presence of suicidal intent as t i ∈ T = {t 1 , t 2 , · · · , t N }, authored by a user u j ∈ U = {u 1 , u 2 , · · · , u M }, posted at time τ i curr . Each tweet t i is associated with history is a historic tweet authored by user u j posted at time τ i k with τ i 1 < τ i 2 < · · · < τ i L < τ i curr . Moreover, two users are connected if they interact with each other's tweets on Twitter. We acknowledge that modeling suicidal intent as a binary classification task is a strong simplification.
Tweet to be assessed: We utilize the final 768-dimension hidden state corresponding to the [CLS] token as the aggregate representation of emotions in a tweet. Formally, we encode each tweet to be assessed (t i ) to an emotion representation vector T i = BERT finetuned (t i ); T i ∈ R 768 .
Historical Tweets: We encode user's historical tweets h i k using our fine-tuned BERT to learn representations of a user's emotional spectrum over time as e i k = BERT finetuned (h i k ), e i k ∈ R 768 . These representations can be indicative of a user's mental state and emotion buildup over time (Aragón et al., 2019;Tarrier et al., 2007), and better contextualize temporal behavior to ascertain suicidal intent (Links et al., 2008;Palmier-Claus et al., 2012).

Modeling Personal Historical Context
To model historical emotions of a user and factor in the natural irregularities in posting time of historical tweets (Lei et al., 2018;Wojcik and Hughes, 2019), we propose the HEAT mechanism: Hawkes temporal Emotion AggregraTion. HEAT leverages Hawkes Process (Hawkes, 1971), a selfexciting temporal point process to model the in-tensity of emotions whenever a tweet is posted in the past (Guo et al., 2019). Intuitively, it assumes that emotions exhibited in different historic tweets can influence one another. To obtain the final historic representation (E i j ∈ R 768 ) of the tweet to be assessed t i , HEAT aggregates encoded historical emotions e i k using an exponential kernel as: where, ∆τ k is the time gap between a historical tweet and the tweet to be assessed (current tweet) posted at time τ k and τ curr , respectively. and β are hyperparameters such that < β.

Modeling Social Network Context
Studies show that users' emotions (Hill et al., 2010(Hill et al., , 2015, depressive behavior , and loneliness (Cacioppo et al., 2009) can be transmitted through social connections. Hence, leveraging social relationships between users can contextualize potential suicidal intent (Mueller and Abrutyn, 2015;Burnap et al., 2015;Colombo et al., 2016). We model such relationships as a graph G = (V, E), where each edge e U ∈ E represents one of three types of interaction between two users u x , u y ∈ U : i) User u x quotes (retweets) a tweet t i , posted by user u y , ii) User u x mentions user u y in a tweet t i , iii) User u x replies to user u y , by posting a tweet t i . We further extend the social graph G by introducing tweet nodes t ∈ T , which represent labeled tweets to be assessed. Each tweet node t is connected to its author (user) node u by a user-tweet interaction edge e T ∈ E. The constructed social graph G is heterogeneous, having two types (users and tweets) of nodes V = {U ∪T }, and two types (user-user and user-tweet) of edges E = {e T ∪ e U }, as shown in Figure 2. Note that the tweet nodes t are labeled for the presence of suicidal intent, while the user nodes u are unlabeled.

Hyperbolic Graph Neural Network
To augment language and historical context-based features, we leverage GNNs to learn representations of the constructed social graph G. However, most GNNs such as Graph Convolution Networks (GCNs) operate in the Euclidean space, and often do not generalize well to the kind of hierarchical, tree-like networks users on social media, particularly those exhibiting suicidal behavior (Chen et al., 2013). Sociological studies (Bild et al.,

Frechet Mean
Hyperbolic Non-linear Activation 2015; Aparicio et al., 2015), show that such networks show scale-free characteristics (Scatà et al., 2018), which follow the power law, i.e., the degree distribution of nodes decreases exponentially with a few nodes having a large number of connections (Ravasz and Barabási, 2003). To capture such hierarchical and scale-free structural properties in the social network graph, we propose the use of a Hyperbolic Graph Convolution Network (HGCN) (Chami et al., 2019). HGCNs project language and historical feature embeddings in the hyperbolic space to minimize distortions and learn a better representation of the underlying scale-free nature of social networks (Krioukov et al., 2010;Papadopoulos et al., 2012).

Embedding in hyperbolic Space
Initialization: Our proposed HGCN aggregates features from neighboring nodes based on graph convolutions in the hyperbolic space to enrich learned language and historical emotion features. We initialize user nodes with their historical emotional spectrum E i j obtained through the HEAT mechanism, and tweet nodes with their emotional representation T i . Hyper-SOS then performs hyperbolic graph convolutions on these user and tweet features on the social graph G with |U | user nodes and |T | tweet nodes, which can also be represented by: its adjacency matrix A ∈ R (|U |+|T |)×(|U |+|T |) , a diagonal degree matrix D, where D ii = j A ij and a feature matrix F E ∈ R (|U |+|T |)×768 in the Euclidean space (denoted by E ), containing the 768-dimensional representation of each node.

Feature Aggregation by Hyperbolic Graph
Convolutions: To capture the network's hierarchical structure, Hyper-SOS first uses Poincaré ball manifold (exp K o ) with a sectional curvature −1/K, to map the features F E to the hyperbolic space Next, we perform a linear transformation to capture macroscopic neighborhood structures on the Poincaré ball manifold, followed by a Frechet Mean operation (Fréchet, 1948) denoted by FM.
Owing to the trainable curvature K, Hyper-SOS utilizes a hyperbolic non-linear activation with varying curvature (σ ⊗ K i−1 ,K i ) to allow a different curvature at each HGCN layer. ⊗ is the Möbius transformation operator. Formally, the feature aggregationbased update rule at the i th HGCN layer is: where −1/K i−1 and −1/K i are the hyperbolic curvatures at layer i − 1 and i, respectively.Ã = D − 1 2 AD − 1 2 is the degree normalized adjacency matrix and W is a trainable network parameter.
Finally, Hyper-SOS applies a dense layer with Rectified Linear Unit (ReLU ) to get a prediction vector, followed by softmax to output the probabilities for the presence of SI (ŷ) as: is the output of two stacked convolutions (Equation 2), with input O (0) set as the initial features F H .

Hyper-SOS Training and Optimization
Tweets with SI present form a very small proportion of the data (Ji et al., 2019). To address this problem of class imbalance (the imbalance is much greater in the real world), we train HGCN using Class-Balanced Focal Loss Cui et al., 2019). This loss function re-weights loss inversely with the effective number of samples per class, thereby yielding a class-balanced loss L as: where CB f ocal is class-balanced focal loss,ŷ i is the predicted label and y i is the label of the tweet to be assessed. β cb and γ are hyperparameters.

Data description
We use an existing Twitter dataset curated by Mishra et al. (2019b). The dataset contains Twitter timelines of 32,558 unique users, spanning over ten years of historical tweets from 2009 to 2019, summing up to 2.3M unlabeled tweets. The users were selected based on a seed lexicon of 143 suicidal phrases (e.g., "wanting to die", "last day"), which identified 34,306 tweets potentially containing suicide ideation. Two psychology students then annotated these tweets under the supervision of a professional psychologist, achieving Cohen's κ of 0.72, under the below guidelines: SI Present: Tweets where suicide ideation or attempts are discussed in a somber, non-flippant tone. SI Absent: Tweets with no evidence for risk of suicide, e.g., song lyrics, condolences, news. 3984 of the annotated tweets were identified as truly containing suicidal ideation. We feed all the 2.3M tweets to the HEAT mechanism to build user representations ( §3.3). The number of historical tweets per user (748 ± 789) and the time difference between consecutive tweets (2 ± 24 days) are indicative of large variations across users. 4070 users were found to have no historical tweets.

Data Split
We perform a stratified temporal 70:10:20 split, such that the train, validation, and test sets consist of 24014, 3431, and 6861 labeled tweets, respectively, and ensure that there is no overlap between users in these sets.

Network Analysis
In Table 1, we outline quantitative analyses of the social network G and report Gromov's δhyperbolicity of the graph (Jonckheere et al., 2008).

Contextual vs. Non-Contextual Models:
We compare Hyper-SOS with a variety of models in Table 2. We categorize the models as non-contextual, i.e., using the current tweet only, and more recent user-contextual, spanning both social and historical context. We note that user-contextual models drastically outperform RF+TF and LSTM that only leverage the language of the tweet without any additional user context. We attribute these improvements to the ability of personally contextual models to better ascertain a user's mental state through their historical activity and communities they interact with (Flek, 2020).
Contextual Models: Amongst models utilizing user's historical tweeting activity, we note methods modeling user tweets as temporal sequence (Du-alContextBERT, Hyper-SOS ) outperform bag-oftweets based models (C-CNN, SDM). On the other hand, prior work leveraging shallow features from social graph's structure without any temporal context (SNAPBATNET), is competitive to historical context models. This sets the premise for leveraging user's social relations as shallow features in neural methods, validating the effectiveness of social context for suicide ideation detection. Hyper-SOS significantly (p < 0.005) outperforms both social and historical contextual models, by virtue of its design. Hyper-SOS's design captures the scalefree nature of social relations through deep graph convolutions that blend language features across a user's historical tweeting activity to ascertain suicide ideation. These results validate the potential of utilizing social and historical context, as reflected in psychological works discussing the interpersonal theory of suicide (Joiner, 2007(Joiner, , 2009Orden et al., 2010). The higher Macro F1 of STATENet can be attributed to its compute-intensive, learnable historical modeling component. We leave using a learnable model to encode personal historical context to our future research directions. Hyper-SOS advances prior work on multiple fronts: i) combining social and historical context, ii) deep graph convolutions rather than shallow structural features, iii) capturing the scale-free nature of social networks through hyperbolic transformation, and iv) modeling a user's emotions based on the HEAT Mechanism. We explore the impact of each of these design choices through a series of ablative and exploratory analyses next. We analyze Hyper-SOS's components through an ablation study in Figure 5. We start by examining how the predictive power of the base (CurrentTweet) model changes when enriched with user's historical emotional context (HEAT), then gradually with social context (GCN), and finally on adding hyperbolic transformations over graph convolutions (Hyper-SOS). We note that incorporating a user's historical emotion spectrum via HEAT in a time-sensitive manner improves performance. Specifically, we note improvement in recall in terms of correctly identifying the presence of suicide ideation, likely due to the contextualization of a user's mental state via temporal context. We note significant (p < 0.05) improvements by leveraging social context, learning representations through feature aggregations within a user's neighborhood. These aggregations enrich the learned representations through the structure and historical emotion-based features of the communities the user interacts with, further amplifying the predictive power by greater contextualization.

Hyper-SOS Ablation Study
Lastly, building on the scale-free nature of social networks (Cox et al., 2012), leveraging feature transformations and graph convolutions in the hyperbolic space brings further improvements, as plain GCNs are unable to generalize over such hierarchical scale-free structures (Fronczak, 2018). Our observations revalidate the utility of Hyper-SOS for suicide ideation detection, specifically the influence of social context, and correctly capturing the network's scale-free traits . We analyze Hyper-SOS's sensitivity to the choice of temporal kernels for aggregating user's historical tweets as shown in Figure 6a. Overall, we notice that all user features learned via temporal aggregations outperform the CurrentTweet representation that does not use any historical information. The temporal kernels' performance improves as we factor in more historical tweets up to a year. We also find that Linear Decay performs better than Exponential Decay, hinting towards the importance of older tweets (> 3 months), in some cases, for contextualizing user's more recent suicide ideation with past emotional states.

Impact of Historical Context Aggregation
We note that using the HEAT mechanism as a temporal kernel consistently bestows significant improvements in Hyper-SOS's performance over time compared to all other variants. Self-exciting Historic Tweets 17 Aug 2016: This is a lot of grief, isn't it? 1 Jan 2016: Another setback. This is sad but I hope we do better next time 29 July 2014: Too many people died in the disaster, can we not do anything collectively? Humanity is hopeless

User C Suicidal Intent Present
Historic  Figure 7: We study four users in a social graph, with their tweet to be assessed, historical tweets, and timestamps. The social graph shows the four users and their interactions among themselves and other users. We also show aggregated historical emotions through HEAT mechanism over time.
temporal point processes such as the Hawkes mechanism have shown great promise in modeling social media dynamics (Rizoiu et al., 2017) and user behavior over time (Guo et al., 2019), revalidating the effectiveness of our proposed HEAT mechanism for learning user representations. Further, we note that Hyper-SOS's performance saturates on adding history beyond a year (Figure 6b). This is in line with psychology research, noting the depreciating importance of user's emotions over longer time periods (Selby et al., 2013;Kaplow et al., 2014;Glenn et al., 2020).  We analyze the importance of different types of social network relations based on how two users interact, by removing each relation type from the graph, as shown in Table 3. We note that removing relations based on user mentions, Hyper-SOS's performance drastically drops. We postulate this drop to the physical and cognitive effort a mention requires, as opposed to other forms of user interactions, in fact, it is the strongest form of user interaction on Twitter (Fink et al., 2016). This observation aligns with the findings of prior social network research that explore the influence of different communications on Twitter (Grabowicz et al., 2012), especially in networks where users can be influenced by a few "known" users (Cha et al., 2010). We note that relatively weaker forms of interactions such as quotes and replies do not contribute towards social context as much as mentions for suicide ideation detection. As suggested in past studies (Sultana et al., 2017), we observe that combining all the user interactions significantly (p < 0.005) improves Hyper-SOS's performance.

Exploratory and Error Analysis
We now present a qualitative analysis (Figure 7) to derive deeper insights into Hyper-SOS's predictive power. We see that the most recent tweet by user A shows explicit signs of suicidal intent. However, from their historical tweets, we notice that User A is talking about their gaming experience. Hence, studying the tweet to be assessed in isolation is not sufficient to assess users' risk, even for humans. Indeed, only temporally contextual models (HEAT, Hyper-SOS) correctly predict the absence of suicidal intent. In a more challenging case, that of user B, the tweet to be assessed shows no overt signs of suicidal intent, and their historical activity is not concerning either. Hyper-SOS's graph-based learning alleviates this issue by learning from a user's social context. Upon analyzing the network, we note User B's interaction with user C's tweets, which are suicidal, which might influence the tendency of User B to show suicidal behavior. Moreover, user C is a highly connected, influential node and has the potential to impression the emotions of users who interact with it (Chung and Zeng, 2020). User D presents an error case. We find that the tweet to be assessed is ambiguous, and historical activity is not informative either. Moreover, user D is isolated, highlighting that suicide ideation detection in the absence of contextual elements (historical activity, network interactions) can be highly subjective, and paves the way for future work.

Broader Impact and Ethics
Emphasizing the sensitive nature of this work, we acknowledge the trade-off between privacy and effectiveness (Eskisabel-Azpiazu et al., 2017). To avoid coercion and intrusive treatment, we work within the purview of acceptable privacy practices suggested by Chancellor et al. (2019b) and considerations discussed by Fiesler and Proferes (2018). Although informed consent of each user was not sought as it may be deemed coercive, we perform automatic de-identification of the dataset using named entity recognition (Benton et al., 2017a,b) to reduce the risk of including any identifying data in the raw data. We paraphrase all examples shown in this work to protect user privacy (Chancellor et al., 2019a,b). All the user data is kept separately on protected servers linked to the raw text and network data only through anonymous IDs.
We acknowledge that it is almost impossible to prevent abuse of released technology even when developed with good intentions (Jonas, 1984;Hovy and Spruit, 2016). Hence, we ensure that this analysis is shared only selectively and subject to IRB approval (Zimmer, 2009(Zimmer, , 2010 to avoid misuse such as Samaritan's Radar (Hsin et al., 2016).
Limitations: We acknowledge that suicidality is subjective (Keilp et al., 2012), the interpretation of this analysis may vary across individuals on social media (Puschman, 2017), and we do not know the true intentions of the user behind the post. We further acknowledge that suicide risk exists on a diverse spectrum (Bryan and Rudd, 2006), and a binary distinction is a task simplification intended to alert the human in the loop about exceeding a possible intervention threshold. We also recognize that the studied data may be susceptible to demographic, annotator, and medium-specific biases (Hovy and Spruit, 2016).

Future Practical Applicability
In the future, we would want to focus on creating a differentially private public model that can be shared with the community while preserving user privacy (Lyu et al., 2020;Yu et al., 2019). Further, suicide ideation detection on social media can involve failure modes that could potentially incorrectly ascertain suicide risk. To this end, we focus on Hyper-SOS as a preliminary tool for prioritizing human expert, clinical psychologist-based assessment.

Conclusion
Motivated by psychological studies, we propose a framework jointly leveraging emotional history from user's past tweets and social information from user's neighborhood in a network to contextualize the interpretation of the latest tweet of a user. To our knowledge, this is the first deep graph neural network study to automatically identify suicide ideation on social media. Reflecting upon the scalefree nature of social network relationships, we propose the use of Hyperbolic Graph Convolution Networks, and demonstrate that these are more suitable for our Twitter task than their euclidean counterparts. Inspired by geophysics, we further propose the use of HEAT Mechanism to learn the historic emotional spectrum of a user in a time-sensitive manner. When analyzing the contributions of its individual components to assess suicidal intent, we demonstrate the beneficial impact of both the social and personal context representations. Aditya Grover and Jure Leskovec. 2016. node2vec: Scalable feature learning for networks. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining, pages 855-864.