A Semantics-based Approach to Disclosure Classification in User-Generated Online Content

As users engage in public discourse, the rate of voluntarily disclosed personal information has seen a steep increase. So-called self-disclosure can result in a number of privacy concerns. Users are often unaware of the sheer amount of personal information they share across online forums, commentaries, and social networks, as well as the power of modern AI to synthesize and gain insights from this data. This paper presents an approach to detect emotional and informational self-disclosure in natural language. We hypothesize that identifying frame semantics can meaningfully support this task. Specifically, we use Semantic Role Labeling to identify the lexical units and their semantic roles that signal self-disclosure. Experimental results on Reddit data show the performance gain of our method when compared to standard text classification methods based on BiLSTM, and BERT. In addition to improved performance, our approach provides insights into the drivers of disclosure behaviors.


Introduction
With the growth of social networking sites and increased user engagement with public discourse online, heightened rates of user disclosure of personal information (henceforth, self-disclosure) have raised privacy and security concerns. Prior research (Keep et al., 2012) suggests that selfdisclosure may be more common online thanks to the relative anonymity afforded in this environment and the lack of non-verbal cues to signal thoughts or feelings. Users are often unaware of the aggregate amount of personal information they share, as well as the power of modern AI to synthesize and gain insights from this data.
Automating the process of identification and classification of private information in text is challenging (Abril et al., 2011). A large volume of textual data needs to be processed, and a number of real-time requirements need to be met (Agerri et al., 2015), (Singh and Nene, 2013), and significant ambiguities arise from nuanced use of natural language.
In this work, we adopt the existing framework of Semantic Role Labeling to support self-disclosure identification and classification. Semantic role labeling (SRL) is a process which aims to recognize all predicate-argument pairs along with their roles in a given sentence and its predicates (usually verbs). SRL is a task with numerous applications to Natural Language Processing (NLP) like Question-Answering (Abujabal et al., 2017), Information Extraction (Christensen et al., 2011), Machine Translation (Xiong et al., 2012), text-to-scene generation (Coyne et al., 2012), dialog systems (Chen et al., 2013) and social-network extraction (Agarwal et al., 2014). We hypothesize that the inclusion of semantic frames can provide valuable context for the detection of self-disclosure. Our code is available here 1 .
Self-disclosure in social media can take two nonexclusive forms: emotional disclosure, in which the user reveals their feelings towards something or someone; and informational disclosure, where the user reveals objective personal information, e.g., age, career, or address. Following, we propose an approach detecting emotional and informational self-disclosure in text. Specifically, we leverage the structured representations of frame semantics. Our method outperforms standard classification methods based on CNN, BiLSTM, and BERT by 9% for emotional disclosure and 2% for informational disclosure.

Related Work
Detection of private and sensitive information from user texts has been studied extensively. However, approaches to date appear to be either confined to specific application domains or targeted to specific identifying attributes. Many automated methods for detection of self-disclosure rely on the presence of first-person pronouns, disregard context, and suffer from poor generalizability (Caliskan Islam et al., 2014;Wang et al., 2016a;Vasalou et al., 2011;Bak et al., 2014;Chow et al., 2008;Choi et al., 2013).
Fundamentally, most studies equate disclosure to the revelation of explicitly private information (Wang et al., 2016b). We posit that this frame is insufficient to capture the breadth of victimization that can result from voluntarily shared personal information (e.g., cyberbullying (Joinson and Paine, 2007)), and critically, harms supported by increasingly powerful inference algorithms operating on massive-scale longitudinal datasets (e.g., targeting, manipulation (Paramarta et al., 2018)).
Recent advances in language models have shown improved applicability to classification tasks. Vaswani et al. (2017) introduced a deep bidirectional transformer (BERT) which provided state-of-the-art results on numerous NLP tasks (Devlin et al., 2018). We use BERT as a baseline in this paper. Mehdy et al. (2019) proposed a method to detect disclosures of private information in natural language text through linguistically-motivated artificial neural networks. However, these models do not provide insights into the drivers of disclosure. Sundar et al. (2020) propose heuristics to predict information disclosure, but these heuristics are not exhaustive. Gildea and Jurafsky (2002) first introduced the task of detecting the semantic frames evoked in text (Semantic Role Labeling; SRL), along with their arguments, formalized in Baker et al. (2007). There are several SRL annotation conventions, such as PropBank (Palmer et al., 2005) and FrameNet (Baker et al., 2007). Propbank provides a more general role labeling, whereas FrameNet provides much denser annotations with more than 1200 frame types. Several studies have explored (Guan et al., 2019) SRL with deep learning techniques. Sikos and Padó (2018) shows that the semantic frames defined in FrameNet can be extended across languages.
Apart from this, several studies have applied SRL features to other Natural Language Processing tasks. Marzinotto et al. (2019) adapted a FrameNet semantic parser for spoken language understanding using adversarial learning. Abujabal et al. (2017) used semantic parsing to generate templates for question answering tasks. Christensen et al. (2011) used semantic role labeling to extract relations in the text without predefining domain or vocabulary. Xiong et al. (2012) utilized the predicate-argument structure of semantic role labeling to enhance Machine Translation. Coyne et al. (2012) extends the existing FrameNet database to bridge visual cues with semantic frames for the text-to-scene generation task. Chen et al. (2013) used semantic parsers to induce and fill semantic slots in dialog systems automatically. While, Agarwal et al. (2014) extract social networks from unstructured text using the FrameNet-defined tree kernel representations.
Our work is motivated in part by Tenney et al. (2019). Authors show that BERT contains elements of the natural language processing pipeline: POS tagging, parsing, NER, semantic roles, and coreference. We explore semantic role labeling specifically for the disclosure detection problem.

Frame Semantics
The theory of Frame Semantics asserts that people understand the meaning of words largely by the frames which they evoke. The frames represent story fragments, which serve to connect a group of words to a bundle of meanings; for example, the term avenger evokes the Revenge frame, which describes a complex series of events and a group of participants. The study of Frame Semantics attempts to define frames and the "participants and props" involved in each of them.
A frame is composed of lexical units with frame elements. A lexical unit (LU) is a pairing of a word with a meaning. Typically, each sense of a word belongs to a different semantic frame, a script-like conceptual structure that describes a particular type of situation, object, or event along with its participants and props. For example, the Apply heat frame describes a common situation involving a Cook, Food, and a Heating Instrument. These semantic roles are referred to as frame elements (FEs).
Frame-evoking words are LUs in the Apply heat frame. This frame is evoked by words such as bake, blanch, boil, broil, brown, simmer, steam, etc.
The FrameNet (Baker and Sato, 2003;Ruppenhofer et al., 2006) lexical database currently contains more than 13,000 lexical units, around 7,000 of which are hierarchically annotated. A total of approximately 1200 semantic frames are exemplified in more than 200,000 annotated sentences.

Frame Semantics for Disclosure Detection
We approach the problem of disclosure detection through the learning of semantic-role based labels common to disclosure. The intuition behind Semantic Role Labeling is to assign semantic roles consistent with the frame semantics that are predefined in FrameNet (Baker and Sato, 2003;Ruppenhofer et al., 2006) database. Accordingly, SRL models recover the latent predicate-argument structure of a sentence. Exemplar sentences and frame semantics are shown in Figures 1 and 2. Target words and text spans are highlighted in the sentence, and their lexical units are shown italicized below. Frames are shown in colored blocks, and frame element segments appear horizontally alongside the frame.
The SRL-labeled sentence in Figure 1, provides an example of a sentence containing emotional disclosure. The frame Emotion Active is invoked by the predicate "worried". This frame has two lexical units containing words "I" and "worried". The lexical unit "I" is assigned a semantic role of Experiencer. We call Experiencer a frameelement of Emotion Active frame. Clearly, Emotion Active with an Experiencer as "I" leads to a self-disclosure of emotion.  Figure 2 shows the case of multiple frames invoked by different predicates. The Make Agreement On Action is invoked by the predicate "agree". This frame has multiple lexical units, but two frame elements. The frame element Party 1 is assigned to "I" and Obligation is assigned to the span "with all the other points you made". This frame support informational self-disclosure.
Our model predicts disclosure in a sentence based on the semantic frames present. We formulate our disclosure classification model as follows. A sentence S contains a set of semantic frames Every semantic frame F j has a frame identification I j and frame elements set E j = {E j1 , E j2 , . . . , E jk } where k ≤ K and E jk ∈ E represents k th frame element of j th semantic frame in the sentence. E is a set of pre-defined frame elements in FrameNet. In our formulation of the problem, the sentence S contains a disclosure if at least one of the frames F contained in S is associated with disclosure, according to our classifier. Formally, where F is the set of semantic frames in the sentence. D is a disclosure-frame classification function which takes frame f ∈ F as the argument. D is the disclosure-sentence classification function for S and σ is the classification function.

Semantic Frame Embedding
The FrameNet project (Baker and Sato, 2003;Ruppenhofer et al., 2006) has developed a lexicon of more than 1,200 semantic frames, and a corpus of sentences annotated with frames. We use the FrameNet database to extract semantic frames from the sentences in our dataset. Frame-semantic parsing is a pipeline of three sub-tasks: predicate identification (Which words evoke the frames?); frame identification (Which frames does each predicate evoke?); and argument (frame-element) identification (Which span of the text provides possible roles from E?). Target identification is usually a classification problem. For the purpose of frame semantics extraction, we use open-SESAME (SEmi-markov Softmaxmargin ArguMEnt parser; Swayamdipta et al. (2017)), a framework that provides a pipeline for the three steps mentioned above. Open-SESAME uses Bi-LSTM to classify whether each word in the sentence is a predicate. For each detected predicate (mapped to all possible spans in the sentence), the framework classifies the semantic frame invoked using another Bi-LSTM. Then the framework uses segmental RNN (SegRNN; Kong et al. (2015)) for predicting frame-elements for the semantic frames detected in the previous step. A semantic role labeling of a sentence S is set of semantic frames F = {F 1 , . . . , F m } where m ≤ M . Every semantic frame F j has a name I j and frame elements set E j = {E j1 , . . . , E jk } where k ≤ K. E jk is the k th frame element of j th semantic frame in the sentence S. A frame-element has a name and a span of the sentence.

Frame-semantic feature representation
We represent the frame semantics in sentence S as the set of embeddings for each semantic frame. A semantic frame is represented as a combination of two parts. The first part is the predicate embedding P j , a concatenation of the word embedding for predicate w j and the word embedding for frame name I j .
where I j is the frame name, w j is the corresponding predicate. The second part of the semantic frame embedding is frame-element embedding F E j . The embedding for frame-elements set E j is calculated as the combination of embeddings for all frame elements E j1 , E j2 , . . . , E jK . The embedding for each frame element E jk is a concatenation of word embeddings for frame-element name and corresponding span.
where s jk is the span for k th frame-element and E jk is the frame-element name. Thus, a frame is embedded as where P j is the predicate-frame embedding and F E j is the frame-elements embedding. W f , b f are weight and bias parameters for a fully connected layer. Thus, the sentence S with M frames has a frame semantic representation as

Classification model
In this section, we discuss our model for the disclosure function described in Eq.1. The semantic frame representations extracted in Section 5.1 are stacked to form the sentence representation We model the function D as a multi-layer perceptron that is applied on each semantic frame. The max function in Eq.1 is modeled as the MaxPool layer that outputs maximum activation from all frames.
Figure 4: Classification model: The frame representation is shown with two red cells. The green cell is the sentence representation. σ is SoftMax layer whose output is sent to the max function.
The output of max layer is normalized with Soft-Max again. The maximum likelihood loss of the final two outputs is optimized.
Through this approach, we classify emotional disclosure and informational disclosure. Gold training data from Affcon 2020 is used as the labeled data.
6 Experimental Evaluation

Dataset
Reddit 2 is a popular discussion forum platform consisting of a large number of subreddits focus-ing on different topics and interests. The Reddit dataset (Jaidka et al., 2020) consists of 12,860 labeled sentences and 5,000 unlabeled sentences. The sentences are sampled from comments in two subreddits: r/CasualConversation, a 'friendlier' sub-community where people are encouraged to share what is on their mind about any topic, and r/OffMyChest, intended as a mutually supportive community where 'deeply emotional things people cannot tell others they know can be told'. The topics of the collected posts are limited to relationships, with the following tags: "wife"; "girlfriend"; "gf"; "husband"; "boyfriend" and "bf". The statistics of the data from each community are detailed in Table 1  The dataset contains six gold labels for each sentence: emotional disclosure; information disclosure; support; general support; information support; and, emotional support. For the purpose of this paper, we only use gold labels of emotional and information disclosure.

Label Frequency
Emotional 0.31 Informational 0.38 The Open-SESAME framework assumes a grammatically correct sentence input for which parts-ofspeech can be extracted easily. However, Reddit data is prone to ungrammatical sentences, particularly in long paragraphs. To ameliorate this, we exclude from our analysis sentences with more than 50 words. For our dataset, our model provides frame-semantics with M = 6 and K = 5.

Semantic frames and frame elements closely linked with SD
In this section, we study frame relevance to emotional and informational disclosure. We operationalize the relevance of a frame as a correlation with emotional (or informational) disclosure. For this analysis, we take the Term Frequency -Inverse Document Frequency (TF-IDF) representation of each sentence with respect to the semantic frames evoked in the sentences. With the TF-IDF representation as features of the sentences, we calculate and normalize feature importance using a random forest classifier. In Figure 5, we show the semantic frame relevance for the top 40 most relevant semantic frames to emotional and informational disclosure. As reported, frames related to emotion (Feeling, Emotion directed) show high correlation with emotional disclosure as compared to informational disclosure. For example, "The parent is INFURI-ATED" is a sentence containing the word 'INFU-RIATED' which evokes 'Emotion directed' frame.
We also see the frame 'Kinship,' a frame that is evoked by kinship relational words has a high correlation to the informational disclosure as compared to emotional disclosure. For example, "Matilde is Gilberto's COUSIN" is a sentence containing the word 'COUSIN' which evokes 'Kinship' frame. The information disclosed in this sentence is Matilde and Gilberto are related by a kinship relation called cousin.
Although the disclosure is not directly correlated with frame invoked, the observations above are strong motivation for us to explore our method in Section 5 to embed this information in our classification model for high performance.

B1: biLSTM model
We use the text classification model Bidirectional LSTM as our baseline. This model is an extension of traditional LSTM. Bidirectional LSTM trains two (instead of one) LSTM on the input sequence. The first input sequence is as-is and the second is on a reversed copy of the input sequence. This provides a forward and backward context for each token in the input sentence. biLSTMs provide competitive performance on text classification tasks. (Devlin et al., 2018) We use Glove.6B.200d embedding for the input tokens. The model is trained with an Adam optimizer and a learning rate of 5e-4 for ten epochs. The results are as shown in Tables 3 and 4. As shown, informational disclosure labels are detected with higher accuracy, and achieve 0.61 F1 score, mostly due to a higher precision rate than what we report for emotional disclosure labels.

B2: BERT model
BERT is a deeply bidirectional, unsupervised language representation,pre-trained (Devlin et al., 2018) using only a plain text corpus from BooksCorpus (800M words) (Zhu et al., 2015) and English Wikipedia (2,500M words). This makes it particularly suitable for our baseline task, as it allows us to input training text as-is, without imposing predefined and possibly biased features or setting hyper-parameters that would require further analysis.
We train the bert-base-uncased version of BERT (12 layers, with a hidden size of 768 and 12 selfattention heads) with an Adam optimizer and learning rate 1e − 5 for two epochs. The results of this baseline method are shown in Tables 3 and 4.

Model Classification Results
In our model A1, we use Glove.6B.200d embedding for the input tokens. The frame-semantic feature representation of each frame in the sentence is extracted, as described in section 5.1. A regular dropout layer will not help regularize the activations independently for the equivalent features in different frames. Accordingly, we add a spatial 1D-Dropout layer after the frame-semantic embedding layer to help regularize the model. We apply 1D convolution with 32 kernels after the Dropout layers. The output of this layer is passed to a Max-Pool layer applied to each kernel output. Finally, the output of the MaxPool layer is passed to a fully-connected layer and a classification (Sigmoid) layer.
Another variation of our model comes with replacing the Glove + biLSTM layers with an ELMO (Peters et al., 2018) or BERT (Devlin et al., 2018) embedding. ELMO is a shallow bidirectional model. BERT, unlike ELMO, is a deeply bidirectional model. We present two results with these layers as the contextual word embedding layers for extracting frame-semantic representations.

Model
Precision   For the BERT version of our model, we take frame identification name embedding as the hidden state of the BERT last layer when the input is a tokenized version of the name. The results in Tables 3  and 4 indicate the performance of our model with BERT embeddings. There is a marginal improvement in the F1-score compared to the model with Glove embedding.

Ablation
We study the effect of frame-semantic features on the classification task. Our model, when compared with the biLSTM model, improves the F1-score on Emotional Disclosure by 9% and Informational Disclosure by 2%. We considered the contextual word embeddings of tokens in the sentence, frame identification name, and frame elements. This ablation study is to measure the effect of frameelements on the classification task qualitatively.  We use a smaller version A1-FE of our model. In this version, we remove the frame-element embeddings (from gray cells in Figure 3) from the semantic frame feature representation. Thus, framesemantic representation concatenates the embedding of a frame identification name and the predicate. The classification model remains the same as A1 (Figure 4).
Our results are summarized in Table 5. Emotional disclosure classification improves the F1score by 7% with frame identification features. This improves further (2%) with frame-element features. This result indicates that frame identification names carry maximum information pertained to Emotional Disclosure.
Informational disclosure classification performs similarly with or without frame-identification features. However, there is an improvement in F1score by 2% with frame-element features. This improvement suggests that we cannot infer informational disclosure from the frame identification name alone. Frame-elements are crucial for detecting informational disclosure.
10 The Role of Conversation in Self-Disclosure In this section, we explore the effect of peer influence features on disclosure detection in the conversation. Given the highly contextual and inherently social character of self-disclosure, we are motivated to explore peer influence as a meaningful signal for this behavior. We incorporate peer effects in models aiming to detect and predict disclosure in conversation. We touch upon some early findings in this direction. Practically we develop a model that addresses the problem of predicting disclosure in a given comment using the comment text and peer influence features extracted from previous comments.
We have completed an early exploration of conversational modeling of the effects of peer influence in Reddit conversations. We augmented our original Reddit dataset (Jaidka et al., 2020) with the missing responses from the original comments using Python Reddit API Wrapper (PRAW) 3 . We sample 1200 conversations (about 1600 users) from the comment trees and manually annotate the emotional and informational disclosure in comments using Mechanical Turk with consensus from three workers. We labeled the dataset into three labels: No Disclosure vs Low Disclosure vs High Disclosure for each of emotional and informational disclosure. We then calculate reliability metrics 4 which indicate high reliability scores for binary: No Disclosure (no) vs Disclosure (low or high) labels.  A Reddit post is composed of a post text written by an author, comments and votes. A comment is composed of comment text written by an author, reply comments and votes. Recursive comments within each post form a comment tree. We sample conversations from this comment tree by recursively taking the comments where the successor comment is a direct reply to the predecessor comment. Unlike the original dataset (Jaidka et al., 2020) which provides annotations for sentences sampled from the comments, here we classify the entire comment.

Disclosure
Positive Total Emo Disclosure 826 1200 Info Disclosure 543 1200 Our model is a simple modification of BertForSe-quenceClassification from Facebook's Hugging-Face library (Wolf et al., 2019). The model predicts disclosure in the fifth comment using the comment text and the following peer influence factors: number of unique self-disclosures in the last four comments; whether there is a disclosure from the same user in last four comments; the total number of users in the given conversation; previous disclosure; time elapsed since the most recent prior disclosure; and time elapsed since the most recent prior comment. Elapsed time is normalized to [0, 1], where 1 represents one day and any more than a day.
The bert-base-uncased architecture of the BERT model is enhanced with the peer influence features listed above. The pooled output of the BERT model of dimension 768 is passed through a dropout layer with a dropout rate of 0.05 and then passed through a linear layer with an output dimension of 16. The peer influence features are appended to this output and passed though a binary classification layer. We train the model using an Adam optimizer with a learning rate of 4 × 10 −5 for one epoch.
The emotional disclosure classification model achieves 87.9% F1-score (representing a 2.2% improvement on the model without peer influence features) and 0.54 Matthews Correlation Coefficient (MCC) with 5-fold cross validation. Similarly, the informational disclosure classification model achieves 73.3% F1-score (1.5% improvement) and 0.61 MCC with 5-fold cross validation.
These early promising results in affect analysis of peer-influence on disclosure, point to further exploration of frame semantics in conversation/dialogue systems as a promising avenue for future work.

Conclusion
In this paper, we have presented a study using semantic role labels to support the detection of voluntarily disclosed private information in a usergenerated text. To the best of our knowledge, ours is the first study performing in-depth semantic analysis to facilitate detection and analysis of selfdisclosure. In doing so, we have simultaneously improved upon state-of-the-art performance for detection of disclosure in sentences and furnished meaningful semantic information about tagged disclosures. The success of frame semantics in helping to identify sentences containing disclosure is perhaps unsurprising given its power in distilling meaning from groups of individual words. Yet, our models have potential for more insightful analysis, beyond what is presented here. For example, semantic frames across sentences in the comment can be linked in a graph-like structure if the same entities evoke the semantic frames. Moreover, the same can be applied across comments. We will explore these graph-based approaches in future work.