Modelo social y política educativa hacia la diversidad como fuente de valores.

Labeling the Text Segments: we recorded four meetings per each group over four weeks. Immediately before the start of each meeting we held an interview with each participant, asking them to describe everything they remembered from their previous meeting. Thus one week after each meeting, we held what we call a recall session where each participant described everything one could recall while being audio recorded. Then, similarly to the meetings, the recordings of the recall sessions were transcribed.

Finally, by computing the LSI[67] topic similarity (after preprocessing steps such as stop words removal, converting all words to lower case, etc.) on all seg- ments of a meeting we created a topic model of that meeting. Subsequently, by querying the model with the two corresponding recall sessions, we automatically computed objective labels on how memorable each segment of a meeting was for an involved participant. This was done by comparing every segment of the meet- ing with the corresponding recall sessions based on the LSI topic model on the segments. Finally, the similarity between each segment and the corresponding recall sessions are computed based on cosine similarity. Therefore, by comput- ing the semantic similarity between each segment and a segment we produce objective labels of how much a participant remembered or forgot.

Meetings differ from one another in terms of topics, depth, breadth, etc. Therefore, the range of similarity scores that are computed based on LSI differ from one meeting to another. Hence, for each meeting we normalize the scores by subtracting from the computed similarity score the average similarity score of all segments within that meeting. Finally, we determine a decision threshold for discriminating memorable from non-memorable. For this purpose, we examined the dataset of meetings to determine the decision threshold empirically. In order to determine this threshold, we manually looked into the dataset. We found out that a threshold value of 0.10 is optimal for distinguishing memorable segments from the others. The overall number of segments in our dataset is 1008, from which 616 segments are labeled as memorable and 392 labeled as forgotten. We note that the number of segments per meeting vary, because as explained earlier the segments were computed using the texttiling algorithm and each meeting may vary in the number of segments which are topically coherent.

6.5

Preliminary Analysis

In this section we present a preliminary analysis of our dataset through a case study to verify the applicability of our research goal. We will first, however, briefly describe the sentiment analysis algorithm that we implemented along with

103 6.5 Preliminary Analysis

a brief explanation of EDA signals.

6.5.1

Sentiment Analysis Classifier

For analyzing sentiment, we implemented an unsupervised sentiment classifier similar to the one used in[19, 20]. The reason that we used this classifier over the commonly used SentiStrength classifier[131] was that it significantly outper- formed the SentiStrength classifier[131] on three different datasets. Therefore, by using this classifier we opted for a sentiment analyzer superior to the Sen- tiStrength.

Here we briefly explain the sentiment classifier that we implemented. Each conversation segment is split into smaller text snippets based on the punctuation marks ‘.’, ‘!’ and ‘?’. We therefore define a text snippet as a number of words that occur in between two punctuation marks.

The algorithm first replaces slangs with their equivalences using a slang dic- tionary to identify sentiment words more accurately. To build this slang dictio- nary, we manually collected slang phrases to include in the dictionary by using as many online resources that we could find, and furthermore adding the slang dictionary of SentiStrength[131] to our collected dictionary.

Then in a second step, we used a modified sentiment lexicon[22] to tag all sentiment-bearing words in each conversation with their corresponding senti- ment scores. We further tagged all intensifier words (e.g. absolutely) and di- minishers (e.g. might) with their corresponding scores. Additionally, we tagged negation words. Finally, if a word did not belong to any of the mentioned cate- gories, it was tagged with the score ‘0’.

After having all the words in a document tagged either by their score or type, now we should handle occurrence of intensifiers, diminishers, and negations. First, the algorithm intensifies the strength of a sentiment-bearing word that ap- pears after an intensifier word, by the score of that intensifier word. Analogously, in the case of diminishers, we weaken the strength of a sentiment-bearing word that appears after a diminisher word by the strength of that diminisher. Finally, for handling negations, we flip the polarity of the score of a sentiment-bearing word that appears after a negation. Furthermore, we weaken the flipped senti- ment score by 1. That is, if the flipped score is positive, we subtract 1 from it and if it’s negative we add 1 to it. Note that, while performing the above mentioned computations, in all cases we ignore the ‘0’ tagged words that appear in between one of the above mentioned valence shifters and a sentiment-bearing words in a single text snippet.

104 6.5 Preliminary Analysis

words’ scores. We define the decision threshold for classifying text snippets as ‘0’. That is, if the overall sentiment score of a document is less than or equal to ‘0’ it is classified as negative, otherwise it is classified as positive.

As mentioned earlier, this classifier outperformed SentiStrength in terms of classification accuracy and F1 measure and area under the curve of a receiver

operating characteristic curve on three different datasets[20].

Finally, for computing the overall sentiment score of each segment, we aver- age the sentiment score of all text snippets present in that segment.

6.5.2

EDA Signal

EDA sensor reading is a measure of electrical skin response excreted by the ec- crine glands, which is connected to the sympathetic nervous system [44]. It is most commonly associated with changes in sympathetic arousal. Anger, hap- piness, interest, excitement as well as disgust can all cause changes in EDA. It is important to note that EDA is not correlated with valence, so both positive and negative affect can alter sympathetic arousal [44]. Thus, changes in EDA are most commonly associated with changes in sympathetic arousal. However, factors that cause increased sweat production, such as humidity, room tempera- ture, and physical exertion, can also cause increased EDA amplitude. However, in the case of our meetings dataset, since the participants stay in the same room throughout a meeting, such effects are controlled.

In the case of our meetings dataset, each meeting involving two persons who are both engaged in the conversation, it is reasonable to believe that most alter- ation in the EDA signal is caused by the content of the meeting. If there would have been more than two people in a meeting, there would have been the possi- bility that a person would have been left out of the conversation or would have not followed the conversation, hence our assumption in that case would not have been realistic. An EDA biophysical recording will contain an individual’s response to the content of a meeting and the behavior of the other individual involved in the meeting.

6.5.3

Case Study

Now that we explained our sentiment analyzer and the basics of EDA, we can proceed with analyzing the data. We first start our analysis with a case study on the dataset. The goal of this case study is to examine whether there is any correlation between the sentiment expressed in the transcribed conversation and memorability, between EDA signal and memorability or even between EDA and

105 6.5 Preliminary Analysis

the sentiment expressed in the text. The result of this analysis not only can help us in designing a model for predicting the memorable/ non-memorable moments of a conversation, but also can be used as a helpful finding for other researchers in the field.

Since we are dealing with time series data (a sequence of topically coher- ent segments extracted from a meeting along with their corresponding time- synchronized EDA signal) we use the Cross Correlation Function (CCF) which in essence is the convolution between two signals. The reason behind using CCF is that it analyzes the correlation between two signals at different lags (i.e., dis- placements of the signals). In our case each lag is a segment of a conversation. Such correlation analysis, for example, can reveal not only a possible correlation between sentiment of a segment and its corresponding memorability score, but also can show if there is a correlation between the sentiment of the previous or next segments and the memorability of the current segment. Thus, since CCF is a useful method for assessing the effects of different parts of two signals on one another we utilize it in this experiment.

The results of our case study on randomly selected participants and meetings are presented in Figures 6.4, 6.5, and 6.6. We randomly selected five partici- pants from five different meetings of our dataset for this case study. Figure 6.4 shows that the there is a strong correlation between memorability and negative sentiment. Also, it shows that the segments that are preceded by segments with negative sentiment are more memorable. A segment of a meeting expressing negative sentiment, for example, could be regarding difficulties and problems in a project, or in extreme cases arguing and strong disagreements between the in- volved participants. This finding also mirrors the results of psychology research such as [139] which show that negative experiences are more memorable for people than positive ones. The fact that we could support the findings of a psy- chology lab study with our study on a in-the-wild dataset using information re- trieval techniques is an interesting finding.

We note that in Figures 6.4, 6.5, and 6.6 the height of the dashed lines (in both positive and negative directions) indicate the significance threshold and all bars that cross these lines being statistically significant.

Figure 6.5 reveals that memorability correlates with a rising EDA signal. How- ever, we also observe that a memorable segment is preceded by a falling EDA signal. Thus, we conclude that shortly after a local minimum of an EDA signal we can find a significant reduction in forgetting, and thus an increase in memo- rability.

Furthermore, Figure 6.6 shows that an increase in sentiment intensity, re- gardless of the polarity, triggers the rise of the EDA signal. We note that since

106 6.5 Preliminary Analysis

Figure 6.4. Cross Correlation between Sentiment and Memorability at different lags (each lag is a segment of a meeting)

Figure 6.5. Cross Correlation between EDA and Memorability at different lags (each lag is a segment of a meeting)