Word associations contribute to machine learning in automatic scoring of degree of emotional tones in dream reports

Abstract

Scientific study of dreams requires the most objective methods to reliably analyze dream content. In this context, artificial intelligence should prove useful for an automatic and non subjective scoring technique. Past research has utilized word search and emotional affiliation methods, to model and automatically match human judges’ scoring of dream report’s negative emotional tone. The current study added word associations to improve the model’s accuracy. Word associations were established using words’ frequency of co-occurrence with their defining words as found in a dictionary and an encyclopedia. It was hypothesized that this addition would facilitate the machine learning model and improve its predictability beyond those of previous models. With a sample of 458 dreams, this model demonstrated an improvement in accuracy from 59% to 63% (kappa = .485) on the negative emotional tone scale, and for the first time reached an accuracy of 77% (kappa = .520) on the positive scale.

Introduction

Scientific research on dreaming and dream content has strived to establish objective methods of collecting dreams and analyzing them. In order to limit the distortion and the loss of recall of dreaming activity, subjects are asked to immediately report their dreams upon awakening in laboratory or at home. The reports are then coded using a number of scales that have been developed (Winget & Kramer, 1979). The most comprehensive system was developed by Hall and Van de Castle (1966) which has eight dimensions ranging from characters to emotions. Domhoff (1996) has improved this system notably with DreamSAT which utilizes spreadsheets as means for data entry and statistical analyses (Domhoff, 2003a).

Such a scoring system relies on human judges’ reliability and inter-judge agreements as a requirement for scientific studies (for example, Barcaro, Cavallero, & Navona, 2005). As such, these studies may benefit from an automatic scoring technique. Our current research program attempts to develop an automatic scoring system that ultimately can match human judges’ and the dreamer’s own scoring. As a first step, we have focused on dream emotions. While it is well established that dream production is primarily a cognitive process (Cavallero and Foulkes, 1993, Foulkes, 1985), there is increased interest in the emotional component of dreams. Hartmann (2007) has gone so far as to propose that dream construction starts from central emotions. There are also some physiological studies that attempt to implicate emotions in REM sleep (Benca et al., 2000, Dang-Vu et al., 2007, Mancia, 2005). These observations are consistent with the typical prevalence of negative emotions in REM dreams (Roussy, Raymond, & De Koninck, 2000).

Emotions in dreams can be evaluated by human judges who read the transcript of dreams or by the dreamer. Studies have shown discrepancies between human judges and dreamer in estimating levels of emotions (Fosse, Stickgold, & Hobson, 2001). For example, dreamers attribute more emotions to their dreams than human judges. The Hall and Van de Castle system has five subscales: Happiness, Apprehension, Anger, Sadness and Confusion, which have been derived after extensive research (reviewed by Domhoff (2003b)). Given the complexity of these dimensions, however, we have chosen to start with two four-level scales to score the global positive and negative emotional tone of dreams in an attempt to model the human judge’s scoring, using machine learning. These scores are global measures of the degree of an emotion within a dream which may rely on the dynamics of emotions within the narrative.

In a first study, experienced dream researchers scored the emotional tone of 100 dreams. With disagreements rarely more than one level apart, it was found that the degree of agreement for negative sentiment of dream reports was 81% (MSE = .19) and for positive sentiment it was 58% (MSE = .54) (Nadeau, Sabourin, De Koninck, Matwin, & Turney, 2006). Therefore, further work concentrated on the negative scale. An application called Linguistic Inquiry and Word Count (LIWC) was first used to identify words pertaining to expression of positive and negative emotions in each dream report. The frequency of such words was used to generate additional attributes using mathematical functions; log ratio, square and square root. These attributes were used to train a linear regression model to predict negative affect score of a human judge on a subset of all dream reports, and tested against the remaining dream reports. The model proved to be efficient with an accuracy of 48% (MSE = .608) (Nadeau et al., 2006). The use of individual words as attributes in modeling a dream report was in line with other dream content analysis research (Bulkeley & Domhoff, 2010). In a second study, Razavi et al. (2008) recognized that a more accurate model could be implemented by including patterns of emotional fluctuation; deemed a good source of attributes as per Hartmann’s (2007) theory of emotionally driven dream. As such, attributes of emotional fluctuations, such as frequency of rises/falls of emotions, length of rise/falls, and maximum and minimum, were quantified and used in the model building. These attributes were recognized as Emotion Progression attributes. Finally in the same study, it was suspected that words may be linked to other sources (Barcaro et al., 2005) and to other words through proximal co-occurrence. Hence, the number of words separating any two words was used as a measure of their semantic closeness. In this manner, a new set of attributes was generated and used in this study. In conjunction with all these extracted attributes, the dreamer’s own report of Joy, Fear and Anxiety, and Hall and Van de Castle’s dimensions of Anger, Apprehension, Happiness, Sadness and Confusion were included. Where Hall and Van de Castle recognized the presence or absence of these dimensions, here the dreamers were asked to report the emotional intensities on a 4 level scale. In this experiment, the model produced an accuracy of 59% for negative sentiment. Matwin, Razavi, De Koninck, and Amini (2010) took one step further by incorporating co-occurrence vectors and advance machine learning techniques to achieve an agreement of 64% on the negative scale.

In the present study, we improve upon Razavi et al.’s (2008) model by replacing their word co-occurrence vectors (established using the dream reports under analysis) with word associations (established from external source). This is based on the notion that associative memory is used by dreamers when they report, and by human judges when they read and score for emotional tone. In this process it is assumed that there is communication of meaning through written words. This shared meaning exhibits itself in part in the network of word associations. Silberman, Bentin, and Miikkulainen (2007) suggest that these associations form over our life time, as semantic and episodic factors coincide. These associations can be observed to be syntagmatically structured in childhood and paradigmatically structured in adulthood (Gomes, 1995). In analyzing dream content of adults, the network of word associations can be built using words expressed in dictionary and encyclopedia definitions, as a whole, in order to better approximate paradigmatic word association. Alternatively, sequential proximity of words in definitions would better approximate syntagmatical structure which would benefit analysis of dream content for children. The frequency of word-coincidences can be a good basis for word association, although high frequencies do not necessarily mean high word association as suggested in the Search of Associative Memory (SAM) Model of semantic memory (Nelson and McEvoy, 2000, Raaijmakers and Schiffrin, 1981). As such, in this study, we normalized these frequencies starting with the highly frequent words first.

Incorporation of word-association inherently demands a different data structure than those of the past. Recently, word search (Bulkeley & Domhoff, 2010), word-strings (Domhoff & Schneider, 2008) and bag of words (Nadeau et al., 2006) provided means to identify and count specific occurrences. With the assumption that there is a hierarchical structure in meaning (Murphy & Lassaline, 1997), these frequencies were then used for grouping words to form discrete categories (Bulkeley, 2009) or fuzzy categorical membership (Frantova & Bergler, 2009) which were used to measure specific aspects: personality, emotions, etc. (Hall & Van de Castle, 1966). These aspects are quantified by means of search, which have the strength of objectivity, but suffer from narrow scope of observation due to discretization. It is proposed that word-association can open up this scope for a greater perspective on dream content. One dream report, as represented by a network of associations, can be compared to another’s for similarities and differences using machine learning algorithms. In the former paradigm, one is required to search for specific words in order to quantify and compare frequencies. That approach is limited by its narrow scope; by focusing on a few variables, one has to ignore other potential variables. In the proposed paradigm, differences in the network of associations can be identified using machine learning, relinquishing the need to focus on specific items.

Therefore, it was hypothesized that, with the greater scope of and access to implicit information provided by word associations, the accuracy in automatic scoring of the emotional tone of dreams as assessed by human judges will improve. More specifically, it was predicted that the inclusion of associated words would increase the percentage of agreement between machine and human judges.