Comparing transcranial direct current stimulation and transcranial random noise stimulation over left dorsolateral prefrontal cortex and left inferior frontal gyrus: Effects on divergent and convergent thinking

Introduction

The essential role of creativity has been highlighted in several areas, including science (Wilcox et al., 2018), mathematics (Mann, 2006), organizations (Ionescu et al., 2012), or economic and social development (Tremblay and Pilati, 2013) among others. Therefore, any attempt to promote it is a highly appreciated aim in science (Plucker et al., 2020).

According to several authors, there are two main components in creativity (Guilford, 1950; Cropley, 2006; Pick and Lavidor, 2019); convergent thinking (CT) and divergent thinking (DT). CT requires using deductive reasoning to find a single solution to a closed-ended problem (Zmigrod et al., 2015). According to many authors (Chermahini and Hommel, 2010, 2012; Dewhurst et al., 2011; Hommel et al., 2011; Colzato et al., 2013a,b, 2017; Sellaro et al., 2014; Mayseless and Shamay-Tsoory, 2015; Yamada and Nagai, 2015; Hoşgören Alici et al., 2019), the remote associates test (RAT) (Mednick, 1962) seems to be the most used instrument for CT assessment in the literature. However, although RAT seems to measure mainly CT, there is evidence that, to some extent, it may be reflecting also DT (Cortes et al., 2019).

Divergent thinking, on the other hand, is postulated as a type of thinking that produces multiple alternative and original responses to an open-ended problem (Guilford, 1967). Although many studies have used Alternative Uses Task (Guilford, 1967) or the Torrance Test of Creative Thinking (TTCT) (Torrance, 1966) to assess DT, similar to what has been previously mentioned with RAT, there is evidence suggesting that they also require CT processes (Cortes et al., 2019).

In recent years the interest to disentangle the neural underpinnings of creativity has increased (Weinberger et al., 2017). In this context, both neuroimaging studies neuroimaging (Gonen-Yaacovi et al., 2013; Beaty et al., 2015, 2016, 2019, 2021; Chrysikou, 2019; Sun et al., 2019) and transcranial stimulation studies have investigated this issue (Weinberger et al., 2017). A meta-analysis of neuroimaging studies (Boccia et al., 2015) suggests that the areas most related to DT tasks were the dorsolateral prefrontal cortex (DLPFC), inferior frontal gyrus (IFG), postcentral, and supramarginal gyri, inferior parietal lobule, insula, temporal gyrus (middle and superior), and middle occipital gyrus.

Results from transcranial stimulation studies mainly indicate that anodal tDCS over left DLPFC may increase DT (Colombo et al., 2015; Zmigrod et al., 2015; Peña et al., 2021) and CT (Cerruti and Schlaug, 2009; Metuki et al., 2012; Zmigrod et al., 2015; Peña et al., 2021, 2019). However, in separate studies, cathodal tDCS over the left IFG also found an increase in DT (Mayseless and Shamay-Tsoory, 2015; Kleinmintz et al., 2018; Hertenstein et al., 2019; Ivancovsky et al., 2019; Khalil et al., 2020; Chrysikou et al., 2021; Kenett et al., 2021).

Although less studied than tDCS, transcranial random noise stimulation (tRNS) in the high frequency range (100–500 Hz) has been used as a promising alternative to tDCS with the difference of being excitatory in both electrodes (Terney et al., 2008). Although the mechanisms for tRNS are not still completely understood, there are two main hypotheses. One hypothesis proposes that it induces a repetitive opening of the Na⁺ channels and therefore shortens the hyperpolarization phase (Terney et al., 2008; Chaieb et al., 2015). Another possible hypothesis suggests that the neuronal excitability increases through stochastic resonance, a phenomenon whereby the accumulation of random interference (i.e., noise) can increase the detection of weak stimuli or enhance the information content of a signal (Moss et al., 2004; Ward, 2009; Miniussi et al., 2013; van der Groen and Wenderoth, 2016).

Previous studies indicate that tRNS may provide additional advantages over tDCS. For example, a meta-analysis (Simonsmeier et al., 2018) showed that the effect of tRNS on language and mathematics was stronger compared to tDCS. Some authors have suggested that the effect of tRNS on visual perceptual learning (Terney et al., 2008) and working memory (Murphy et al., 2020) may be larger when compared to tDCS. Similarly, it is also suggested that tRNS may show longer term effects than tDCS (Snowball et al., 2013; Brevet-Aeby et al., 2019; Berger et al., 2021). Inukai et al. (2016) compared tDCS and tRNS for increasing cortical excitability and reported that tRNS produced the most significant increase. Vanneste et al. (2013) directly compared the response of patients with tinnitus after using tDCS, transcranial alternating current stimulation and tRNS techniques, showing that tRNS was superior to both tDCS and transcranial alternating current stimulation. Additionally, the possible adverse effects seem to be more tolerable after receiving tRNS (Fertonani et al., 2011). Moreover, tRNS is not as perceptible as tDCS regarding skin perception (Ambrus et al., 2010). In creativity, previous studies using only tRNS have shown a significant improvement in visual DT (originality) and RAT after posterior parietal cortex stimulation (Peña et al., 2020), and an improvement in RAT, unusual uses (fluency and originality) after stimulating DLPFC (Peña et al., 2019).

On the other hand, previous literature suggests that cognitive flexibility, measured with Stroop task, generally shows positive correlations with DT performance (Groborz and Necka, 2003; Edl et al., 2014; Xuejun et al., 2014; Sharma and Babu, 2017). Results from tDCS studies on cognitive flexibility suggest that the anodal stimulation of the DLPFC improves cognitive flexibility performance (Metuki et al., 2012; Dajani and Uddin, 2015; Borwick et al., 2020; Perrotta et al., 2021). However, previous studies have not tested if the effect of tDCS on creativity after targeting the DLPFC is partially due to the positive effect of tDCS on cognitive flexibility. In other words, we do not know if the improvement in cognitive flexibility after DLPFC partially mediates the impact on creativity enhancement.

The first objective of this study was to explore if tDCS with anode over the left DLPFC and cathode over the left IFG improves both CT and DT compared to sham. Additionally, we wanted to explore the effect of excitatory tRNS over the same brain areas. Given that the left DLPFC is related to cognitive control, we hypothesize that its stimulation would facilitate CT through the maintenance of focused attention and top-down support for relevant information (Fischer and Hommel, 2012; Zhang et al., 2020). Therefore, we hypothesize that both tDCS and tRNS groups will enhance CT due to the stimulation of left DLPFC compared to sham. Additionally, we hypothesize that tDCS group will relax top-down inhibitory constraints and improve bottom-up information processing due to the inhibition of left IFG with cathodal stimulation (Chrysikou, 2019), which in turn will improve DT compared to sham. The tRNS group, on the other hand, will have excitatory effects on both left DLPFC and left IFG. Finally, we hypothesize that the effect of tDCS and tRNS on creativity will be partially mediated by the enhancement in cognitive flexibility produced by the transcranial stimulation.

Materials and methods

Participants

We recruited 66 healthy and native Spanish speaking volunteers (aged 18 years or above) from the general population without restrictions on gender or handedness of participants. Participants fulfilled a screening questionnaire for transcranial electrical stimulation contra-indications that included: (1) previous history of brain surgery; (2) being pregnant; (3) suffering from frequent or severe headaches or migraines; (4) previous history or presence of neurological disorder or injury (epileptic or convulsive seizure, brain stroke, severe brain injury); and (5) presence of any brain metallic implant. The study obtained the ethical approval from the Research Ethics Committee of Deusto University (Ref: ETK-31/17-18). Participants did not receive any course credit or monetary or compensation for participating in the study. All volunteers provided written informed consent to participate in the study and they were free to withdraw at any time. All experimental procedures were conducted in accordance with the Declaration of Helsinki (2013).

Design and procedure

This was a double-blind, sham-controlled, parallel-group between-subjects design study and consisted of one single session. Figure 1 shows the study design and procedure. The participants were randomly assigned to one of the three groups (n = 22 in each group): tDCS group, tRNS group or sham group. Group assignment was based on a computer-generated randomization software.¹ All raters were blind to stimulation group condition.

Figure 1. Study design. Schematic representation of the experimental procedure for the creativity assessments. RAT, remote associates test; PC, picture completion task; UU, unusual uses; tDCS, transcranial direct current stimulation; tRNS, transcranial random noise stimulation.

After signing the consent form, participants reported sociodemographic information along with tobacco consumption, hours of sleep and stimulant drinks ingested before the session.

The creativity assessment at baseline was carried out just before starting stimulation (see Figure 1). Participants were given 2 min and 45 s to complete the RAT and 2 min each for the UU and PC from the Torrance Test of Creative Thinking Test. Afterward, the participants underwent the Stroop test. They had 45 s for each Stroop condition (words, colors, and color-words).

Five minutes after the stimulation started, the participants were assessed with the parallel versions of RAT, UU, and PC with the same time limitations as the baseline assessment. The order of the version of UU, RAT, and PC were counterbalanced. Afterward, they completed again Stroop test. Participants filled the adverse effects questionnaire. In order to study the blinding efficacy, participants were asked to answer the following question: “Please, tell us if you think you were receiving real stimulation, no stimulation (placebo) or you do not know?”

Application of transcranial direct current stimulation and transcranial random noise stimulation

Transcranial direct current stimulation and tRNS were delivered by using a wireless battery-operated Starstim8 device (Neuroelectrics Inc., Barcelona, Spain) attached to the back of a neoprene cap that follows the International 10–20 system. In the tDCS group, the anode was placed over the left DLPFC (F3 according to the 10/20 electrode placement EEG-System) and the cathode over the left posterior IFG (F7). In the tRNS group, the position of the electrodes (anodes) was the same as in tDCS group. Both real tDCS and tRNS groups received 20 min (with 30 s ramp up/down) of 1.5 mA tDCS or tRNS (high-frequency: 100–500 Hz) via two saline-soaked (5 ml per sponge) circular rubber-sponge electrodes (area of 8 cm²). The sham group received 30 s (with 30 s ramp-up/down) of real stimulation and kept the cap for 20 min as well. The impedance of the electrodes was checked before and during the stimulation to guarantee that it was maintained below 10 kΩ.

The stimulation protocol was created and monitored using the NIC 2.0 software.² Stimulation groups were labeled as “Group A,” “Group B,” and “Group C” in the NIC2 software with the double-blind mode enabled. Therefore, the experimenters that applied the stimulation conditions were also blinded.

Measures

The Torrance Test of Creative Thinking

We included unusual uses (UU) and picture completion (PC) subtests from The Torrance Test of Creative Thinking (Torrance, 1966). We included two different forms (Form A and B) for the baseline and during the stimulation assessments. We measured three dimensions for both UU and PC: fluency, originality, and flexibility.

In the PC task the participants are requested to complete ten unfinished figures by drawing additional elements in a paper and pencil task. Fluency was assessed as the total number of appropriate responses and the participants were given 1 point for each figure completed. The originality score was based on the statistical infrequency of each response based on the list of normative data (Torrance, 1966). They were given 1 point for each response considered original. Flexibility was assessed as the number of different ideational categories produced in the pictures, based on the list of categories from the Spanish adaptation of the TTCT (Jiménez et al., 2007). Fluency, originality and flexibility measures were converted to z-scores to obtain a PC composite. The internal consistency was good (Cronbach’s alpha = 0.76).

In the UU task, participants had to write down as many unusual uses as possible for an item. In the Form A of the test Cardboard Boxes was used as a stimulus. In the Form B Tin Cans was used. We measured three dimensions in UU: Fluency, Originality, and Flexibility. Fluency was obtained considering the number of different unusual uses produced (1 point for each response). Originality was based on the statistical unusualness of each response. We used the criteria based on the list of items from the manual (Torrance, 1966). A flexibility score was obtained from the number of different categories represented in the responses. Each different category was given 1 point. We also converted fluency, originality and flexibility measures to z-scores to obtain a UU composite. The internal consistency was good (Cronbach’s alpha = 0.86).

Remotes associates test

The Spanish version of the RAT (Mednick, 1962) was administered. Two different forms of the test were used for the baseline and during stimulation assessment. In RAT task, participants were asked to identify a word that is associated (either forming a compound word or semantically related) with three cue words. Each form included 30 items and participants had 2 min and 45 s for write down as many items as possible. The items were presented in the same sheet and participants could go backward and forward if they wished. The internal consistency of the test was high (Cronbach’s alpha = 0.81).

Stroop test

The Spanish version of the Stroop Test (Mednick, 1962; Golden, 2010) was administered. The Stroop test is a neuropsychological assessment tool composed of three parts: word, color, and word-color, each lasting 45 s. It is a widely used test for the assessment of processing speed (word subtest and color subtests) and cognitive flexibility (word-color and interference) in both clinical and healthy populations. It has a high internal consistency (Cronbach’s alpha = 0.80).

Questionnaire of adverse effects

A questionnaire with 11 items was used to measure any perceived side effects. We included numbness, tingling, skin redness, headache, itching sensation, concentration difficulties, burning, phosphenes, mood change, sore throat, and scalp pain study.

Statistical analyses

Baseline characteristics were compared using the ANOVA test for continuous variables and X² test for categorical data. Baseline correlation analyses were performed with Pearson’s R or Spearman’s Rho in case of non-normal distribution. Analysis of covariance (ANCOVA) was used to compare during stimulation scores (controlling for baseline scores) between the three groups for each of the creativity variables and Stroop (Wan, 2021). Effect size ($n_p^2$) was calculated for ANCOVA analyses. IBM SPSS software version 23.0 (IBM Corp. Released, 2015) was used for statistical analyses. All tests were two-tailed and the significance level was set at 0.05.

Results

Baseline characteristics of the groups

There were no significant differences between groups in any of the variables assessed at baseline (see Table 1). Participants were also asked to indicate if the number of drinks with stimulants ingested, number of hours slept, or number of tobacco consumption was less than usual, more than usual or the same as usual. There were no significant differences between the groups in sleeping hours [X² (4, N = 66) = 6.46, p = 0.167], stimulant drinks [X² (4, N = 66) = 3.41, p = 0.490], or tobacco consumption [X² (4, N = 66) = 4.00, p = 0.406].

Table 1. Participant characteristics of transcranial direct current stimulation (tDCS), transcranial random noise stimulation (tRNS), and sham groups at baseline.

The RAT, UU, and PC scores of tDCS, tRNS, and sham groups at baseline and during stimulation are shown in Table 2 whereas the scores in Stroop Test are shown in Figure 2.

Table 2. Creativity scores of transcranial direct current stimulation (tDCS), transcranial random noise stimulation (tRNS), and sham groups at baseline and during stimulation.

Figure 2. Stroop scores of transcranial direct current stimulation (tDCS), transcranial random noise stimulation (tRNS), and sham group at baseline and during stimulation.

Stimulation effects on remote association test, unusual uses, and picture completion

The ANCOVA results are shown in Table 3. RAT scores did not reach a statistically significant results in overall ANCOVA (F = 2.65, p = 0.079). Exploratory post-hoc analyses indicated that the comparison between tDCS and sham group was significant (p = 0.038) and between tRNS and sham was marginally significant (p = 0.071). We must take these exploratory results with cautious, since the overall ANCOVA was not significant.

Table 3. Differences among groups in remote association test (RAT), unusual uses (UU), and picture completion (PC) scores during stimulation after controlling for baseline scores.

Regarding PC subdomains, we found significant differences among the groups in PC fluency (F = 3.35, p = 0.042) and originality (F = 3.35, p = 0.042). Post-hoc analyses revealed that tRNS group scored significantly higher than sham on fluency (p = 0.012) and originality (p = 0.010) whereas there were no significant differences among the rest of comparisons. Finally, the composite PC score was significantly different between groups (F = 3.35, p = 0.042). Post-hoc analyses indicated that the tRNS group (0.27 ± 0.11 standard error) scored significantly higher than sham group (−0.26 ± 0.14 standard error, p = 0.012) but not higher than tDCS group (−0.02 ± 0.15 standard error, p = 0.162).

Regarding UU scores, contrary to expected, we did not find any significant difference in any of the domains analyzed, nor in the UU composite score.

Based on the fluency scores, we calculated the percentage of original responses (number of original responses × 100/total number of correct responses) for both PC and UU. Although the number of original responses was significant for PC, there were not significant differences in the percentage of original responses in PC [F(2,62) = 1.72, p = 0.188]. Similarly, results in the percentage of original responses in UU showed no significant differences [F(2,62) = 0.42, p = 0.658].

Stimulation effects on Stroop test

We did not find any significant effect on any of the measures of Stroop Test (see Table 4). Therefore, we could not further test if the improvement in creative performance was partially due to an improvement in cognitive flexibility.

Table 4. Differences among groups in Stroop scores during stimulation after controlling for baseline scores.

Correlation analyses between Stroop performance and creativity measures at baseline

Table 5 shows the correlation analyses between Stroop test and creativity scores at baseline in the whole sample. UU dimension of fluency, originality, and flexibility were significantly correlated with Stroop-W, Stroop-C, and Stroop-WC subtests, whereas interference score did not correlate with any of the creativity measures included.

Table 5. Baseline correlations between Stroop scores and divergent thinking (DT) and convergent thinking (CT) tasks.

Adverse effects and blinding

None of the participants reported having experienced any significant adverse effects that made them leave the study. There were significant differences among the groups in minor adverse effects [F(2,62) = 3.46, p = 0.035]. Post-hoc analyses indicated that tDCS group (M = 2.18, SD = 1.70) reported more adverse effects than tRNS (M = 1.29, SD = 1.34, p = 0.046) and sham groups (M = 1.29, SD = 1.34, p = 0.015). We additionally investigated if the total number of adverse effects was correlated with the change in creative scores. However, we did not find any significant effect of adverse effects in any of the creativity changes analyzed nor in the whole sample (p-values ranged from 0.193 to 0.970) neither in the stimulation groups (p-values ranged from 0.133 to 0.909).

We did not find significant differences in stimulation guess between real and sham conditions [χ² (2, N = 66) = 5.78, p = 0.216]. From the tDCS group, 36.4% guessed that they had received stimulation, 22.7% guessed they had received the placebo and 40.9% were undecided. From the tRNS group, 9.1% guessed that they had received stimulation, 40.9% guessed they had received the placebo, and 50.0% were undecided. Finally, from the sham group, 34.8% guessed that they had received the placebo, 27.3% that they had received stimulation, and 33.3% were undecided. We did not find any significant difference in creative performance changes between those who guessed they were receiving real stimulation compared to those guessing they received sham or they did not know (p-values ranged from 0.181 to 0.990).

Discussion

This study compared the effect of tDCS and tRNS over the left DLPFC and left IFG simultaneously on DT and CT performance. The results showed that fluency and originality dimensions of visual DT benefited from tRNS stimulation compared to sham, whereas contrary to expected, tDCS did not show any significant effect on DT. The positive effect of tRNS over the left DLPFC on the fluency dimension of DT is to some extent consistent with previous literature (Lustenberger et al., 2015; Zmigrod et al., 2015; Grabner et al., 2018; Peña et al., 2019, 2021). However, the significant effect of tRNS on originality was not expected since some of the previous studies have shown even a reduction in novelty of responses after the excitatory stimulation of the left IFG, which was what the tRNS group received (Ivancovsky et al., 2019; Kenett et al., 2021). We initially hypothesized that the anodal stimulation with tDCS over the left DLPFC with the simultaneous cathodal stimulation over the left IFG would enhance both verbal and visual DT more than the stimulation of both DLPFC and IFG with tRNS, since previous studies have shown independently that the stimulation of the left DLPFC is associated with DT (Colombo et al., 2015; Zmigrod et al., 2015) and separate studies using cathodal stimulation over the left IFG have also shown an improvement in DT (Chrysikou et al., 2013; Mayseless and Shamay-Tsoory, 2015; Hertenstein et al., 2019; Ivancovsky et al., 2019; Khalil et al., 2020). However, the current results suggest that it was tRNS and not tDCS that enhanced DT. A possible reason for this result may be that the simultaneous stimulation of both left DLPFC and IFG with tRNS may have had a positive on DT by enhancing one×s thoughts toward a specific aim under an increase in selection demands, possibly mediated by the left DLPFC (Peña et al., 2019). The left DLPFC has been related to executive functioning and working memory (Andrews et al., 2011), which in turn affect cognitive control (Andrews et al., 2011). Therefore, the stimulation of left DLPFC may enhance DT through the maintenance of focused-attention and inhibiting task-irrelevant information (Beaty and Schacter, 2018). An additional evidence for this possible explanation comes from the results obtained in the percentage of original responses, which showed not significant results. Therefore, the main effect of tRNS may have been on the fluency dimension, which in turn may have produced more total original ideas, but not a higher proportion of original ideas.

Regarding CT, our results were only marginally significant and it may mean a lack of statistical power, so we cannot conclude that tDCS neither tRNS over left DLPFC and IFG do significantly affect RAT scores. However, the trend toward a better performance after both tDCS and tRNS would be consistent with many studies that have shown that the stimulation of the left DLPFC is associated with an improvement in RAT using both tDCS (Cerruti and Schlaug, 2009; Zmigrod et al., 2015) and tRNS (Peña et al., 2019, 2021). A possible explanation for this result is that the stimulation of left DLPFC would enhance RAT performance by strengthening the maintenance of focused attention, top-down support for relevant information, manipulation of information in working memory and inhibition of task-irrelevant information (Fischer and Hommel, 2012; Zhang et al., 2020).

Further exploratory analyses found that cognitive flexibility, measured with Stroop, positively correlated with verbal DT score and its three subdimensions (fluency, originality, and flexibility). This result is consistent with previous literature (Groborz and Necka, 2003; Edl et al., 2014; Xuejun et al., 2014; Sharma and Babu, 2017). However, contrary to expected, we did not find any significant effect of tDCS or tRNS on cognitive flexibility measured with the Stroop Test. Therefore, we could not show that the effect of brain stimulation over the left DLPFC and IFG on creative performance was partially mediated by the enhancement of cognitive flexibility, which in turn would enhance creativity. Previous studies using tDCS over the DLPFC have found a significant effect, revealing the prominent role of DLPFC on cognitive flexibility measured with the Stroop Test (Metuki et al., 2012; Dajani and Uddin, 2015; Borwick et al., 2020; Perrotta et al., 2021).

Our results indicate that tRNS produced less adverse effects compared to tDCS. These results are consistent with previous studies that found that tRNS-induced sensations were less frequently perceived compared to tDCS (Poreisz et al., 2007; Ambrus et al., 2010), especially with itching irritation and burning sensations (Fertonani et al., 2011). Based on these results, some authors suggest that tRNS may be an optimal tool for experimental designs (Fertonani et al., 2011).

Although the interesting results, there are several limitations in this study. Firstly, we included only one item per time point for both UU and PC, which may explain the lack of significant results in verbal DT. Additionally, the RAT measure included mostly associative items but also compound items. Another limitation is that since high frequency tRNS is mainly excitatory, we could not completely compare the effect of tDCS and tRNS, since it is expected to affect in a different way over the left IFG. In other words, the excitatory effect over the left DLPFC is supposed to be similar. However, the cathodal tDCS over the left IFG is suggested to exert an inhibitory effect whereas tRNS is expected to be excitatory. Similarly, we only included two electrodes in the montage. Given the complex nature of creativity, future studies should use multichannel stimulation in order to stimulate cerebral networks instead of discrete brain areas.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Research Ethics Committee of Deusto University (Ref: ETK-31/17-18). The patients/participants provided their written informed consent to participate in this study.

Author contributions

JP: conceptualization, planning, data collection, data analysis, and writing of the manuscript. AS: conceptualization, data collection, supporting data analysis, and proofreading. YB-B and NO: supporting data analysis and proofreading. NI-B: data collection and proofreading. LZ-E: supporting data collection and proofreading. MG-G: conceptualization and supporting data analysis. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by a 2018 Leonardo Grant for Researchers and Cultural Creators (BBVA Foundation), Department of Education and Science of the Basque Government (IT946-16), a Predoctoral Fellowship (Fundación Tatiana Pérez de Guzmán el Bueno), and a Research Staff Training Programme Grant from the University of Deusto (Bilbao, Spain).

Acknowledgments

We thank all the participants in the experiment.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

1. ^ www.randomizer.org
2. ^ www.neuroelectrics.com/products/software/nic2/

References