What to believe: Bayesian methods for data analysis

doi:10.1016/j.tics.2010.05.001

Trends in Cognitive Sciences

Volume 14, Issue 7, July 2010, Pages 293-300

https://doi.org/10.1016/j.tics.2010.05.001 Get rights and content

Although Bayesian models of mind have attracted great interest from cognitive scientists, Bayesian methods for data analysis have not. This article reviews several advantages of Bayesian data analysis over traditional null-hypothesis significance testing. Bayesian methods provide tremendous flexibility for data analytic models and yield rich information about parameters that can be used cumulatively across progressive experiments. Because Bayesian statistical methods can be applied to any data, regardless of the type of cognitive model (Bayesian or otherwise) that motivated the data collection, Bayesian methods for data analysis will continue to be appropriate even if Bayesian models of mind lose their appeal.

Section snippets

Cognitive science should be Bayesian even if cognitive scientists are not

An entire issue of Trends in Cognitive Sciences was devoted to the topic of Bayesian models of cognition [1] and there has been a surge of interest in Bayesian models of perception, learning and reasoning 2, 3, 4, 5, 6. The essential premise of the Bayesian approach is that the rational, normative way to adjust knowledge when new data are observed is to apply Bayes’ rule (i.e. the mathematically correct formula) to whatever representational structures are available to the reasoner. The promise

Null hypothesis significance testing (NHST)

In NHST, after collecting data, a researcher computes the value of a summary statistic such as t or F or χ², and then determines the probability that so extreme a value could have been obtained by chance alone from a population with no effect if the experiment were repeated many times. If the probability of obtaining the observed value is small (e.g. p < 0.05), then the null hypothesis is rejected and the result is deemed significant.

Bayesian data analysis

In Bayesian data analysis, the researcher uses a descriptive model that is easily customizable to the specific situation without the computational restrictions in conventional NHST models. Before considering any newly collected data, the analyst specifies the current uncertainty for parameter values, called a prior distribution, that is acceptable to a skeptical scientific audience. Then Bayesian inference yields a complete posterior distribution over the conjoint parameter space, which

Models of cognition and models of data

The posterior distribution of a Bayesian analysis only tells us which parameter values are relatively more or less credible within the realm of models that the analyst cares to consider. Bayesian analysis does not tell us what models to consider in the first place. For typical data analysis, descriptive models are established by convention: most empirical researchers are familiar with cases of the generalized linear model 30, 31 such as linear regression, logistic regression and ANOVA. Given

Glossary

Analysis of variance (ANOVA): when metric data (e.g. response times) are measured in each of several groups, traditional ANOVA decomposes the variance among all data into two parts: the variance between group means and the variance among data within groups. The underlying descriptive model can be used in Bayesian data analysis.
Bayes’ rule: a simple mathematical relationship between conditional probabilities that relates the posterior probability of parameter values, on the one hand, to the

References (46)

F. De Santis
Statistical evidence and sample size determination for Bayesian hypothesis testing
J. Stat. Plan. Infer.
(2004)
C.C. Liu et al.
Bayes factors: prior sensitivity and model generalizability
J. Math. Psychol.
(2008)
D.J. Navarro
Modeling individual differences using Dirichlet processes
J. Math. Psychol.
(2006)
N. Chater
Probabilistic models of cognition
Trends Cogn. Sci.
(2006)
J.R. Anderson
The Adaptive Character of Thought
(1990)
T.L. Griffiths
Bayesian models of cognition
J.K. Kruschke
Bayesian approaches to associative learning: from passive to active learning
Learn. Behav.
(2008)
W. Edwards
Bayesian statistical inference for psychological research
Psychol. Rev.
(1963)

M.D. Lee et al.

Bayesian statistical inference in psychology: comment on Trafimow (2003)

Psychol. Rev.

(2005)

E.J. Wagenmakers

A practical solution to the pervasive problems of p values

Psychon. Bull. Rev.

(2007)

Kruschke, J.K. (2010) Bayesian data analysis. Wiley Interdisciplin. Rev. Cogn. Sci. DOI:...

J.O. Berger et al.

Statistical analysis and the illusion of objectivity

Am. Sci.

(1988)

D.V. Lindley et al.

Inference for a Bernoulli process (a Bayesian view)

Am. Stat.

(1976)

Maxwell, S.E. and Delaney, H.D. (2004) Designing Experiments and Analyzing Data: A Model Comparison Perspective (2nd...

J. Miller

What is the probability of replicating a statistically significant effect?

Psychon. Bull. Rev.

(2009)

M.D. Lee

Three case studies in the Bayesian analysis of cognitive models

Psychon. Bull. Rev.

(2008)

W. Vanpaemel

BayesGCM: software for Bayesian inference with the generalized context model

Behav. Res. Methods

(2009)

M.D. Lee

BayesSDT: software for Bayesian inference with signal detection theory

Behav. Res. Methods

(2008)

J.N. Rouder

A hierarchical process-dissociation model

J. Exp. Psychol.

(2008)

M.D. Lee et al.

A model of knower-level behavior in number concept development

Cogn. Sci.

(2010)

J.K. Kruschke

Doing Bayesian Data Analysis: A Tutorial with R and BUGS

(2010)

Cited by (239)

Comprehensive comparison of aroma profiles and chiral free and glycosidically bound volatiles in Fujian and Yunnan white teas
2024, Food Chemistry
The Fujian and Yunnan provinces in China are the most representative origins of white tea. However, the key differences in the chemical constituents of the two white teas have rarely been revealed. In this study, a comprehensive comparison of the aroma profiles, chiral volatiles, and glycosidically bound volatiles (GBVs) in Fujian and Yunnan white teas was performed, and 174 volatiles and 28 enantiomers, including 22 volatiles and six GBVs, were identified. Linalool, linalyl-β-primeveroside (LinPrim), and α-terpineol presented the opposite dominant configurations in Fujian and Yunnan white teas, and the chiral GBVs were firstly quantified with significant differences in the contents of R-LinPrim and β-d-glucopyranosides of (2R, 5R)-linalool oxide A and (2R, 5S)-linalool oxide B. Moreover, discrimination functions for Fujian and Yunnan white teas were created using nine key variables with excellent reliability and efficiency. These results provide a new method for objectively distinguishing authentic white teas according to geographical origin.
Guidelines for balancing the number of trials and the number of subjects to ensure the statistical power to detect variability – Implication for gait studies
2024, Journal of Biomechanics
Variability is one of the most crucial outcomes in human movement studies: variance and standard deviation of various parameters have been reported in numerous studies. However, in many of these studies, the numbers of trials and subjects have been intuitively determined and not justified with statistical considerations. Here, we investigated the impact of the numbers of trials and subjects on statistical power, based on the assumption that results per trial follow a normal distribution, using mathematical analysis and numerical simulation. An inverse-like relationship was observed between the number of trials and subjects required to ensure the statistical power for detecting differences in variance between subject groups or conditions. For instance, assuming a 1.2-times difference in population variance between pre-and post-training sessions as an alternative hypothesis, our simulation demonstrated that combinations of the number of subjects and trials, such as measuring 100 trials from each of 12 subjects under each condition, or measuring 20 trials from each of 60 subjects, can guarantee an 80 % of statistical power. Planning research based on such mathematical considerations will enable meaningful statistical interpretations in studies focusing on movement variability, such as gait studies.
A new way of classifying developmental prosopagnosia: Balanced Integration Score
2024, Cortex
Despite severe everyday problems recognising faces, some individuals with developmental prosopagnosia (DP) can achieve typical accuracy scores on laboratory face recognition tests. To address this, studies sometimes also examine response times (RTs), which tend to be longer in DPs relative to control participants. In the present study, 24 potential (according to self-report) DPs and 110 age-matched controls completed the Cambridge Face and Bicycle Memory Tests, old new faces task, and a famous faces test. We used accuracy and the Balanced Integration Score (BIS), a measure that adjusts accuracy for RTs, to classify our sample at the group and individual levels. Subjective face recognition ability was assessed using the PI20 questionnaire and semi structured interviews. Fifteen DPs showed a major impairment using BIS compared with only five using accuracy alone. Logistic regression showed that a model incorporating the BIS measures was the most sensitive for classifying DP and showed highest area under the curve (AUC). Furthermore, larger between-group effect sizes were observed for a derived global (averaged) memory measure calculated using BIS versus accuracy alone. BIS is thus an extremely sensitive novel measure for attenuating speed-accuracy trade-offs that can otherwise mask impairment measured only by accuracy in DP.
ABA, ABC, and AAB renewal effects and individual differences of learning on fear conditioning using social stimuli
2023, Learning and Motivation
We examined whether ABA, ABC, and AAB renewal effects could be observed using fear conditioning with social stimuli as both conditioned stimulus (CS) and unconditioned stimulus (US) to investigate the possibility of relapse following exposure interventions for social anxiety in a laboratory setting. Sixty-four participants were randomly allocated to four groups (AAA, ABA, ABC, and AAB). They received nine pairings with a neutral face (CS) and aggressive facial expressions and insults (US) during the acquisition phase, and then received 18 pairings with the same CS and a neutral comment in the extinction phase. Following the extinction, the testing phase (three trials) was conducted. The background colors presented in each phase were manipulated to serve as the contextual stimuli. ANOVA showed the occurrence of the three types of renewals, but there were no consistent results showing differences between the renewals. Bayesian modeling demonstrated that there are some relationships between each parameter in the learning curves and social anxiety. The participants were non-clinical populations; therefore, it remains unclear whether the findings can be generalized to clinical populations. The results suggest that relapse following exposure therapy is related to a contextual change from the therapeutic environment and that individual differences in learning are related to social anxiety symptoms.
Spontaneous dyadic behavior predicts the emergence of interpersonal neural synchrony
2023, NeuroImage
Synchronization of neural activity across brains – Interpersonal Neural Synchrony (INS) – is emerging as a powerful marker of social interaction that predicts success of multi-person coordination, communication, and cooperation. As the origins of INS are poorly understood, we tested whether and how INS might emerge from spontaneous dyadic behavior. We recorded neural activity (EEG) and human behavior (full-body kinematics, eye movements, and facial expressions) while dyads of participants were instructed to look at each other without speaking or making co-verbal gestures. We made four fundamental observations. First, despite the absence of a structured social task, INS emerged spontaneously only when participants were able to see each other. Second, we show that such spontaneous INS, comprising specific spectral and topographic profiles, did not merely reflect intra-personal modulations of neural activity, but it rather reflected real-time and dyad-specific coupling of neural activities. Third, using state-of-art video-image processing and deep learning, we extracted the temporal unfolding of three notable social behavioral cues – body movement, eye contact, and smiling – and demonstrated that these behaviors also spontaneously synchronized within dyads. Fourth, we probed the correlates of INS in such synchronized social behaviors. Using cross-correlation and Granger causality analyses, we show that synchronized social behaviors anticipate and in fact Granger cause INS. These results provide proof-of-concept evidence for studying interpersonal neural and behavioral synchrony under natural and unconstrained conditions. Most importantly, the results suggest that INS could be conceptualized as an emergent property of two coupled neural systems: an entrainment phenomenon, promoted by real-time dyadic behavior.
The Value of Bayesian Methods for Accurate and Efficient Neuropsychological Assessment
2022, Journal of the International Neuropsychological Society

View all citing articles on Scopus

View full text

Trends in Cognitive Sciences

OpinionWhat to believe: Bayesian methods for data analysis

Section snippets

Cognitive science should be Bayesian even if cognitive scientists are not

Null hypothesis significance testing (NHST)

Bayesian data analysis

Models of cognition and models of data

Glossary

J. Stat. Plan. Infer.

J. Math. Psychol.

J. Math. Psychol.

Probabilistic models of cognition

Trends Cogn. Sci.

The Adaptive Character of Thought

Bayesian models of cognition

Bayesian approaches to associative learning: from passive to active learning

Learn. Behav.

Bayesian statistical inference for psychological research

Psychol. Rev.

Bayesian statistical inference in psychology: comment on Trafimow (2003)

Psychol. Rev.

A practical solution to the pervasive problems of p values

Psychon. Bull. Rev.

Statistical analysis and the illusion of objectivity

Am. Sci.

Inference for a Bernoulli process (a Bayesian view)

Am. Stat.

What is the probability of replicating a statistically significant effect?

Psychon. Bull. Rev.

Three case studies in the Bayesian analysis of cognitive models

Psychon. Bull. Rev.

BayesGCM: software for Bayesian inference with the generalized context model

Behav. Res. Methods

BayesSDT: software for Bayesian inference with signal detection theory

Behav. Res. Methods

A hierarchical process-dissociation model

J. Exp. Psychol.

A model of knower-level behavior in number concept development

Cogn. Sci.

Doing Bayesian Data Analysis: A Tutorial with R and BUGS

Opinion
What to believe: Bayesian methods for data analysis