The English language now dominates scientific communications. Yet, many scientists have English as their second language. Their English proficiency may therefore often be more limited than that of a ‘native speaker’, and their scientific contributions (e.g. manuscripts) in English may frequently contain linguistic features that disrupt the fluency of a reader’s, or listener’s information processing even when the contributions are understandable. Scientific gatekeepers (e.g. journal reviewers) sometimes cite these features to justify negative decisions on manuscripts. Such justifications may rest on (...) the prima facie plausible assumption that linguistic characteristics that hinder fast and easy understandability of scientific contributions are epistemically undesirable in science. I shall raise some doubts about this assumption by drawing on empirical research on processing fluency and its biasing effects. I also argue that directing scientists with English as a foreign language toward approaching ‘native-level’ English can have the negative consequence of reducing their potential to make scientific belief formation more reliable. These points suggest that one seemingly compelling justification for linguistically discriminating against scientific contributions in ‘non-native’ English is questionable and that the common insistence by scientific gatekeepers on ‘native-like’ English may be epistemically harmful to science. (shrink)

This paper presents 59 new studies (N = 72,310) which focus primarily on the “bat and ball problem.” It documents our attempts to understand the determinants of the erroneous intuition, our exploration of ways to stimulate reflection, and our discovery that the erroneous intuition often survives whatever further reflection can be induced. Our investigation helps inform conceptions of dual process models, as “system 1” processes often appear to override or corrupt “system 2” processes. Many choose to uphold their intuition, even (...) when directly confronted with simple arithmetic that contradicts it – especially if the intuition is approximately correct. (shrink)

Mathematics word problems provide students with an opportunity to apply what they are learning in their mathematics classes to the world around them. However, students often neglect their knowledge of the world and provide nonsensical responses (e.g., they may answer that a school needs 12.5 buses for a field trip). This study examined if the question design of word problems affects students' mindset in ways that affect subsequent sense‐making. The hypothesis was that rewriting standard word problems to introduce inherent uncertainty (...) about the result would be beneficial to student performance and sense‐making because it requires students to reason explicitly about the context described in the problem. Middle school students (N = 229) were randomly assigned to one of three conditions. In the standard textbook condition, students solved a set of six word problems taken from current textbooks. In the modified yes/no condition, students solved the same six problems rewritten so the solution helped answer a “yes” or “no” question. In the disfluency control condition, students solved the standard problems each rewritten in a variety of fonts to make them look unusual. After solving the six problems in their assigned condition, all students solved the same three “problematic” problems designed to assess sense‐making. Contrary to predictions, results showed that students in the modified yes/no condition solved the fewest problems correctly in their assigned condition problem set. However, consistent with predictions, they subsequently demonstrated more sense‐making on the three problematic problems. Results suggest that standard textbook word problems may be able to be rewritten in ways that mitigate a “senseless” mindset. (shrink)

Three experiments explored the effects of word problem cueing on debiasing versions of the bat-and-ball problem. In the experimental condition order, participants solved a simpler isomorphic version of the problem prior to solving a standard version that, critically, had the same item-and-dollar amounts. Conversely, in the control condition order, participants solved the standard version prior to solving the isomorph. Across the first 2 experiments, participants cued with the isomorph were more likely to correctly solve the standard version of the problem. (...) In the third experiment, this cueing effect was not replicated, but importantly cued participants were more likely to correctly solve the standard version of the problem with other item-and-dollar amounts, indicating near transfer. Far transfer was evident in faster response times to other problems in the Cognitive Reflection Test following practice. Along with providing converging evidence for the cueing account, these results are consistent with recent attentional debiasing findings. (shrink)

Four studies explore the role of perceptual fluency in attenuating bullshit receptivity, or the tendency for individuals to rate otherwise meaningless statements as “profound”. Across four studies, we presented participants with a sample of pseudoprofound bullshit statements in either a fluent or disfluent font and found that overall, disfluency attenuated bullshit receptivity while also finding little evidence that this effect was moderated by cognitive thinking style. In all studies, we measured participants’ cognitive reflection, need for cognition, faith in intuition, and (...) superstitious beliefs. Superstition strongly predicted bullshit receptivity regardless of fluency. Inconclusive results were found for the remaining scales. Potential links for the role of perceptual disfluency in promoting analytic thinking are discussed. (shrink)

The replicability of findings in experimental psychology can be improved by distinguishing sharply between hypothesis-generating research and hypothesis-testing research. This distinction can be achieved by preregistration, a method that has recently attracted widespread attention. Although preregistration is fair in the sense that it inoculates researchers against hindsight bias and confirmation bias, preregistration does not allow researchers to analyze the data flexibly without the analysis being demoted to exploratory. To alleviate this concern we discuss how researchers may conduct blinded analyses. As (...) with preregistration, blinded analyses break the feedback loop between the analysis plan and analysis outcome, thereby preventing cherry-picking and significance seeking. However, blinded analyses retain the flexibility to account for unexpected peculiarities in the data. We discuss different methods of blinding, offer recommendations for blinding of popular experimental designs, and introduce the design for an online blinding protocol. (shrink)

Cognitive reflection, or the ability to inhibit intuitive and incorrect responses in favour of correct responses, predicts performance on a variety of cognitive tasks. The present study examined interventions to improve cognitive reflection. In two experiments, college students (N = 491) were assigned to one of three conditions, completed two versions of a cognitive reflection test (CRT), and then completed transfer tasks. Between the two CRTs, some participants were provided with elaborative feedback, others were instructed to consider additional responses for (...) their initial responses and the final group was a control. In both experiments, CRT performance increased between the first and second CRT in the feedback and instruction groups, but not in the control group. There was little evidence, however, for transfer to other tasks. These results suggest that cognitive reflection performance can be improved with brief interventions but that this improvement may not transfer to related tasks. (shrink)

Philosophy is a reflective activity. So perhaps it is unsurprising that many philosophers have claimed that reflection plays an important role in shaping and even improving our philosophical thinking. This hypothesis seems plausible given that training in philosophy has correlated with better performance on tests of reflection and reflective reasoning has correlated with demonstrably better judgments in a variety of domains. This article reviews the hypothesized roles of reflection in philosophical thinking as well as the empirical evidence for these roles. (...) This reveals that although there are fairly reliable links between reflection and philosophical judgment among both laypeople and philosophers, the role of reflection in philosophical thinking may nonetheless depend in part on other factors, some of which have yet to be determined. So progress in research on reflection in philosophy may require further innovation in experimental methods and psychometric validation of philosophical measures. (shrink)

Prior research found correlations between reflection test performance and philosophical tendencies among laypeople. In two large studies (total N = 1299)—one pre-registered—many of these correlations were replicated in a sample that included both laypeople and philosophers. For example, reflection test performance predicted preferring atheism over theism and instrumental harm over harm avoidance on the trolley problem. However, most reflection-philosophy correlations were undetected when controlling for other factors such as numeracy, preferences for open-minded thinking, personality, philosophical training, age, and gender. Nonetheless, (...) some correlations between reflection and philosophical views survived this multivariate analysis and were only partially confounded with either education or self-reported reasoning preferences. Unreflective thinking still predicted believing in God whereas reflective thinking still predicted believing that (a) proper names like ‘Santa’ do not necessarily refer to entities that actually exist and (b) science does reveal the fundamental nature of the world. So some robust relationships between reflection and philosophical tendencies were detected even among philosophers, and yet there was clearly more to the link between reflection and philosophy. To this end, demographic and metaphilosophical hypotheses are considered. (shrink)

Intertemporal choice requires one to decide between smaller sooner and larger later payoffs and is captured by discount rates. Across two preregistered experiments testing three language pairs (Polish vs. English, Spanish, and German; Experiment 1) and with incentivized participants (Experiment 2), we found no evidence that using a foreign language decreased the strength or increased the consistency of intertemporal choices. On the contrary, there was some evidence of stronger discounting when a foreign language was used. We confirmed prior findings that (...) more reflective individuals discount less strongly but observed that they were just as (un)affected by using foreign language as less reflective individuals. Thus, we provide preliminary evidence that the foreign language effect is robust to individual differences in cognitive reflection. (shrink)

Intertemporal choice requires one to decide between smaller sooner and larger later payoffs and is captured by discount rates. Across two preregistered experiments testing three language pairs (Polish vs. English, Spanish, and German; Experiment 1) and with incentivized participants (Experiment 2), we found no evidence that using a foreign language decreased the strength or increased the consistency of intertemporal choices. On the contrary, there was some evidence of stronger discounting when a foreign language was used. We confirmed prior findings that (...) more reflective individuals discount less strongly but observed that they were just as (un)affected by using foreign language as less reflective individuals. Thus, we provide preliminary evidence that the foreign language effect is robust to individual differences in cognitive reflection. (shrink)