Abstract

Skeptical wisdom holds that parapsychological studies that produce significant results must have a low methodological quality. In most meta-analyses of parapsychological paradigms such as ‘Ganzfeld telepathy’ and so-called ‘presentiment’, the meta-analyzers, who are generally proponents of the psi hypothesis, also present a quality-effect size relation. These relations generally are at odds with the ‘skeptical wisdom’, i.e. they produce a positve or nonsignificant relation. One can and should wonder what the quality is of the assessment of these relations. I became aware of this question when I found that a so-called PK–RNG (or Mind-over-Matter) study of mine got a low quality rating in one of the first large-scale meta-analyses (Radin & Nelson 2000). With my pride hurt, I delved a bit deeper into this and soon found that quality is assessed on the basis of the written reports. These reports generally have the standard structure of scientific writings but often there is a special paragraph dealing with ‘alternative explanations’ where the authors go to some length to discuss sensory leakage, randomization problems, and other potential alternative explanations of their anomalous results. Never will one find here remarks such as: the sensory shielding was inadequate or the randomization was done by hand-shuffling. For the vast majority, these paragraphs carry information to persuade the reader that there are no alternative explanations. I had no paragraph on alternative explanations simply because I didn’t need one because the results were nonsignificant. So my nonsignificant study got a low quality rating while of course I did extensive randomization tests before even starting the study.