An astonishing volume and diversity of evidence is available for many hypotheses in the biomedical and social sciences. Some of this evidence—usually from randomized controlled trials (RCTs)—is amalgamated by meta-analysis. Despite the ongoing debate regarding whether or not RCTs are the ‘gold-standard’ of evidence, it is usually meta-analysis which is considered the best source of evidence: meta-analysis is thought by many to be the platinum standard of evidence. However, I argue that meta-analysis (...) falls far short of that standard. Different meta-analyses of the same evidence can reach contradictory conclusions. Meta-analysis fails to provide objective grounds for intersubjective assessments of hypotheses because numerous decisions must be made when performing a meta-analysis which allow wide latitude for subjective idiosyncrasies to influence its outcome. I end by suggesting that an older tradition of evidence in medicine—the plurality of reasoning strategies appealed to by the epidemiologist Sir Bradford Hill—is a superior strategy for assessing a large volume and diversity of evidence. (shrink)

Scholars have proposed a number of courses and programs intended to improve the ethical behavior of scientists in an attempt to maintain the integrity of the scientific enterprise. In the present study, we conducted a quantitative meta-analysis based on 26 previous ethics program evaluation efforts, and the results showed that the overall effectiveness of ethics instruction was modest. The effects of ethics instruction, however, were related to a number of instructional program factors, such as course content and delivery (...) methods, in addition to factors of the evaluation study itself, such as the field of investigator and criterion measure utilized. An examination of the characteristics contributing to the relative effectiveness of instructional programs revealed that more successful programs were conducted as seminars separate from the standard curricula rather than being embedded in existing courses. Furthermore, more successful programs were case based and interactive, and they allowed participants to learn and practice the application of real-world ethical decision-making skills. The implications of these findings for future course development and evaluation are discussed. (shrink)

In evolutionary psychology predictions, women’s mate preferences shift between fertile and nonfertile times of the month to reflect ancestral fitness benefits. Our meta-analytic test involving 58 independent reports (13 unpublished, 45 published) was largely nonsupportive. Specifically, fertile women did not especially desire sex in short-term relationships with men purported to be of high genetic quality (i.e., high testosterone, masculinity, dominance, symmetry). The few significant preference shifts appeared to be research artifacts. The effects declined over time in published work, were (...) limited to studies that used broader, less precise definitions of the fertile phase, and were found only in published research. (shrink)

This meta-analysis of 31 studies over 20 years advances our understanding of the gender gap in entrepreneurial bank finance. Findings from previous research on the relationship between entrepreneurs’ gender and bank financing are mixed, which suggests the need to pay particular attention to entrepreneurs’ social context. In this study, we develop a model of how social gender norms explain variation in women entrepreneurs’ (vis-à-vis men entrepreneurs’) access to bank finance. Specifically, we theorize how women’s formal (their nations’ political (...) ideologies) and informal (women’s empowerment) social standing within their societies influence gender discrimination in entrepreneurial bank financing. Consistent with most previous studies, our baseline results show that women entrepreneurs’ business loan applications are rejected to a greater extent than men entrepreneurs’ loan applications. Women entrepreneurs also pay higher interest rates on loans than men entrepreneurs. Further, in societies dominated by a conservative (rather than a liberal) political ideology, the positive relationship between women entrepreneurs and loan interest rates is more positive. Interestingly, gender discrimination in loan rejection and interest rates is magnified in societies with greater women’s empowerment. Taking a social gender-norm perspective, our analysis establishes a gender gap in entrepreneurial bank finance, and we outline an agenda for further research. (shrink)

E-learning has gained popularity since the outbreak of COVID-19. This study aims to identify gender differences in e-learners' self-efficacy, satisfaction, motivation, attitude, and performance across the world. Through a meta-analysis and systematic review, this study concludes that there are generally no significant gender differences in e-learning outcomes except in a few countries. Females significantly outperformed males in Spain and the UK. In Austria, India, and mixed countries, females hold significantly more positive attitudes toward e-learning than males. In the (...) USA, females present significantly higher self-efficacy than males. Future research into the gender issue in e-learning across the world may adopt cross-disciplinary research methods except for a meta-analysis. (shrink)

Meta‐analysis: the glass eye of evidence‐based practice?Meta‐analysis was developed as a technique for combining the results of many different quantitative studies: it is often used to produce quantitative estimates of causal relations and/or association between variables. Meta‐analysis is sometimes regarded as a central component of evidence‐based practice. We draw attention to an incompatibility in the epistemology and methods of reasoning in quantitative meta‐analysis and the epistemology and reasoning implicit in expert practice. We (...) argue that this may be because the common perception of meta‐analysis appeals to truth as correspondence; we suggest that rejecting the naïve realism that underpins truth as correspondence allows meta‐analysis to be understood in terms of truth as coherence. We can then develop an account of meta‐analysis that does not depend upon reduction to a mathematical procedure but is an attempt to maximise coherence in beliefs about what works that is consistent with clinical reasoning in expert practice. (shrink)

Mental contrasting with implementation intentions is a self-regulation strategy that enhances goal attainment. This meta-analysis evaluated the efficacy of MCII for goal attainment and explored potential moderators. A total of 21 empirical studies with 24 independent effect sizes were included in the analysis. Results showed that MCII to be effective for goal attainment with a small to medium effect size. The effect was mainly moderated by intervention style. Specifically, studies with interventions based on interactions between participants and (...) experimenters had stronger effects than studies with interventions based on interactions between participants and documents. The results revealed that MCII is a brief and effective strategy for goal attainment with a small to moderate effect; however, because of some publication bias, the actual effect sizes may be smaller. Due to small number of studies in this meta-analysis, additional studies are needed to determine the role of moderator variables. (shrink)

Emotional clarity and attention to emotions represent the extent to which people understand and attend to their own emotions, respectively, and are broad facets of emotional awareness, alexithymia, and emotional intelligence. To examine the extent to which these two constructs are associated, we conducted a meta-analysis of studies including well-validated self-report measures of trait clarity and attention to emotion. Clarity and attention were moderately, positively associated. Assessment instrument, but not sample gender or age, moderated the association between clarity (...) and attention. Analyses of between-study heterogeneity and publication bias suggested that results were valid and generalizable. We discuss potential causes of the association between clarity and attention, and elaborate on the implications of our results for emotion regulation, well-being, and psychopathology. (shrink)

A meta-analysis of 190 cross-cultural emotion studies, published between 1967 and 2000, was performed to examine (1) to what extent reported cross-cultural differences in emotion variables could be regarded as valid (substantive factors) or as method-related (statistical artefacts, cultural bias), and (2) which country characteristics could explain valid cross-cultural differences in emotion. The relative contribution of substantive and method-related factors at sample, study, and country level was investigated and country-level explanations for differences in emotions were tested. Results indicate (...) that a correction for statistical artefacts and method-related factors reduced the observed cross-cultural effect sizes considerably. After controlling for valence (positive vs. negative emotions) and kind of study (self-report vs. recognition studies), the remaining cross-cultural variance was associated with subsistence mode, political system, values, and religiosity. Values explained more variance than did ecological or sociopolitical variables. It was concluded that both method-related factors (13.8% of variance explained) and culture-level factors (27.9% of variance explained) underlie observed cross-cultural differences. (shrink)

The “erasing race” effect is the reduction of the salience of “race” as an alliance cue when recalling coalition membership, once more accurate information about coalition structure is presented. We conducted a random-effects model meta-analysis of this effect using five United States studies (containing nine independent effect sizes). The effect was found (ρ = 0.137, K = 9, 95% CI = 0.085 to 0.188). However, no decline effect or moderation effects were found (a “decline effect” in this context (...) would be a decrease in the effect size over time). Furthermore, we found little evidence of publication bias. Synthetically correcting the effect size for bias stemming from the use of an older method for calculating error base rates reduced the magnitude of the effect, but the it remained significant. Taken together, these findings indicate that the “erasing race” effect generalizes quite well across experimental contexts and would, therefore, appear to be quite robust. We reinterpret the theoretical basis for these effects in line with Brunswikian evolutionary-developmental theory and present a series of predictions to guide future research in this area. (shrink)

OBJECTIVES: This reliability generalization study aimed to estimate the mean and variance of the interrater reliability coefficients (ryy) of supervisory ratings of overall, task, contextual, and positive job performance. The moderating effect of the appraisal purpose and the scale type was examined. It was hypothesized that the ratings collected for research purposes and multi-item scales have higher ryy. It was also examined whether ryy was similar for the four performance dimensions. METHOD: A database consisting of 224 independent samples was created (...) and hierarchical sub-grouping meta-analyses were conducted. RESULTS: The appraisal purpose was a moderator of ryy for the four performance dimensions. Scale type was a moderator of ryy for overall and task performance collected for research purposes. The findings also suggest that supervisors seem to have less difficulty evaluating overall job performance than task, contextual, and positive performance. The best estimates of the observed ryy for overall job performance are.61 for research-collected ratings and.45 for administrative-collected ratings. MAIN CONCLUSIONS: (1) Appraisal purpose moderates ryy and researchers and practitioners should be aware of its effects before collecting ratings or using empirically-derived interrater reliability distributions, (2) Scale type seems to moderate ryy in the case of the ratings collected for research purposes, only, (3) overall job performance is more reliably rated than task, contextual, and positive performance. Implications for research and practice are discussed. (shrink)

The biophilia hypothesis posits an innate biological and genetic connection between human and nature, including an emotional dimension to this connection. Biophilic design builds on this hypothesis in an attempt to design human-nature connections into the built environment. This article builds on this theoretical framework through a meta-analysis of experimental studies on the emotional impacts of human exposure to natural and urban environments. A total of 49 studies were identified, with a combined sample size of 3,201 participants. The (...) primary findings indicated that exposure to natural environments had a medium to large effect on both increasing positive affect and decreasing negative affect. This finding supported the anticipated emotional dimension of the biophilia hypothesis and lends credibility to biophilic design theory. Evidence was revealed in support of the affective/arousal response model. Immersion in environments indicated a larger effect size than laboratory simulation of environments. Methodological recommendations for future experimental research were few, however the Positive and Negative Affect Schedule outcome measure was recommended as a measure of both positive and negative affect for further studies. A combination measurement of stress related outcome variables was proposed to further explore the affective/arousal response model and its potential relationship to the biophilia hypothesis. The meta-analysis provides evidence for fundamental theories regarding human-nature connection, while revealing gaps in current knowledge. (shrink)

This paper presents a series of meta-analyses of the validity of general mental ability (GMA) for predicting five occupational criteria, including supervisory ratings of job performance, production records, work sample tests, instructor ratings, and grades. The meta-analyses were conducted with a large database of 467 technical reports of the validity of the General Aptitude Test Battery (GATB) which included 630 independent samples. GMA showed to be a consistent predictor of the five criteria, but the magnitude of the operational (...) validity was not the same across the five criteria. Results also showed that job complexity is a moderator of the GMA validity for the performance criteria. We also found that the GMA validity estimates are slightly smaller than the previous ones obtained by Hunter (1983a; Hunter & Hunter, 1984). Finally, we discuss the implications of these findings for the research and practice of personnel selection. (shrink)

Emotional intelligence is an emerging field since the 1990s due to its important outcomes for employees. This study is a psychometric meta-analysis examining the links between emotional intelligence and organizational commitment, organizational citizenship behavior, job satisfaction, job performance, and job stress of employees. In this meta-analysis, carefully selected studies on emotional intelligence since the origin of the concept in 1990 were included along with studies examining its outcomes. For this analysis, three streams of emotional intelligence, (...) consistent with previous meta-analyses, were considered: ability, self-report, and mixed emotional intelligence. This meta-analysis is an attempt to add to the literature by analyzing the relationships between emotional intelligence and selected employee outcomes over a period of time beginning in 1990. The three streams of emotional intelligence were separately analyzed to examine their relationship with employee outcomes. These outcomes were included in the study based on select research studies. Our study results showed that emotional intelligence and its three streams were positively related to organizational commitment, organizational citizenship behavior, job satisfaction, and job performance and negatively related to job stress. (shrink)

Chow's defense of the null-hypothesis significance- test procedure is thoughtful and compelling in many respects. Nevertheless, techniques such as meta-analysis, power analysis, effect size estimation, and confidence intervals can be useful supplements to NHSTP in furthering the cumulative nature of behavioral research, as illustrated by the history of research on the spontaneous recovery of verbal learning.

Emotion Review, Volume 14, Issue 1, Page 66-83, January 2022. To better define the boundaries of conceptually overlapping constructs of intrapersonal emotion knowledge, we examined meta-analytic correlations among five intrapersonal EK-related constructs and attention to emotion. Affect labelling, alexithymia, and emotional clarity were strongly associated, and they were moderately associated with attention to emotion. Alexithymia and emotional awareness were weakly associated, and emotion differentiation was unrelated with emotional clarity. Sample characteristics and measures moderated some of the associations. Publication bias (...) was not found, except for the alexithymia-emotional awareness association. This study helped to clarify the extent to which similarly defined constructs overlap or are distinct, which can inform our decision to adequately label important constructs and employ corresponding measures. (shrink)

Based on research on the increasingly popular unconscious thought effect (UTE), it has been suggested that physicians might make better diagnostic decisions after a period of distraction than after an equivalent amount of time of conscious deliberation. However, published attempts to demonstrate the UTE in medical decision making have yielded inconsistent results. In the present study, we report the results of a meta-analysis of all the available evidence on the UTE in medical decisions made by expert and novice (...) clinicians. The meta-analysis failed to find a significant contribution of unconscious thought to the accuracy of medical decisions. This result cannot be easily attributed to any of the potential moderators of the UTE that have been discussed in the literature. Furthermore, a Bayes Factor analysis shows that most experimental conditions provide positive support for the null hypothesis, suggesting that these null results do not reflect a simple lack of statistical power. We suggest ways in which new studies could usefully provide further evidence on the UTE. Unless future research shows otherwise, the recommendation of using unconscious thought to improve medical decisions lacks empirical support. (shrink)

Philosophical work on meta-analysis focuses on biomedical research and revolves around the question: Is meta-analytic evidence the best kind of evidence? By focusing on conservation science rather than biomedical science, I identify further questions and puzzles for meta-analysis and show their importance for the epistemology of meta-analysis.

This meta-analysis synthesizes quantitative findings of the gender differences in moral sensitivity retrieved from 19 primary studies. We found the average effect size of 0.25, favoring women, with a standard deviation of 0.14. The variation in the observed effect sizes could not be attributed to differences in participants' educational level, the utilized measure of moral sensitivity, or the publication format in which the study was reported. This suggests that gender differences in moral sensitivity are consistent across different levels (...) of participants' education regardless of the instrument used to assess moral sensitivity or the format in which the research was reported. (shrink)

Arguments that medical decision making should rely on a variety of evidence often begin from the claim that meta-analysis has been shown to be problematic. In this paper, I first examine Stegenga’s argument that meta-analysis requires multiple decisions and thus fails to provide an objective ground for medical decision making. Next, I examine three arguments from social epistemologists that contend that meta-analyses are systematically biased in ways not appreciated by standard epistemology. In most cases I (...) show that critiques of meta-analysis fail to account for the full range of meta-analytic procedures. In the remainder of cases, I argue that the critiques identify problems that do not uniquely cut against meta-analysis. I close by suggesting one reason why it may be pragmatically rational to violate the principle of total evidence and by outlining the criteria for a successful argument against meta-analysis. A set of criteria I contend remain unmet. (shrink)

Emotion Review, Volume 14, Issue 1, Page 66-83, January 2022. To better define the boundaries of conceptually overlapping constructs of intrapersonal emotion knowledge, we examined meta-analytic correlations among five intrapersonal EK-related constructs and attention to emotion. Affect labelling, alexithymia, and emotional clarity were strongly associated, and they were moderately associated with attention to emotion. Alexithymia and emotional awareness were weakly associated, and emotion differentiation was unrelated with emotional clarity. Sample characteristics and measures moderated some of the associations. Publication bias (...) was not found, except for the alexithymia-emotional awareness association. This study helped to clarify the extent to which similarly defined constructs overlap or are distinct, which can inform our decision to adequately label important constructs and employ corresponding measures. (shrink)

We conducted a rapid review and quantitative summary of meta-analyses that have examined interventions which can be used by individuals during quarantine and social distancing to manage anxiety, depression, stress and subjective well-being. A literature search yielded 34 meta-analyses (total number of studies k = 1,390, n = 145,744) that were summarized. Overall, self-guided interventions showed small to medium effects in comparison to control groups. In particular, self-guided therapeutic approaches (including cognitive-behavioral, mindfulness, and acceptance-based interventions), selected positive psychology (...) interventions, and multi-component and activity-based interventions (music, physical exercise) showed promising evidence for effectiveness. Overall, self-guided interventions on average did not show the same degree of effectiveness as traditional guided individual or group therapies. There was no consistent evidence of dose effects, baseline differences and differential effectiveness of eHealth interventions. More research on the effectiveness of interventions in diverse cultural settings is needed. (shrink)

Lloyd (2011) presents highly suggestive results regarding the specificity of the link between particular brain areas and cognitive tasks. Some of his evidence is derived from the analysis of data from the BrainMap database (available: www.brainmap.org), which has become a fundamental resource for the conduct of functional neuroimaging meta-analysis. In the present note, some observations regarding the possibilities and pitfalls of meta-analysis of functional neuroimaging data are given as a complement to Lloyd's excellent exposition of (...) the topic. Additionally, some comments are made on the particular meta-analytic results presented by Lloyd.Functional neuroimaging studies usually present their findings in the form .. (shrink)

In the present meta-analysis, we examined the effect of cognitive training on the Executive Function (EF) of preschool children (age range: 3-6 years). We selected a final set of 32 studies from 27 papers with a total sample of 123 effect sizes. We found an overall effect of cognitive training for improving EF (g =.352; k = 123; p <.001), without significant difference between near and far transfer effects on executive domains. No significant additional outcome effects were found (...) for behavioral- and learning-related outcomes. Cognitive training programs for preschoolers are significantly more effective for developmentally at-risk children (ADHD or low socio-economic status) than for children with typical development and without risks. Other significant moderators were computerized versus non-computerized training, individual versus group sessions, and length of training. The number of sessions was not a significant moderator. This is the first demonstration of cognitive training far transfer effects among different executive processes. We discuss this result in relationship with the lower level of modularization of EFs in younger children. (shrink)

This study aims to reveal the impact of proactive personality on career success and the sequential mediation effect of organizational citizenship behavior and task performance on the relationship. Utilizing meta-analytic structural equation modeling technology sampling 101,131 employees from multiple organizations and industries, which deeply decreased sampling error, the results indicated slightly different findings of proactive personality and three types of career success. Specifically, in relation to salary, OCB and task performance independently transmit the effects of proactive personality to subjective (...) career success, but they sequentially mediate this link as well. In regard to subjective career success and promotion, OCB mediates the relationship between proactive personality and promotion. OCB and task performance sequentially mediate these links. We discussed findings cautiously and purpose future research directions. (shrink)

The present study aimed to quantify the magnitude of sex differences in humans' ability to accurately recognise non-verbal emotional displays. Studies of relevance were those that required explicit labelling of discrete emotions presented in the visual and/or auditory modality. A final set of 551 effect sizes from 215 samples was included in a multilevel meta-analysis. The results showed a small overall advantage in favour of females on emotion recognition tasks (d = 0.19). However, the magnitude of that sex (...) difference was moderated by several factors, namely specific emotion, emotion type (negative, positive), sex of the actor, sensory modality (visual, audio, audio-visual) and age of the participants. Method of presentation (computer, slides, print, etc.), type of measurement (response time, accuracy) and year of publication did not significantly contribute to variance in effect sizes. These findings are discussed in the context of social and biological explanations of sex differences in emotion recognition. (shrink)

Perceived behavioral integrity involves the employee’s perception of the alignment of the manager’s words and deeds. This meta-analysis examined the relationship between perceived behavioral integrity of managers and the employee attitudes of job satisfaction, organizational commitment, satisfaction with the leader and affect toward the organization. Results indicate a strong positive relationship overall (average r = 0.48, p<0.01). With only 12 studies included, exploration of moderators was limited, but preliminary analysis suggested that the gender of the employees and (...) the number of levels between the employee and the manager are potential moderators of the relationship. In the current sample of studies, country where the research was conducted did not seem to have any moderating effects. In addition to suggesting further investigation of potential moderators, we call for research that examines the relationship between behavioral integrity and outcomes that include individual behavior and organizational performance. (shrink)

Background Moral distress is a common challenge among professional nurses when caring for their patients, especially when they need to make rapid decisions. Therefore, leaving moral distress unconsidered may jeopardize patient quality of care, safety, and satisfaction. Aim To estimate moral distress among nurses. Methods This systematic review and meta-analysis conducted systematic search in Scopus, PubMed, ProQuest, ISI Web of Knowledge, and PsycInfo up to end of February 2022. Methodological quality of included studies was assessed using the Newcastle (...) Ottawa checklist. Data from included studies were pooled by meta-analysis with random effect model in STATA software version 14. The selected key measure was mean score of moral distress total score with its’ 95% Confidence Interval was reported. Subgroup analyses and meta-regressions were conducted to identify possible sources of heterogeneity and potentially influencing variables on moral distress. Funnel plots and Begg’s Tests were used to assess publication bias. The Jackknife method was used for sensitivity analysis. Ethical consideration The protocol of this project was registered in the PROSPERO database under decree code of CRD42021267773. Results Eighty-six manuscripts with 19,537 participants from 21 countries were included. The pooled estimated mean score of moral distress was 2.55 on a 0–10 scale [95% Confidence Interval: 2.27–2.84, I2: 98.4%, Tau2:0.94]. Publication bias and small study effect was ruled out. Moral distress significantly decreased in the COVID-19 pandemic versus before. Nurses working in developing countries experienced higher level of moral distress compared to their counterparts in developed countries. Nurses' workplace (e.g., hospital ward) was not linked to severity of moral disturbance. Conclusion The results of the study showed a low level of pooled estimated score for moral distress. Although the score of moral distress was not high, nurses working in developing countries reported higher levels of moral distress than those working in developed countries. Therefore, it is necessary that future studies focus on creating a supportive environment in hospitals and medical centers for nurses to reduce moral distress and improve healthcare. (shrink)

Business Ethics, the Environment &Responsibility, EarlyView.

Computational models of child language development can help us understand the cognitive underpinnings of the language learning process, which occurs along several linguistic levels at once (e.g., prosodic and phonological). However, in light of the replication crisis, modelers face the challenge of selecting representative and consolidated infant data. Thus, it is desirable to have evaluation methodologies that could account for robust empirical reference data, across multiple infant capabilities. Moreover, there is a need for practices that can compare developmental trajectories of (...) infants to those of models as a function of language experience and development. The present study aims to take concrete steps to address these needs by introducing the concept of comparing models with large-scale cumulative empirical data from infants, as quantified by meta-analyses conducted across a large number of individual behavioral studies. We formalize the connection between measurable model and human behavior, and then present a conceptual framework for meta-analytic evaluation of computational models. We exemplify the meta-analytic model evaluation approach with two modeling experiments on infant-directed speech preference and native/non-native vowel discrimination. (shrink)

My goal in this paper is to see the extent to which judgment aggregation methods subsume meta-analytic ones. To this end, I derive a generalized version of the classical Condorcet Jury Theorem, the aggregative implications of which have been widely exploited in the area of rational choice theory, but not yet in philosophy of science. I contend that the generalized CJT that I prove below is useful for modelling at least some meta-analytic procedures.

Are there relationships between consciousness and the material world? Empirical evidence for such a connection was reported in several meta-analyses of mind-matter experiments designed to address this question. In this paper we consider such meta-analyses from a statistical modeling perspective, emphasizing strategies to validate the models and the associated statistical procedures. In particular, we explicitly model increased data variability and selection mechanisms, which permits us to estimate 'selection profiles ' and to reassess the experimental effect in view of (...) potential other effects. An application to the data pool considered in the influential meta-analysis of Radin and Nelson (1989) yields indications for the presence of random and selection effects Adjustment for possible selection is found to render the,without such an adjustment significant, experimental effect non-significant. Somewhat different conclusions apply to a subset of the data deserving separate consideration. The actual origin of the data features that are described as experimental, random, or selection effects within the proposed model cannot be clarified y our approach and remains open. (shrink)