Francesco Guala once wrote that ‘The problem of extrapolation is a minor scandal in the philosophy of science’. This paper agrees with the statement, but for reasons different from Guala’s. The scandal is not, or not any longer, that the problem has been ignored in the philosophy of science. The scandal is that framing the problem as one of external validity encourages poor evidential reasoning. The aim of this paper is to propose an alternative—an alternative which constitutes much (...) better evidential reasoning about target systems of interest, and which makes do without consideration of external validity. (shrink)

I first propose to distinguish between two kinds of external validity inferences, predictive and explanatory. I then argue that we have a satisfactory answer to the question of the conditions under which predictive external validity inferences are good. If this claim is correct, then it has two immediate consequences: First, some external validity inferences are deductive, contrary to what is commonly assumed. Second, Steel’s requirement that an account of external validity inference break (...) what he calls the ‘Extrapolator’s Circle’ is misplaced, at least when it comes to predictive external validity inferences. (shrink)

Francesco Guala has developed some novel and radical ideas on the problem of external validity, a topic that has not received much attention in the experimental economics literature. In this paper I argue that his views on external validity are not justified and the conclusions which he draws from these views, if widely adopted, could substantially undermine the experimental economics enterprise. In rejecting the justification of these views, the paper reaffirms the importance of experiments in economics.

We agree that external validity of social psychological experiments is a concern, we disagree these models are useless. Experiments, reconsidered from a situated cognition perspective and non-linearly combined with other methods allow grasping decision dynamics beyond bias outcomes. Dynamic insights regarding these processes are key to understand missing forces and bias in real-world social groups.

Discussions of reproducibility are casting doubts on the credibility of experimental outcomes in the life sciences. Although experimental evolution is not typically included in these discussions, this field is also subject to low reproducibility, partly because of the inherent contingencies affecting the evolutionary process. A received view in experimental studies more generally is that standardization (i.e., rigorous homogenization of experimental conditions) is a solution to some issues of significance and internal validity. However, this solution hides several difficulties, including a (...) reduction of external validity and reproducibility. After explaining the meaning of these two notions in the context of experimental evolution, we import from the fields of animal research and ecology and suggests that systematic heterogenization of experimental factors could prove a promising alternative. We also incorporate into our analysis some philosophical reflections on the nature and diversity of research objectives in experimental evolution. (shrink)

Gigerenzer’s ‘external validity argument’ plays a pivotal role in his critique of the heuristics and biases research program (HB). The basic idea is that (a) the experimental contexts deployed by HB are not representative of the real environment and that (b) the differences between the setting and the real environment are causally relevant, because they result in different performances by the subjects. However, by considering Gigerenzer’s work on frequencies in probability judgments, this essay attempts to show that there (...) are fatal flaws in the argument. Specifically, each of the claims is controversial: whereas (b) is not adequately empirically justified, (a) is inconsistent with the ‘debiasing’ program of Gigerenzer’s ABC group. Therefore, whatever reason we might have for believing that the experimental findings of HB lack experimental validity, this should not be based on Gigerenzer’s version of the argument. (shrink)

Experimental “localism” stresses the importance of context‐specific knowledge, and the limitations of universal theories in science. I illustrate Latour's radical approach to localism and show that it has some unpalatable consequences, in particular the suggestion that problems of external validity (or how to generalize experimental results to nonlaboratory circumstances) cannot be solved. In the last part of the paper I try to sketch a solution to the problem of external validity by extending Mayo's error‐probabilistic approach.

The COPE Inventory is the most frequently used measure of coping; yet previous studies examining its factor structure yielded mixed results. The purpose of the current study, therefore, was to validate the factor structure of the COPE Inventory in a representative sample of over 2,000 adults in Slovakia. Our second goal was to evaluate the external validity of the COPE inventory, which has not been done before. Firstly, we performed the exploratory factor analysis with half of the sample. (...) Subsequently, we performed the confirmatory factor analysis with the second half of the sample. Both factor analyses with 15 factor solutions showed excellent fit with the data. Additionally, we performed a hierarchical factor analysis with fifteen first-order factors and three second-order factors which showed good fit with the data. Moreover, the COPE Inventory’s external validity was evaluated using consensual qualitative research analysis on data collected by in-depth interviews. Categories of coping created using CQR corresponded with all COPE first-order factors. Moreover, we identified two additional first-order factors that were not present in the COPE Inventory: self-care and care for others. Our study shows that the Slovak translation of the COPE Inventory is a reliable, externally valid, and well-structured instrument for measuring coping in the Slovak population. (shrink)

It is sometimes argued that experimental economists do not have to worry about external validity so long as the design sticks closely to a theoretical model. This position mistakes the model for the theory. As a result, applied economics designs often study phenomena distinct from their stated objects of inquiry. Because the implemented models are abstract, they may provide improbable analogues to their stated subject matter. This problem is exacerbated by the relational character of the social world, which (...) also sets epistemic limits for the social science laboratory more generally. (shrink)

This paper develops an account of rigour in the use of thought experiments in ethics. I argue that there are two separate challenges to be faced. The first is internal validity: is the thought experiment designed in a way that allows its readers to make judgements that are confident and free of bias about the hypothesis or point of principle that it aims to test? The second is external validity: to what extent do ethical judgements that are (...) correct of the world of the thought experiment generalize to a wide variety of other contexts, including ethical decision-making in the actual world? Ensuring external validity is the harder and more important problem of rigour, yet it is one that few philosophers have even noticed, let alone begun to solve. (shrink)

According to a classical work on experimental design internal validity “refers to the approximate validity with which we infer that a relationship between two variables is causal or that the absence of a relationship implies the absence of cause”. External validity “refers to the approximate validity with which we can infer that the presumed causal relationship can be generalized to and across alternate measures of the cause and effect and across different types of persons, settings, (...) and times”. Since then, these have been the accepted definitions, with slight variations, of both concepts. However, there have been deep differences in interpreting the scope and limits of validity concepts in economics. From theory-driven experimental designs that defend the priority of the internal over the external validity to those researches that are demanding the importance of external validity in economics, we can find the most diverse points of view. What can experimental ethics learn, then, from recent debates on external and internal validity in experimental economics? If we want to elicit moral intuitions into the lab, are we testing moral theories that way? Could intuitions be generalized “across different types of persons, settings, and times”? Is that really possible? (shrink)

Cross-cultural comparisons often investigate values that are assumed to have long-lasting influence on human conduct and thought. To capture and compare cultural values across cultures, Geert Hofstede's Cultural Dimensions Theory has offered an influential framework. Hofstede also provided a survey instrument, the Values Survey Module (VSM), for measuring cultural values as outlined in his Cultural Dimensions Theory. The VSM has since been subject to a series of revisions. Yet, data on countries have been derived from the original VSM — and (...) not on one of the revised versions of VSM. We tested three scales (indulgence, power distance, and individualism) from the latest version, the VSM 2013, as part of a larger survey across 57 countries. Two main findings emerged. For one thing, country scores based on the VSM 2013 scales correlated only weakly with country scores of the same cultural dimensions obtained in a large previous study. Thus, the validity of the VSM 2013 is in doubt. For another thing, the internal consistency of the VSM 2013 scales was overall poor, indicating that the scales did not reliably measure well-defined constructs. We discuss implications for cross-cultural research. (shrink)

Reliability and external validity are two fundamental values that pose incompatible constraints on neurobiological experiments. The more reliability an experimental result achieves, the less external validity it earns, and vice versa. In this article, I propose an externalist interpretation of external validity: the external validity of an experimental result depends not only on how much complexity is built into an experimental design, but also on the relationship between the experimental result and other (...) related experiments. This externalist interpretation, which explains how a neurobiological experiment can be reliable and externally valid simultaneously, suggests a new way to understand the epistemology of neurobiological experimentation. (shrink)

It is sometimes argued that experimental economists do not have to worry about external validity so long as the design sticks closely to a theoretical model. This position mistakes the model for the theory. As a result, applied economics designs often study phenomena distinct from their stated objects of inquiry. Because the implemented models are abstract, they may provide improbable analogues to their stated subject matter. This problem is exacerbated by the relational character of the social world, which (...) also sets epistemic limits for the social science laboratory more generally. (shrink)

Narratives, also called stories, can be found in conversations, children’s play interactions, reading material, and television programs. From infancy to adulthood, narrative comprehension processes interpret events and inform our understanding of physical and social environments. These processes have been extensively studied to ascertain the multifaceted nature of narrative comprehension. From this research we know that three overlapping processes (i.e., knowledge integration, goal structure understanding, and causal inference generation) proposed by the constructionist paradigm are necessary for narrative comprehension, narrative comprehension has (...) a predictive relationship with children’s later reading performance, and comprehension processes are generalizable to other contexts. Much of the previous research has emphasized internal and predictive validity; thus, limiting the generalizability of previous findings. We are concerned these limitations may be excluding underrepresented populations from benefits and implications identified by early comprehension processes research. This review identifies gaps in extant literature regarding external validity and argues for increased emphasis on externally valid research. We highlight limited research on narrative comprehension processes in children from low-income and minority populations, and argue for changes in comprehension assessments. Specifically, we argue both on- and off-line assessments should be used across various narrative types (e.g., picture books, televised narratives) with traditionally underserved and underrepresented populations. We propose increasing the generalizability narrative comprehension processes research can inform persistent reading achievement gaps, and have practical implications for how children learn from narratives. (shrink)

Most of the existing literature on social preferences either tests whether certain characteristics of the social context influence individual decisions, or tries to estimate parameters of social preference functions describing such behavior at the level of the entire population. In the present paper, we are concerned with measuring parameters of social preference functions at the individual level. We draw upon concepts developed for eliciting other types of utility functions, in particular the literature on decision making under incomplete information. Our method (...) derives parameters of social preference functions from indifference statements about the distribution of payoffs a group. We apply our method in a controlled social preference experiment to establish the external validity of estimated parameters. Our results show the expected relationships to some external factors and also a strong correspondence between parameter estimates and factors that, according to the subjects’ own descriptions, influenced their behavior. We also find that some concepts discussed in the literature on social preferences, in particular envy toward players receiving a larger payoff, have very diverse and complex effects at the individual level. (shrink)

The experimental revolution in the social sciences is one of the most significant methodological shifts undergone by the field since the ‘quantitative revolution’ in the nineteenth century. One of the often valued features of social science experimentation is precisely the fact that there are clear methodological rules regarding hypothesis testing that come from the methods of the natural sciences and from the methodology of RCTs in the biomedical sciences, and that allow for the adjudication among contentious causal claims. We examine (...) critically this claim and argue that some current understandings of the practices that surround social science experimentation overestimate the degree to which experiments can actually fulfil this role as “objective” adjudicators, by neglecting the importance of shared background knowledge or assumptions and of consensus regarding the validity of the constructs involved in an experiment. We take issue with the way the distinction between internal and external validity is often used to comment on the inferential import of experiments, used both among practitioners and among philosophers of science. We describe the ways in which the more common use of the internal/external distinction differs from Cook and Campbell’s original methodological project, in which construct validity and the four-fold validity typology were all important in assessing the inferential import of experiments. We argue that the current uses of the labels internal and external, as applied to experimental validity, help to encroach a simplistic view on the inferential import of experiments that, in turn, misrepresents their capacity to provide objective knowledge about the causal relations between variables. (shrink)

I show that Eric Winsberg’s principles of model-building given in Science in the Age of Computer Simulation are insufficient to argue for the external validation of simulation data in cases in which simulation results conflict, and that laboratory experiments have an advantage over simulations because conflicting experimental results can be decided between on the basis of reproducibility. I also argue that robustness of predictions serves the same function for simulations as repeatability does for laboratory experiments in either adjudicating between (...) conflicting results or allowing us to say that we do not have sufficient justification to validate the results. Finally, I argue for an interpretation of the argument from robustness that appeals to the convergence of many well-built and diverse models rather than the more common interpretation which appeals to the probability that one of a set of models is likely to be true. (shrink)

Researchers often aim to make correct inferences both about that which is actually studied and about what the results generalize to. The language of internal and external validity is not used by everyone, but many of us would agree that intuitively the distinction makes a lot of sense. Two claims are commonly made with respect to internal and external validity. The first is that internal validity is prior to external validity since there is (...) nothing to generalize if the findings obtained in, for instance, the experimental setting do not hold. The first claim is explicit in many writings. See for instance Francisco Guala’s influential book The methodology of experimental economics. And it is often implicitly relied on. The second claim is that researchers have to make a trade-off between internal and external validity. When one is increased, the other will decrease. The second claim was made already from the start by D.T Campbell in his classic Factors relevant to the validity of experiments in social settings. There is a certain tension between the first and the second claim. It has been argued before that it might be difficult to combine them. We intend to make the stronger point that both claims are misconstrued. Our hypothesis is that the relationship between internal and external validity has to be re-conceptualized, and we will briefly indicate how. (shrink)

In Persson & Wallin we show that the common claim that internal validity should be understood as prior to external validity has, at least, three epistemologically problematic aspects: experimental artefacts, the implications of causal relations, and how the mechanism is measured. Each aspect demonstrates how important external validity is for the internal validity of the experimental result. This note is an invited summary of these results.

We show that the common claim that internal validity should be understood as prior to external validity has, at least, three epistemologically problematic aspects: experimental artefacts, the implications of causal relations, and how the mechanism is measured. Each aspect demonstrates how important external validity is for the internal validity of the experimental result.

If we know that certain ways of making decisions are associated with real-life success, is this then how we should decide? In this paper the relationship between normative and descriptive theories of decision-making is examined. First, it is shown that the history of the decision sciences ensures that it is impossible to separate descriptive theories from normative ones. Second, recent psychological research implies new ways of arguing from the descriptive to the normative. The paper ends with an evaluation of how (...) this might affect normative theories of decision-making. (shrink)

This paper investigates to what extent laboratory measures of cheating generalise to the field. To this purpose, we develop a lab measure that allows for individual-level observations of cheating whilst reducing the likelihood that participants feel observed. Decisions made in this laboratory task are then compared to individual choices taken in the field, where subjects can lie by misreporting their experimental earnings. We use two field variations that differ in the degree of anonymity of the field decision. According to our (...) measure, no correlation of behaviour between the laboratory and the field is found. We then perform the same analysis using a lab measure that can only detect cheating at the aggregate level. In this case, we do find a weak correlation between the two environments. We discuss the significance and interpretation of these results. (shrink)

As the technosciences, including genomics, develop into a global phenomenon, the question inevitably emerges whether and to what extent bioethics can and should become a globalised phenomenon as well. Could we somehow articulate a set of core principles or values that ought to be respected worldwide and that could serve as a universal guide or blueprint for bioethical regulations for embedding biotechnologies in various countries? This article considers one universal declaration, the UNESCO Declaration on Bioethics and Human Rights ( 2005a (...) ). General criticisms made in a recent special issue of Developing World Bioethics are that the concepts used in the Declaration are too general and vague to generate real commitment; that the so-called universal values are not universal; and, that UNESCO should not be engaged in producing such declarations which are the domain of professional bioethicists. This article considers these and other criticisms in detail and presents an example of an event in which the Declaration was used: the request by the Republic of Sakha, in Siberia, for a UNESCO delegation to advise on the initiation of a bioethics programme. The Declaration was intended to provide an adequate “framework of principles and procedures to guide states in the formulation of their legislation, policies and other instruments in the field of bioethics” (article 2a). The Declaration was produced, and principles agreed upon, in an interactive and deliberative manner with world-wide expert participation. We argue that the key issue is not whether the general principles can be exported worldwide (in principle they can), but rather how processes of implementation and institutionalisation should take shape in different social and cultural contexts. In particular, broader publics are not routinely involved in bioethical debate and policy-making processes worldwide. (shrink)

The self- vs. external-regulation behavior theory, SR-ER Theory model has postulated the Self-Regulation /Non or De-Regulation/Dys-regulation continuum in the person and in their context. The model also generates a behavioral heuristic that allows us to predict and explain the variability of other dependent behavioral variables in a range of scenarios. Consequently, the objective of this study was to validate the different scales prepared on the basis of the theory presented. A total of 469 students voluntarily completed at different times (...) the five questionnaires presented, to give a total of 1,385 completed questionnaires. Using an ex post facto design, descriptive, correlational, confirmatory factorial analysis, reliability, and concurrent validity analyses were carried out. The scales were analyzed individually and as a whole. The results showed the acceptable structure of scale and consistent levels of reliability. The five levels generated by the SR-NR-DR combinatory heuristic that arises from the theoretical model determined significant differences in the levels of the variables analyzed for each psychological context. We discuss the theoretical implications and the implications for the assessment and improvement of the behaviors analyzed in function of the personal and contextual regulation levels evaluated. (shrink)

Much of the methodological discussion around experiments in economics and other social sciences is framed in terms of the notions of internal and external validity. The standard view is that internal validity and external validity stand in a relationship best described as a _trade-off_. However, it is also commonly held that internal validity is a _prerequisite_ to external validity. This article addresses the problem of the compatibility of these two ideas and analyzes (...) critically the standard arguments about the conditions under which a trade-off between internal and external validity arises. Our argument stands against common associations of internal validity and external validity with the distinction between field and laboratory experiments and assesses critically the arguments that link the artificiality of experimental settings done in the laboratory with the purported trade-off between internal and external validity. We conclude that the idea of a trade-off or tension between internal and external validity seems, upon analysis, far less cogent than its intuitive attractiveness may lead us to think at first sight. (shrink)

Curry’s paradox is well known. The original version employed a conditional connective, and is not forthcoming if the conditional does not satisfy contraction. A newer version uses a validity predicate, instead of a conditional, and is not forthcoming if validity does not satisfy structural contraction. But there is a variation of the paradox which uses “external validity”. And since external validity contracts, one might expect the appropriate version of the Curry paradox to be inescapable. (...) In this paper we show that this is not the case. We consider two ways of formalising the notion of external validity, and show that in both of these the paradox is not forthcoming without the appropriate forms of contraction. (shrink)

The article identifies a latent debate in the recent literature on the role and worth of experiments in economics and other social sciences concerning the relationship between the external and the internal validity of experimental designs. Our work identifies two incompatible views regarding the relationship between internal and external validity of experiments. While in the methodological literature references to the idea that there is a trade-off between the internal and external validity of experiments abound, (...) this view coexists with the position stating that internal validity is rather a prerequisite of external validity. By identifying the contours of this implicit debate in the recent methodological literature around the use of experiments in the social sciences we call attention upon a series of insufficiently conceptualized issues regarding the central notions of internal and external validity and we question the standard view positing a trade-off between the two. This article stands against common associations of internal validity and external validity with the distinction between field and laboratory experiments and assesses critically the arguments that link the artificiality of experimental settings done in the laboratory with the purported trade-off between internal and external validity. (shrink)

Ethical decision making measures are widely applied as the principal dependent variable used in studies of research integrity. However, evidence bearing on the internal and external validity of these measures is not available. In this study, ethical decision making measures were administered to 102 graduate students in the biological, health, and social sciences, along with measures examining exposure to ethical breaches and the severity of punishments recommended. The ethical decision making measure was found to be related to exposure (...) to ethical events and the severity of punishments awarded. The implications of these findings for the application of ethical decision making measures are discussed. (shrink)

Clark and Chalmers claim that an external resource satisfying the following criteria counts as a memory: the agent has constant access to the resource; the information in the resource is directly available; retrieved information is automatically endorsed; information is stored as a consequence of past endorsement. Research on forgetting and metamemory shows that most of these criteria are not satisfied by biological memory, so they are inadequate. More psychologically realistic criteria generate a similar classification of standard putative external (...) memories, but the criteria still do not capture the function of memory. An adequate account of memory function, compatible with its evolution and its roles in prospection and imagination, suggests that external memory performs a function not performed by biological memory systems. External memory is thus not memory. This has implications for: extended mind theorizing, ecological validity of memory research, the causal theory of memory. (shrink)

Cognitive load theory has been widely used to help understand the process of learning and to design teaching interventions. The Cognitive Load Scale developed by Leppink and colleagues has emerged as one of the most validated and widely used self-report measures of intrinsic load, extraneous load, and germane load. In this paper we investigated an expansion of the CLS by using a multidimensional conceptualization of the EL construct that is relevant for physical and online teaching environments. The Multidimensional Cognitive Load (...) Scale for Physical and Online Lectures goes beyond the CLS's operationalization of EL by expanding the EL component which originally included factors related to instructions/explanations with sub-dimensions including EL stemming from noises, and EL stemming from both media and devices within the environment. Through three studies, we investigated the reliability, and internal and external validity of the MCLS-POL using the Partial Credit Model, Confirmatory Factor Analysis, and differences between students either attending a lecture physically or online. The results of Study 1 provide initial evidence for the validity and reliability of the MCLS-POL within a higher education sample, but also highlighted several potential improvements which could be made to the measure. These changes were made before re-evaluating the validity and reliability of the measure in a new sample of higher education psychology students, and psychological testing students. Together the studies provide evidence for a multidimensional conceptualization cognitive load and provide evidence of the validity, reliability, and sensitivity of the MCLS-POL and provide suggestions for future research directions. (shrink)

Cognitive diagnostic assessment has been developed rapidly to provide fine-grained diagnostic feedback on students’ subskills and to provide insights on remedial instructions in specific domains. To date, most cognitive diagnostic studies on reading tests have focused on retrofitting a single booklet from a large-scale assessment. Critical issues in CDA involve the scarcity of research to develop diagnostic tests and the lack of reliability and validity evidence. This study explored the development and validation of the Diagnostic Chinese Reading Comprehension Assessment (...) for primary students under the CDA framework. Reading attributes were synthesized based on a literature review, the national curriculum criteria, the results of expert panel judgments, and student think-aloud protocols. Then, the tentative attributes were used to construct three booklets of reading comprehension items for 2–6 graders at three key stages. The assessment was administered to a large population of students in grades 2–6 from 20 schools in a district of Changchun City, China. Q-matrices were compared and refined using the model-data fit and an empirical validation procedure, and five representative cognitive diagnostic models were compared for optimal performance. The fit indices suggested that a six-attribute structure and the G-DINA model were best fitted for the reading comprehension assessment. In addition, diagnostic reliability, construct, internal and external validity results were provided, supporting CDM classifications as reliable, accurate, and useful. Such diagnostic information could be utilized by students, teachers, and administrators of reading programs and instructions. (shrink)

Using data from four different samples of full-time employees (Ntotal = 2,474), the present study was aimed to introduce and demonstrate the validity and reliability of the Organizational-Based General Self-esteem Scale (OB-GSE) a new six-item self-report scale to measure organizational-based self-esteem (OBSE) at work. Results provided evidence of (1) validity (internal, external, and convergent), (2) reliability, and (3) temporal stability of the OB-GSE scale. All in all, results attested the usefulness and the effectiveness of the OB-GSE scale.

The problem of induction is to validate inferences from some experiences or data to others. By experiences or data I mean such things as red patches, sounds, tastes, pains. Distinctions between private and public data, between internal and external impressions, between data and objects, are not epistemologically primitive or given but are modes of categorizing the given. The application of categories and the construction of objects are cases of, and so presuppose the validity of, induction.To hypostatize a construction, (...) or a system of constructions, is in fact to generate what may be called an external world. Examples are physical objects, and scientific objects. And when analysis proceeds, as it often does, by replacing the original subject by constructs, a hierarchy of external systems is generated. Thus, physical objects—e.g., our bodies—are external to data, and scientific objects are external to physical objects. (shrink)

Professionals in business and law, healthcare providers, educators, policymakers, consumers, and higher education practitioners value ethical reasoning skills. Because of this, we concentrated campus-wide reaccreditation efforts to help students actively engage in ER. In doing so, we re-conceptualized the ER process, implemented campus-wide ER interventions designed to be experienced by all undergraduate students, and created the ethical reasoning identification test to measure students’ ability to engage in a foundational step in the ER process. Using factor analysis, we demonstrated internal (...) class='Hi'>validity evidence for ERIT scores. More specifically, confirmatory factor analysis provided support for a unidimensional factor structure, meaning stakeholders can report and analyze ERIT total scores. The unidimensional factor structure was replicated using two independent samples. Across all samples, ERIT scores demonstrated reliability consistent with professional standards. In addition, we collected external validity evidence for ERIT scores. The ERIT was sensitive to slight differences in ER training. That is, students experiencing a 75-min intervention performed better on the test compared to students without this experience. Overall, results suggested that our ER intervention may effectively increase students ER abilities and the ERIT demonstrated great potential for assessing foundational ethical reasoning skills. To further examine validity, researchers should consider known groups analyses with varying “doses” of the ER intervention, as well as measurement invariance studies. (shrink)

The importance of self-regulated learning has increased during the COVID-19 pandemic and measures for assessing students’ self-regulation skills and knowledge are greatly needed. We present the results of the first thorough adaptation of the Children’s Perceived use of Self-Regulated Learning Inventory. The inventory, consisting of 15 scales measuring nine components of SRL, was administered to a sample of 541 Slovenian ninth graders. Confirmatory factor analyses supported internal structure validity of most components, but two components required some structural modifications. Internal (...) consistency coefficients were acceptable for the majority of scale scores and were highly comparable to the original ones. While metric invariance across gender was confirmed, the scalar invariance of some scales needs further examination. Meaningful correlations with relevant externally assessed and self-reported self-regulation and school performance variables indicated good criterion validity of the inventory. The Slovenian version of the CP-SRLI thus proved to be a sufficiently valid and reliable instrument for assessing pupils’ learning self-regulation. (shrink)

Development and targeting efforts by academic organizations to effectively promote research integrity can be enhanced if they are able to collect reliable data to benchmark baseline conditions, to assess areas needing improvement, and to subsequently assess the impact of specific initiatives. To date, no standardized and validated tool has existed to serve this need. A web- and mail-based survey was administered in the second half of 2009 to 2,837 randomly selected biomedical and social science faculty and postdoctoral fellows at 40 (...) academic health centers in top-tier research universities in the United States. Measures included the Survey of Organizational Research Climate (SORC) as well as measures of perceptions of organizational justice. Exploratory and confirmatory factor analyses yielded seven subscales of organizational research climate, all of which demonstrated acceptable internal consistency (Cronbach’s α ranging from 0.81 to 0.87) and adequate test–retest reliability (Pearson r ranging from 0.72 to 0.83). A broad range of correlations between the seven subscales and five measures of organizational justice (unadjusted regression coefficients ranging from 0.13 to 0.95) document both construct and discriminant validity of the instrument. The SORC demonstrates good internal (alpha) and external reliability (test–retest) as well as both construct and discriminant validity. (shrink)