What makes a measure of well-being valid? The dominant approach today, construct validation, uses psychometrics to ensure that questionnaires behave in accordance with background knowledge. Our first claim is interpretive—construct validation obeys a coherentist logic that seeks to balance diverse sources of evidence about the construct in question. Our second claim is critical—while in theory this logic is defensible, in practice it does not secure valid measures. We argue that the practice of construct (...) class='Hi'>validation in well-being research is theory avoidant, favoring a narrow focus on statistical tests while largely ignoring relevant philosophical considerations. (shrink)

This paper looks at the question of what it means for a psychological test to have construct validity. I approach this topic by way of an analysis of recent debates about the measurement of implicit social cognition. After showing that there is little theoretical agreement about implicit social cognition, and that the predictive validity of implicit tests appears to be low, I turn to a debate about their construct validity. I show that there are two questions at stake: (...) First, what level of detail and precision does a construct have to possess such that a test can in principle be valid relative to it? And second, what kind of evidence needs to be in place such that a test can be regarded as validated relative to a given construct? I argue that construct validity is not an all-or-nothing affair. It can come in degrees, because both our constructs and our knowledge of the explanatory relation between constructs and data can vary in accuracy and level of detail, and a test can fail to measure all of the features associated with a construct. I conclude by arguing in favor of greater philosophical attention to processes of construct development. (shrink)

Carroll (1991) encouraged researchers in Social Issues Management (SIM) to continue to measure Corporate Social Performance (CSP) from a variety of different perspectives utilizing a variety of different measures. In addition, Wolfe and Aupperle (1991) (and others) have asserted that there is no, single best way to measure CSP and that multiple measures and perspectives help develop the field. However, Pfeffer (1993) suggest that a lack of consistent measurement has constrained organization studies (and by implication, the field of social issues (...) management,) in its development as a field. It may be in the best interest of social issues management researchers to try to development a common body of measures and data. Recently, Kinder, Lydenberg and Domini & Co. (KLD — a social choice investment advisory firm) has made available their social performance database. The KLD data have potential to become a widely accepted set of CSP measures. The purpose of this paper is to present a construct validity study comparing the KLD data to other measures of CSP. (shrink)

Scholars and providers are coming to realize that one’s ability to notice and respond to internal body sensations contributes to an overall sense of wellbeing. Research has demonstrated a relationship between interoceptive awareness and anxiety, for example. Currently, however, tools for evaluating one’s interoception lack the conceptual foundation and clarity necessary to identify everyday behaviors that specifically reflect interoceptive awareness. Unlike existing interoceptive measures, the Sensory Profile Interoception scale is participation-based and grounded in Dunn’s Sensory Processing framework. In this study (...) we investigated concurrent validity by correlating the SPI with the Adolescent/Adult Sensory Profile ; we investigated construct validity by correlating the SPI with the Perth Alexithymia Scale, the Body Awareness Scale, and the State-Trait Anxiety Inventory. Using the REDCAP online platform, 74 college-aged participants completed the measures. Using Spearman rank order correlations there were statistically significant relationships between the corresponding sensory pattern subscales on SPI and A/ASP. The PAS was only weakly related to the registration subscale of the SPI. The BAS had significant relationships with seeking and avoiding on the SPI. The STAI had significant relationships with sensitivity and registrations of the SPI. Taken together, these findings provide evidence of construct validity of the SPI to identify participation patterns associated with both high and low interoception. With the more precise information the SPI provides, professionals can design tailored interventions to support everyday life goals and researchers can study interoception within authentic activities. (shrink)

Introduction:The concept of mentalizing is nowadays widely used in research as well as in clinical practice. Despite its popularity, the development of an economic assessment is still challenging. The Mentalization Scale appears to be a promising measurement with good psychometric properties but lacking convergent validity with the Reflective Functioning Scale.Objective:This study aims to test the construct validity of the Mentalization Scale through correlations with the gold standard, the Reflective Functioning Scale, within a clinical sample. Furthermore, it was of interest (...) to replicate its internal consistency.Methods:Twenty-six inpatients of an acute psychiatric ward in Vienna were given the Mentalization Scale (MentS). They were interviewed with the Brief Reflective Function Interview, which was coded with the Reflective Functioning Scale. Correlations and internal consistency were calculated.Results:Concerning the primary aim of this study, the validity was satisfactory for the MentS whole-scale mentalizing as well as for the subscales self- and other-oriented mentalizing. Internal consistency was lower to the findings of the developer and close to the 0.70 threshold.Conclusion:Our findings could foster the psychometric properties of the MentS. Furthermore, the MentS seems to be a promising measurement tool for detecting different dimensions of reflective functioning. Limitations and further research are discussed. (shrink)

Fully automated vehicles (AVs) hold promise toward providing numerous societal benefits including reducing road fatalities. However, we are uncertain about how individuals’ perceptions will influence their ability to accept and adopt AVs. The 28-item Automated Vehicle User Perception Survey (AVUPS) is a visual analog scale that was previously constructed, with established face and content validity, to assess individuals’ perceptions of AVs. In this study, we examined construct validity, via exploratory factor analysis and subsequent Mokken scale analyses. Next, internal consistency (...) was assessed via Cronbach’s alpha (α) and 2-week test–retest reliability was assessed via Spearman’s rho (ρ) and intraclass correlation coefficient (ICC). The Mokken scale analyses resulted in a refined 20-item AVUPS and three Mokken subscales assessing specific domains of adults’ perceptions of AVs: (a)Intention to use; (b)perceived barriers; and (c)well-being. The Mokken scale analysis showed that all item-coefficients of homogeneity (H) exceeded 0.3, indicating that the items reflect a single latent variable. The AVUPS indicated a strong Mokken scale (Hscale= 0.51) with excellent internal consistency (α = 0.95) and test–retest reliability (ρ = 0.76, ICC = 0.95). Similarly, the three Mokken subscales ranged from moderate to strong (rangeHscale= 0.47–0.66) and had excellent internal consistency (range α = 0.84–0.94) and test–retest reliability (range ICC = 0.84–0.93). The AVUPS and three Mokken subscales of AV acceptance were validated in a moderate sample size (N= 312) of adults living in the United States. Two-week test–retest reliability was established using a subset of Amazon Mechanical Turk participants (N= 84). The AVUPS, or any combination of the three subscales, can be used to validly and reliably assess adults’ perceptions before and after being exposed to AVs. The AVUPS can be used to quantify adults’ acceptance of fully AVs. (shrink)

Precise measurements of well-being would be of profound societal importance. Yet, the sceptical worry that we cannot use social science instruments and tests to measure well-being is widely discussed by philosophers and scientists. A recent and interesting philosophical argument has pointed to the psychometric procedures of construct validation to address this sceptical worry. The argument has proposed that these procedures could warrant confidence in our ability to measure well-being. The present paper evaluates whether this type of argument succeeds. (...) The answer is that it depends on which methodological background assumptions are motivating the sceptical worry to begin with. We show this by doing two things. First, we clarify (a) the different types of well-being theories involved in the science of well-being, and (b) the general methodological dimensions of well-being theorising. Second, we apply these distinctions and argue that construct validation is an unsuccessful response to measurement scepticism if this scepticism is motivated by a form of methodological non-naturalism. In the light of this, the overall point of the paper is that philosophers and scientists, when discussing measurement of well-being, should explicate their deeper methodological commitments. We further suggest that making such explicit commitments might present philosophers with a dilemma. (shrink)

Colleges of business rankings purport to address relative performance on programs such as sustainability. The primary criticism of rankings is that providers have not established reliability or validity of the ranking. This study examines whether The Global 100 sustainability ranking is sufficiently unique to claim that it is based on attributes not used for non-sustainability ranking (divergent validity) and whether it is appropriately related to independent characteristics expected to measure this attribute (convergent validity).

This volume is an interdisciplinary attempt to insert a broader, historically informed perspective into current political and academic debates on the issue of evidence and the reliability of scientific knowledge. Evidence in Action is the perfect resource for all those interested in the relationship between science, technology, and the role of knowledge in society.

Validity is a central principle of assessment relating to the appropriateness of the uses and interpretations of test results. Usually, one of the inferences that we wish to make is that the score reflects the extent of a student?s learning in a given domain. Thus, it is important to establish that the assessment tasks elicit performances that reflect the intended constructs. This research explored the use of three methods for evaluating whether there are threats to validity in relation to the (...) constructs elicited in international A level geography examinations: (a) Rasch analysis; (b) analysis of processes expected and apparent when students answer questions; and (c) qualitative analysis of responses to items identified as potentially problematic. The results provided strong evidence to support validity with regard to the elicitation of constructs although one question part was identified as a threat to validity. Strengths and weaknesses of the methods can be identified. (shrink)

Business ethics has gained much attention over the past decade in both work and educational settings. This study used a version of Lysonski and Gaidis' ethical vignettes to examine by gender the ethical views of 165 Canadian undergraduate management students, to examine the psychometric properties and construct validity of the instrument, and to determine if the instrument is a useful tool for introducing undergraduates to the topic of ethics in management practice. Results showed that while the instrument is a (...) useful teaching tool, it has psychometric and validity inadequacies that need to be corrected. (shrink)

The Discounting Inventory, originally developed in polish language, allows the measurement of individual differences in the delay, probabilistic, effort, and social discounting rates. The present study attempted to validate the DI’s psychometric properties using German university students and to compare the results to those from a sample of Polish university students. Over four hundred participants completed the DI and traditional discounting measures. A confirmatory factor analyses indicated that the original four-factor model of the DI provided an excellent fit for the (...) German data and internal consistency was high. These outcomes were similar, if not superior, to those from the Polish sample. DI scores strongly correlated with traditional discounting measures scores in both samples, replicating previous results. These findings indicate that the DI is a valid measure for use in a sample from another cultural setting, which is potentially useful to both researchers and practitioners. (shrink)

Psychologists say a measure has construct validity when it, in fact, measures the construct it is intended to measure. Construct validity is both an important notion in psychological research methods and the source of much confusion and debate among psychologists. I argue that this confusion arises, in part, because of a failure to distinguish between construct validity, a feature of measures relative to a construct, and construct legitimacy, a feature of the construct itself. (...) I propose a prescriptive account of construct validity on the basis of this distinction, then provide evidence for my account through two examples from research psychology. (shrink)

The experimental revolution in the social sciences is one of the most significant methodological shifts undergone by the field since the ‘quantitative revolution’ in the nineteenth century. One of the often valued features of social science experimentation is precisely the fact that there are clear methodological rules regarding hypothesis testing that come from the methods of the natural sciences and from the methodology of RCTs in the biomedical sciences, and that allow for the adjudication among contentious causal claims. We examine (...) critically this claim and argue that some current understandings of the practices that surround social science experimentation overestimate the degree to which experiments can actually fulfil this role as “objective” adjudicators, by neglecting the importance of shared background knowledge or assumptions and of consensus regarding the validity of the constructs involved in an experiment. We take issue with the way the distinction between internal and external validity is often used to comment on the inferential import of experiments, used both among practitioners and among philosophers of science. We describe the ways in which the more common use of the internal/external distinction differs from Cook and Campbell’s original methodological project, in which construct validity and the four-fold validity typology were all important in assessing the inferential import of experiments. We argue that the current uses of the labels internal and external, as applied to experimental validity, help to encroach a simplistic view on the inferential import of experiments that, in turn, misrepresents their capacity to provide objective knowledge about the causal relations between variables. (shrink)

Because of the misspecification of models and specificity of operationalizations, many studies produce claims of limited utility. We suggest a path forward that requires taking a few steps back. Researchers can retool large-scale replications to conduct the descriptive research which assesses the generalizability of constructs. Large-scale construct validation is feasible and a necessary next step in addressing the generalizability crisis.

Accurately assessing youth mental health involves obtaining reports from multiple informants who typically display low levels of correspondence. This low correspondence may reflect situational specificity. That is, youth vary as to where they display mental health concerns and informants vary as to where and from what perspective they observe youth. Despite the frequent need to understand and interpret these informant discrepancies, no consensus guidelines exist for integrating informants’ reports. The path to building these guidelines starts with identifying factors that reliably (...) predict the level and form of these informant discrepancies, and do so for theoretically and empirically relevant reasons. Yet, despite the knowledge of situational specificity, few approaches to integrating multi-informant data are well-equipped to account for these factors in measurement, and those that claim to be well-positioned to do so have undergone little empirical scrutiny. One promising approach was developed roughly 20 years ago by Kraemer and colleagues. Their Satellite Model leverages principal components analysis and strategic selection of informants to instantiate situational specificity in measurement, namely components reflecting variance attributable to the context in which informants observe behavior, the perspective from which they observe behavior, and behavior that manifests across contexts and perspectives. The current study represents the first construct validation test of the Satellite Model. A mixed-clinical/community sample of 134 adolescents and their parents completed six parallel surveys of adolescent mental health. Adolescents also participated in a series of simulated social interactions with research personnel trained to act as same-age, unfamiliar peers. A third informant viewed these interactions and completed the same surveys as parents and adolescents. We applied the Satellite Model to each set of surveys and observed high internal consistency estimates for each of the six-item trait, context, and perspective components. Scores reflecting the trait, context, and perspective components displayed distinct patterns of relations to a battery of criterion variables that varied in the context, perspective, and source of measurement. The Satellite Model instantiates situational specificity in measurement and facilitates unifying conceptual and measurement models of youth mental health. (shrink)

Entry of new organizations, including multinational enterprises from emerging markets, raises the ethical question of will they benefit society. The concept of legitimacy answers this question because it is the overall assessment of the appropriateness of organizational ends and means. Moreover, gaining legitimacy enables EMNEs to succeed in new host countries. Past work examined collective level indicators of the legitimacy of MNEs, but recent research recognizes the importance of individuals’ perceptions as the micro-foundation of legitimacy. This study first uses new (...) pragmatism, deontology, and utilitarianism to demonstrate that legitimacy is fundamentally an ethical concept—a perspective that has been overlooked in management research. Second, this study uses a seven-step procedure to develop and validate a measure of individuals’ perceptions of the legitimacy of Chinese EMNEs operating in The Netherlands, a developed country. Six dimensions of legitimacy were identified. The study also finds support for this legitimacy judgment process linking the dimensions: validating knowledge → propriety judgments → generalized judgment. This work provides additional micro-foundations to research on legitimacy and contributes to the ongoing process of construct validation. Future research could use the validated measure in other settings and use specific ethical theories in depth to refine the concept of legitimacy. (shrink)

Roommate relationships are fundamental to the social environment of many emerging adults. However, no validated, widely used, measure of roommate relationship quality exists for examining the impact of these relationships on individual functioning and health. In this report, we present preliminary evidence of the factor structure, concurrent validity, and construct validity of the Roommate Relationship Scale as a measure of roommate relationship quality using a sample of U.S. college students who participated in a multi-wave study. An exploratory factor analysis (...) at the first wave, and confirmatory factor analyses with independent samples of new participants at each of two subsequent waves showed stable factor loadings and adequate fit. Moreover, the scale demonstrated good fit and reliability in a longitudinal multilevel CFA framework. The RRS significantly positively correlated with relationship length, self-esteem, extraversion, agreeableness, and conscientiousness, and negatively correlated with symptoms of anxiety and avoidant attachment style, indicating concurrent validity of the scale with respect to these constructs. Consistent with findings from other relationship types, self-reported RRS scores decrease longitudinally, both across and between semesters of academic life, indicating construct validity of the scale. We conclude that the RRS is useful for evaluating roommate relationship quality among U.S. college students, and hopefully beyond. Further research should validate the scale’s utility in other, more diverse, populations and refine its underlying generating psychological process. (shrink)

One of the major roadblocks in conducting Environmental Corporate Social Responsibility (ECSR) research is operationalization of the construct. Existing ECSR measurement tools either require primary data gathering or special subscriptions to proprietary databases that have limited replicability. We address this deficiency by developing a transparent ECSR measure, with an explicit coding scheme, that strictly relies on publicly available data. Our ECSR measure tests favorably for internal consistency and inter-rater reliability, as well as convergent and discriminant validity.

We advance the understanding of the philosophy and psychology of curiosity by operationalizing and constructing an empirical measure of Nietzsche’s conception of inquisitive curiosity, expressed by the German term Wissbegier, (“thirst for knowledge” or “need/impetus to know”) and Neugier (“curiosity” or “inquisitiveness”). First, we show that existing empirical measures of curiosity do not tap the construct of inquisitive curiosity, though they may tap related constructs such as idle curiosity and phenomenological curiosity. Next, we map the concept of inquisitive curiosity (...) and connect it to related concepts, such as open-mindedness and intellectual humility. The bulk of the paper reports four studies: an Anglophone exploratory factor analysis, an Anglophone confirmatory factor analysis, an informant study, and a Germanophone exploratory and confirmatory factor analysis. (shrink)