Empirical agreement is often used as an important criterion when assessing the validity of scientific models. However, it is by no means a sufficient criterion as a model can be so adjusted as to fit available data even though it is based on hypotheses whose plausibility is known to be questionable. Our aim in this paper is to investigate into the uses of empirical agreement within the process of model validation.

I introduce a new method for validating models—including stochastic models—that gets at the reliability of a model’s predictions under intervention or manipulation of its inputs and not merely at its predictive reliability under passive observation. The method is derived from philosophical work on natural kinds, and turns on comparing the dynamical symmetries of a model with those of its target, where dynamical symmetries are interventions on model variables that commute with time evolution. I demonstrate that this method (...) succeeds in testing aspects of model validity for which few other tools exist. (shrink)

This paper will discuss the recent LIGO-Virgo observations of gravitational waves and the binary black hole mergers that produce them. These observations rely on having prior knowledge of the dynamical behaviour of binary black hole systems, as governed by the Einstein Field Equations (EFEs). However, we currently lack any exact, analytic solutions to the EFEs describing such systems. In the absence of such solutions, a range of modelling approaches are used to mediate between the dynamical equations and the experimental data. (...) Models based on post-Newtonian approximation, the effective one-body formalism, and numerical relativity simulations (and combinations of these) bridge the gap between theory and observations and make the LIGO-Virgo experiments possible. In particular, this paper will consider how such models are validated as accurate descriptions of real-world binary black hole mergers (and the resulting gravitational waves) in the face of an epistemic circularity problem: the validity of these models must be assumed to justify claims about gravitational wave sources, but this validity can only be established based on these same observations. (shrink)

This paper responds to a recent challenge for the validity of extrapolation of neurobiological knowledge from laboratory animals to humans. According to this challenge, experimental neurobiology, and thus neuroscience, is in a state of crisis because the knowledge produced in different laboratories hardly generalizes from one laboratory to another. Presumably, this is so because neurobiological laboratories use simplified animal models of human conditions that differ across laboratories. By contrast, I argue that maintaining a multiplicity of experimental protocols and simple (...) models is well justified. It fosters rather than precludes the validity of extrapolation of neurobiological knowledge. The discipline is thriving. (shrink)

Verification and validation of numerical models of natural systems is impossible. This is because natural systems are never closed and because model results are always nonunique. Models can be confirmed by the demonstration of agreement between observation and prediction, but confirmation is inherently partial. Complete confirmation is logically precluded by the fallacy of affirming the consequent and by incomplete access to natural phenomena. Models can only be evaluated in relative terms, and their predictive value is always open to question. (...) The primary value of models is heuristic. (shrink)

Introduction to modelling and validation -- Non-hierarchical coloured petri nets -- CPN ML programming -- Formal definition of non-hierarchical coloured petri nets -- Hierarchical coloured petri nets -- Formal defintion of hierarchical coloured petri nets -- State spaces and behavioural properties -- Advanced state space methods -- Formal definition of state spaces and behavioural properties -- Timed coloured petri nets -- Formal definition of timed coloured petri nets -- Simulation-based performance analysis -- Behavioural visualisation -- Examples of industrial applications -- (...) Teaching coloured petri nets. (shrink)

Since Economics survey by Windrum et al., research on empirical validation of agent-based Agent-based model in Economics has made substantial advances, thanks to a constant flow of high-quality contributions. This Chapter attempts to take stock of such recent literature to offer an updated critical review of the existing validation techniques. We sketch a simple theoretical framework that conceptualizes existing validation approaches, which we examine along three different dimensions: Comparison between artificial and real-world Data; Calibration and estimation of model (...) parameters; and parameter space Exploration. Finally, we discuss open issues in the field of ABM validation and Estimation. In particular, we argue that more research efforts should be devoted toward advancing HypothesisTesting in ABM, with specific emphasis on model stationarity and ergodicity. (shrink)

We describe a new approach for developing and validating cognitive process models. We develop graphical models (specifically, hidden Markov models) both from human empirical data on a task, as well as from synthetic data traces generated by a cognitive process model of human behavior on the task. We show that considering differences between the two graphical models can unveil substantive and nuanced imperfections of cognitive process models that can then be addressed to increase their fidelity to empirical data.

본 연구는 “교육학적 추론을 위한 베이지언 모델”의 형식적 타당성 및 이 모델이 갖는 교육학적 함의와 인식론적 함의에 대해 비판적으로 검토한다.BR 베이즈주의 학습이론가들에 따르면, 교육학적 목표를 가장 잘 성취하기 위해서는 “정확한 가설”(h)에 대한 학습자의 믿음을 최대화하는 “데이터”(d)를 교사가 선택해야 한다. 달리 말하면, 학생이 추측하는 문제의 가설(개념)이 교사가 목표로 하는 바로 그 가설(개념)에 최대로 가까워지게 하는 예시를 교사가 학생에게 제공해야 한다. 이를 위해서는 교사가 생산하는 “데이터의 분포”(p teacher (d｜h))가 “가설(h)에 대한 학습자의 사후 믿음”(p learner (h｜d))을 최대화하는 데이터들을 중심으로 균등하게 분포되어야 할 것이다. (...) 학생은 처음부터 문제의 개념을 소유하고 있지 않다. 학생은 교사가 제공하는 예시를 통해서 비로소 개념을 구성해야 하는 위치에 있다. 이를 위해서 우선 교사는 학생의 인지 상태를 알고 있어야 한다. 즉 학생이 어떤 데이터를 접하게 될 때, 어떤 개념을 상상할 것이라고 교사는 예측해야 한다. 결국 교사는 “p teacher (d｜h)” 와 “p learner (h｜d)”, 양자 모두를 알아야 한다. 이러한 맥락에서 교사가 활용할 수 있는 모델이 아래와 같은 방정식이다.BR 이것이 베이지언 학습이론가들이 지지하는 “교육학적 추론 모델”이다. 그들에 따르면, 이 방정식의 해는 문제의 개념(가설)을 가장 잘 전달하게 할 예시 선택을 가능하게 하는 대안이다. 그런데 이 모델이 구체적 교육 현장에서 어떻게 적용될 수 있는가를 보기 위해서, 또한 이 모델이 가지는 논리-형식적 타당성 및 교육-인식론적 함의를 이해하기 위해서, 필자는 베이지언 이론가들이 제시하는 많은 예시들 중에서 가장 단순한 예시인 “선 게임 이론”(theory of line game)을 분석하고자 한다.BR. (shrink)

We agree that external validity of social psychological experiments is a concern, we disagree these models are useless. Experiments, reconsidered from a situated cognition perspective and non-linearly combined with other methods allow grasping decision dynamics beyond bias outcomes. Dynamic insights regarding these processes are key to understand missing forces and bias in real-world social groups.

Take a formula of first-order logic which is a logical consequence of some other formulae according to model theory, and in all those formulae replace schematic letters with English expressions. Is the argument resulting from the replacement valid in the sense that the premisses could not have been true without the conclusion also being true? Can we reason from the model-theoretic concept of logical consequence to the modal concept of validity? Yes, if the model theory is (...) the standard one for sentential logic; no, if it is the standard one for the predicate calculus; and yes, if it is a certain model theory for free logic. These conclusions rely inter alia on some assumptions about possible worlds, which are mapped into the models of model theory. Plural quantification is used in the last section, while part of the reasoning is relegated to an appendix that includes a proof of completeness for a version of free logic. (shrink)

PAMVEC is a novel vehicle modeling tool designed to complement the capabilities of dynamic vehicle simulators and be better-suited to the purposes of vehicle technology assessment. This paper presents a validation of PAMVEC against published data for a range of production and pre-production prototype vehicles whose powertrain technologies span the range currently being exhibited by automotive manufacturers. For each vehicle, the PAMVEC model was used to predict the fuel/energy consumption and required peak powertrain output due to acceleration performance. Errors (...) typically. (shrink)

The current study seeks to validate L2 grit measure among 637 Chinese senior middle school students using a bifactor modeling approach. To do so, we first assessed and compared four alternative measurement models including CFA, bifactor CFA, ESEM, and bifactor ESEM models. Among these models, CFA exhibited the poorest fit to the data collected from the sample. ESEM showed partial fit to the data with a slightly lower factor correlation between two components of L2 grit than CFA. Two bifactor models (...) demonstrated excellent fits to the data. The more parsimonious bifactor CFA model was selected as the optimal one. Based on the bifactor CFA model, we confirmed measurement invariance across gender and predictive validity of L2 grit on subsequent language achievements. Based on these findings, methodological and pedagogical implications were discussed. (shrink)

Cognitive diagnostic assessment has been developed rapidly to provide fine-grained diagnostic feedback on students’ subskills and to provide insights on remedial instructions in specific domains. To date, most cognitive diagnostic studies on reading tests have focused on retrofitting a single booklet from a large-scale assessment. Critical issues in CDA involve the scarcity of research to develop diagnostic tests and the lack of reliability and validity evidence. This study explored the development and validation of the Diagnostic Chinese Reading Comprehension Assessment (...) for primary students under the CDA framework. Reading attributes were synthesized based on a literature review, the national curriculum criteria, the results of expert panel judgments, and student think-aloud protocols. Then, the tentative attributes were used to construct three booklets of reading comprehension items for 2–6 graders at three key stages. The assessment was administered to a large population of students in grades 2–6 from 20 schools in a district of Changchun City, China. Q-matrices were compared and refined using the model-data fit and an empirical validation procedure, and five representative cognitive diagnostic models were compared for optimal performance. The fit indices suggested that a six-attribute structure and the G-DINA model were best fitted for the reading comprehension assessment. In addition, diagnostic reliability, construct, internal and external validity results were provided, supporting CDM classifications as reliable, accurate, and useful. Such diagnostic information could be utilized by students, teachers, and administrators of reading programs and instructions. (shrink)

Controlled elicitation of linguistic and psycholinguistic experimental data facilitate strong inferences about phonological, syntactic, semantic and pragmatic structures and functions, yet neglect the ecological validity of responses. Ecological validity in this paper refers to whether data gathered under controlled conditions are commensurate with routine problem solving and language use in natural settings. All methods produce "white room" effects that compromise data gathering and analysis. Unexamined folk knowledge and experiences also guide the investigator s interpretation of data from field (...) research, laboratories, testing sessions, and target sentences. Story recall data from children in a combined second-third grade classroom and their narrative responses to cartoons without sound are used to illustrate ecological validity issues. The child's spontaneous, imaginative narratives about the story and cartoons resemble adult folk theory, and their semantic content reveal sociocultural knowledge essential for understanding subjects ' reasoning and information processing skills. (shrink)

In recent years, workplace well-being has been a popular research topic, because it is helpful to promote employees’ welfare, thereby bringing valuable personal and organizational outcomes. With the development of technology, the technology industry plays an important role in Taiwan. Although the salary and benefits provided by the technology industry are better than other industries, the work often requires a lot of time and effort. It is worth paying attention to whether a happy workplace will bring subjective well-being for the (...) technology industry in Taiwan. This study explored the influence of workplace well-being, job involvement, and flow on the subjective well-being. The research was conducted by a questionnaire survey. A total of 256 employees in the technology industry in the Nanzi Processing Zone in Kaohsiung City, Taiwan were surveyed. Collected data were analyzed by statistical methods, such as multivariate and structural equation models. The study results indicated that workplace well-being, flow, and job involvement have a positive and significant impact on the subjective well-being. In addition to having a direct impact on subjective well-being, flow is also a significant variable to mediate the impact of workplace well-being to subjective well-being. In addition, job involvement also affects subjective well-being through flow, which means that the state of selflessness at work is the most important factor affecting subjective well-being. Finally, based on the research findings, the researcher provided practical suggestions to the government and the technology industry. (shrink)

This paper provides a historical analysis of a shift in the way animal models of mental disorders were conceptualized: the shift from the mid-twentieth-century view, adopted by some, that animal models model syndromes classified in manuals such as the Diagnostic and Statistical Manual of Mental Disorders (DSM), to the later widespread view that animal models model component parts of psychiatric syndromes. I argue that in the middle of the twentieth century the attempt to maximize the face validity (...) of animal models sometimes led to the pursuit of the ideal of an animal model that represented a behaviorally defined psychiatric syndrome as described in manuals such as the DSM. I show how developments within psychiatric genetics and related criticism of the DSM in the 1990s and 2000s led to the rejection of this ideal and how researchers in the first decade of the twenty-first century came to believe that animal models of mental disorders should model component parts of mental disorders, adopting a so-called endophenotype approach. (shrink)

Psychiatric research is in crisis because of repeated failures to discover new drugs for mental disorders. Lack of measurement validity could partly account for these failures. If researchers do not actually measure the effects of drugs on the disorders they aim to investigate, one should expect suboptimal treatment outcomes. I argue that this is the case, focusing on depression, and fear & anxiety disorders. In doing so, I show how psychiatric research illustrates a more general phenomenon that I call (...) “validity drift.” A validity drift occurs when, in the course of developing new measurement procedures, researchers end up studying something different from what they originally aimed to study. I analyze the different ways in which scientists attempt to validate procedures for measuring mental disorders for the purpose of testing drugs in animal models. I show how those validation efforts might fail, thereby leading to validity drifts. Overall, this analysis highlights the complex interplay between the development of new measurement procedures, their calibration, and scientific theories of the target phenomena. (shrink)

Animal models of human disease play a central role in modern biomedical science. Developing animal models for human mental illness presents unique practical and philosophical challenges. In this article we argue that existing animal models of psychiatric disease are not valid, attempts to model syndromes are undermined by current nosology, models of symptoms are rife with circular logic and anthropomorphism, any model must make unjustified assumptions about subjective experience, and any model deemed valid would be inherently unethical, (...) for if an animal adequately models human subjective experience, then there is no morally relevant difference between that animal and a human. (shrink)

Byrne has demonstrated that although subjects can make deductively valid inferences of the modus ponens and modus tollens forms, these valid inferences can be suppressed by presenting an appropriate additional premise “If R then Q” with the original conditional “If P then Q”. This suppression effect challenges the assumption of all syntactic theories of conditional reasoning that formal rules of inference such as modus ponens is part of mental logic. This paper argues that both the syntactic and the mental (...) class='Hi'>model accounts of the suppression effect are inadequate because they fail to give a principled account of the critical interpretive component involved in reasoning. In contrast, the relative salience model proposed in this study emphasized the centrality of the interpretative processes with the critical component being the relative salience of premises as judged by subjects on the basis of their prior knowledge activated in particular problem situations. Using 120 undergraduates and 120 policemen as subjects, predictions from the model were tested and confirmed in a suppression paradigm and evidence of convergent validity for the construct of salience were obtained. The results cannot be reconciled with either the syntactic view or the mental model view that have dominated theories of conditional reasoning. (shrink)

Prior to statistical mechanics and especially before the advent of the new quantum mechanics, it was traditionally held, following in the main the Kantian philosophy, that the task of science is to attain a unique quantitative representation of reality. It was thought—and with justifiable zeal—that a scientific discipline is exact to the extent to which a mathematical pattern yielding quantitative relations can be applied to its subject matter. This view was based on the implicit assumptions that functional relatedness conducive to (...) quantitative evaluation is bound to reveal the most generic, and hence, the most essential traits of events. (shrink)

A key issue in the business ethics field is the design of effective measures for assessing the ethical culture of organizations. The Corporate Ethical Virtues Model (CEV), developed by Kaptein in 2008, is an instrument for measuring ethical culture, and has been applied, adapted and validated in different contexts. In 2013, DeBode, Armenakis, Field and Walker developed the CEV–S, a shortened version of the original scale. Both the CEV and CEV–S assess eight dimensions based on corporate ethical virtues: clarity, (...) congruency of supervisors, congruency of senior management, feasibility, supportability, transparency, discussability and sanctionability. The objective of this study is to present a Spanish version of the CEV–S and its validation process. The scale was applied to two samples of 114 and 193 Colombian employees from two companies operating in the services sector. The confirmatory factor analyses revealed a good fit for the eight-factor structure of the Spanish CEV–S and the statistical tests applied showed good levels of reliability and validity for this scale. The adequate psychometric properties shown by the Spanish CEV–S may support its use in future research to measure organizational ethical culture in Spanish-speaking settings. (shrink)

How to accept a conditional? F. P. Ramsey proposed the following test in . 'If A, then B' must be accepted with respect to the current epistemic state iff the minimal hypothetical change of it needed to accept A also requires accepting B. In this article we propose a formulation of , which unlike some of its predecessors, is compatible with our best theory of belief revision, the so-called AGM theory , chapters 1-5 for a survey). The new test, which, (...) we claim, encodes some of the crucial insights defended by F. P. Ramsey in , is used to study the conditionals epistemically validated by the AGM postulates. Our notion of validity is compared with the notion of negative validity used by Gärdenfors in . It is observed that the notions of PV and NV will in general differ and that when these differences arise it is the notion of PV that is preferable. Finally we compare our formulation of the Ramsey test with a previous formulation offered by Gärdenfors . We show that any attempt to interpret as delivering acceptance conditions for Ramsey's conditionals is doomed to failure. (shrink)

The problem analysed in this paper is whether we can gain knowledge by using valid inferences, and how we can explain this process from a model-theoretic perspective. According to the paradox of inference (Cohen & Nagel 1936/1998, 173), it is logically impossible for an inference to be both valid and its conclusion to possess novelty with respect to the premises. I argue in this paper that valid inference has an epistemic significance, i.e., it can be used by an agent (...) to enlarge his knowledge, and this significance can be accounted in model-theoretic terms. I will argue first that the paradox is based on an equivocation, namely, it arises because logical containment, i.e., logical implication, is identified with epistemological containment, i.e., the knowledge of the premises entails the knowledge of the conclusion. Second, I will argue that a truth-conditional theory of meaning has the necessary resources to explain the epistemic significance of valid inferences. I will explain this epistemic significance starting from Carnap’s semantic theory of meaning and Tarski’s notion of satisfaction. In this way I will counter (Prawitz 2012b)’s claim that a truth-conditional theory of meaning is not able to account the legitimacy of valid inferences, i.e., their epistemic significance. (shrink)

Qualitative validation consists in showing that a model is able to mimic available observed data. In population level biological models, the available data frequently represent a group status, such as pool testing, rather than the individual statuses. They are aggregated. Our objective was to explore an approach for qualitative validation of a model with aggregated data and to apply it to validate a stochastic model simulating the bovine viral-diarrhoea virus (BVDV) spread within a dairy cattle herd. Repeated (...) measures of the level of BVDV-specific antibodies in the bulk-tank milk (total milk production of a herd) were used to summarise the BVDV herd status. First, a domain of validation was defined to ensure a comparison restricted to dynamics of pathogen spread well identified among observed aggregated data (new herd infection with a wide BVDV spread). For simulations, scenarios were defined and simulation outputs at the individual animal level were aggregated at the herd level using an aggregation function. Comparison was done only for observed data and simulated aggregated outputs that were in the domain of validation. The validity of our BVDV model was not rejected. Drawbacks and ways of improvement of the approach are discussed. (shrink)

ABSTRACTThe present study describes the development and validation of a facial expression database comprising five different horizontal face angles in dynamic and static presentations. The database includes twelve expression types portrayed by eight Japanese models. This database was inspired by the dimensional and categorical model of emotions: surprise, fear, sadness, anger with open mouth, anger with closed mouth, disgust with open mouth, disgust with closed mouth, excitement, happiness, relaxation, sleepiness, and neutral. The expressions were validated using emotion classification and (...) Affect Grid rating tasks [Russell, Weiss, & Mendelsohn, 1989. Affect Grid: A single-item scale of pleasure and arousal. Journal of Personality and Social Psychology, 57, 493–502]. The results indicate that most of the expressions were recognised as the intended emotions and could systematically represent affective valence and arousal. Furthermore, face angle and facial motion information influenced emot... (shrink)

Attempts by 20th century philosophers of science to define inductive concepts and methods concerning the support provided to scientific theories by empirical data have been unsuccessful. Although 20th century philosophers of science largely ignored statistical methods for testing theories, when they did address them they argued against rather than for their use. In contrast, this study demonstrates that traditional statistical methods used for validating computer simulation models provide tests of the scientific theories that those models may embody. This study shows (...) that these statistical methods are applicable regardless of whether the scientific theory or model is deterministic, stochastic, or chaos-based in nature. Traditional statistical methods, such as hypothesis testing, test theories by providing measures of their goodness-of-fit with the portion of the world the theories describe. Nothing more is needed to portray the agreement between theory and the world. Furthermore, the statistical measures of goodness-of-fit are not ampliative inductive measures. Therefore, the ultimate value and direction of the 20 th century ampliative inductive program within philosophy of science is thrown into question. This study also suggests that advanced statistical, visualization, and hybrid quantitative qualitative methods hold promise for offering scientists additional methods for testing scientific theories through validating their computer model representations. As a prerequisite to the above discussions, this study proves that scientific theories, when applied to concrete physical systems, can indeed be represented by computer models. (shrink)

The -logic of Terwijn is a variant of first-order logic with the same syntax in which the models are equipped with probability measures and the quantifier is interpreted as ‘there exists a set A of a measure such that for each,...’. Previously, Kuyper and Terwijn proved that the general satisfiability and validity problems for this logic are, i) for rational, respectively -complete and -hard, and ii) for, respectively decidable and -complete. The adjective ‘general’ here means ‘uniformly over all languages’. (...) We extend these results in the scenario of finite models. In particular, we show that the problems of satisfiability and validity with respect to finite models in E-logic are, i) for rational, respectively -complete and -complete, and ii) for, respectively decidable and -complete. Although partial results toward the countable case are also achieved, the computability of E-logic over countable models still remains largely unsolved. In addition, most o... (shrink)

Simple heuristics and regression models make different assumptions about behaviour. Both the environment and judgment can be described as fast and frugal. We do not know whether humans are successful when being fast and frugal. We must assess both global accuracy and the costs of Type I and II errors. These may be “smart heuristics that make researchers look simple.”.

Recent studies of Italian past definite and past participle forms show that human performance with regular and irregular inflections is not dissociated as Clahsen's model would predict. Some performance profiles, accounted for by dual-mechanism models in terms of an underlying symbol-manipulating combinatorial procedure, are generated in Italian by the higher learnability and generalizability of phonologically regular morphological processes.

BackgroundCurrent policies regulating the use of animals for scientific purposes are based on balancing between potential gain of knowledge and suffering of animals used in experimentation. The balancing process is complicated, on the one hand by plurality of views on our duties towards animals, and on the other hand by more recent discussions on uncertainty in the probability of reaching the final aim of the research and problems of translational failure.MethodsThe study combines ethical analysis based on a literature review with (...) neuropsychiatry-related preclinical research as a case study.ResultsBased on the analysis and the case study we show that neuropsychiatry-related preclinical research is an especially interesting case from an ethical perspective. The 3R principles are used to minimize the negative consequences for the animals used in research. However, neuropsychiatric research is characterized by specific challenges in assessing the probability of success of reaching the final aim, due to our limited mechanistic knowledge of human neuropsychiatric illness. Consequently, the translational value of the currently used animal models may be difficult to prove, which undermines the validity of these models and complicated the ethical assessment.ConclusionsWe conclude that a combined approach that deals with both science and the ethical dimensions is necessary to address the problems of validity and translation in neuropsychiatry-related preclinical research. We suggest this approach to comprise first, improved experimental methods, e.g. by using systematic reviews, second, a more patients-based approach that leads to models that reflect interindividual variation better, and third, more interdisciplinary cooperation. (shrink)