To measure non-cognitive facets of competence, we developed and tested a new method that we refer to as Embedded Experience Sampling (EES). Domain-specific problem-solving competence is a multi-faceted construct that is not limited to cognitive facets such as domain knowledge or problem-solving strategies but also comprises non-cognitive facets in the sense of domain-specific emotional and motivational dispositions such as interest and self-concept. However, in empirical studies non-cognitive facets are usually either neglected or measured by generalized self-report questionnaires that are detached (...) from the performance assessment. To enable an integrated measurement, we developed the EES method to collect data on non-cognitive facets during scenario-based low-stakes assessments. Test-takers are requested to stop at certain times and spontaneously answer short items (EES items) regarding their actual experience of the problem situation. These EES items are embedded in an EES event that resembles typical social interactions with non-player characters. To evaluate the feasibility and validity of the method, we implemented EES in a series of three studies in the context of commercial vocational education and training (VET): A feasibility study with 77 trainees, a pilot study with 20 trainees, and the main study with 780 trainees who worked on three complex problem scenarios in a computer-based office simulation. In the present paper, we investigate how test-takers perceived the EES events, and whether social desirability biased their answers, and investigate the internal structure of the data and the relationship between EES data and data from several other sources. Interview data and survey data indicated no biases due to social desirability and no additional burden for the test-takers due to the EES events. A correlation analysis following the multitrait-multimethod approach as well as the calibration of a multidimensional model based on Item Response Theory (IRT) also supported the construct validity. Furthermore, EES data shows substantial correlations with test motivation but almost zero correlations with data from generalized retrospective self-report questionnaires on non-cognitive facets. Altogether, EES offers an alternative approach to measuring non-cognitive facets of competence under certain conditions. For instance, EES is also based on self-reporting and thus might not be suitable for high-stakes testing. (shrink)

I present a puzzle for the standard, propositional semantic account of belief reports by considering novel inferences which it incorrectly predicts to be invalid under assumptions that are plausible by its advocates’ own lights. In response, I propose a conservative departure from the standard view on which certain ‘that’-clauses designate novel devices of semantic type that I call open propositions. After outlining some desiderata for a theory of open propositions, I provide some reasons for advocates of the standard view to (...) treat them as properties of a certain kind. Then I give a bridge principle between the core notions of belief and belief-about before showing how the resulting view can be implemented in accordance with formal theories of syntax and semantics. I bring out some of the consequences this investigation has beyond our semantic theorizing and conclude, more generally, that any response to the puzzle requires paying some surprising cost or another. (shrink)

In the context of office work, learning to handle an Enterprise Resource Planning system is important as implementation costs for such systems and associated expectations are high. However, these expectations are often not met because the users are not trained adequately. Electronic Performance Support Systems are designed to support employees’ ERP-related problem-solving and informal learning. EPSS are supposed to enhance employees’ performance and informal workplace learning through task-specific and granular help in task performance and problem-solving. However, there is little empirical (...) research on EPSS. Two survey studies addressed this research gap. In the first study, 301 people working in Human Resource -related positions and functions evaluated the learning potential of EPSS as well as potential advantages and obstacles concerning the implementation and use of EPSS. Though other measures are currently assessed as more important for learning, HR employees expect a strong increase in the significance of EPSS for employee learning. In the second study, 652 users of ERP software completed a questionnaire on characteristics of their daily work tasks, team characteristics, individual dispositions, their coping with ERP-related problems, and characteristics of EPSS. Findings indicate that the most frequently available and used approach when dealing with an ERP-related problem is consulting colleagues. Three EPSS types can be distinguished by their increasing integration into the user interface and their context-sensitivity. While external and extrinsic EPSS are available to many users, intrinsic EPSS are less common but are used intensively if available. EPSS availability is identified to be a strong positive predictor of frequency of EPSS use, while agreeableness as well as the task complexity and information-processing requirements show small negative effects. Moreover, more intensive ERP users use EPSS more frequently. In general, ERP users value, features such as context-sensitivity, an integration of the EPSS into the ERP system’s user interface, the option to save one’s own notes, and information displayed in an extra window. It is expected that EPSS will play an important role in workplace learning in the future, along with other measures. (shrink)