In this rejoinder, we respond to two commentaries on the study by Greiff, S.; Stadler, M.; Sonnleitner, P.; Wolff, C.; Martin, R. Sometimes less is more: Comparing the validity of complex problem solving measures. Intelligence 2015, 50, 100–113. The study was the first to address the important comparison between a classical measure of complex problem solving (CPS) and the more recent multiple complex systems (MCS) approach regarding their validity. In the study, we investigated the relations between one classical microworld as (...) the initially developed method (here, the Tailorshop) with three more recently developed multiple complex systems (MCS; here, MicroDYN, Genetics Lab, and MicroFIN) tests. We found that the MCS tests showed higher levels of convergent validity with each other than with the Tailorshop even after reasoning was controlled for, thus empirically distinguishing between the two approaches. The commentary by Kretzschmar and the commentary by Funke, Fischer, and Holt expressed several concerns with how our study was conducted, our data was analyzed, and our results were interpreted. Whereas we acknowledge and agree with some of the more general statements made in these commentaries, we respectfully disagree with others, or we consider them to be at least partially in contrast with the existing literature and the currently available empirical evidence. (shrink)

Influencing students’ educational achievements first requires understanding the underlying processes that lead to variation in students’ performance. Researchers are therefore increasingly interested in analyzing the differences in behavior displayed in educational assessments rather than merely assessing their outcomes. Such analyses provide valuable information on the differences between successful and unsuccessful students and help to design appropriate interventions. Complex problem solving (CPS) tasks have proven to provide particularly rich process data as they allow for a multitude of behaviors several of which (...) can lead to a successful performance. So far, this data has often been analyzed on a rather aggregated level looking at an average number of actions or predefined strategies. with only a few articles investigating the specific actions performed. In this paper, we illustrate and discuss a more detailed report the results of an exploratory analysis of CPS process data based on the specific actions performed.log-files that is aimed at distinguishing between students that applied the correct strategy to a problem but failed to solve it and those applying the strategy successfully. In that, the sequence of behavior displayed is reduced to interpretable parts (n-grams) that allow searching for meaningful differences between the two groups of students. This level of analysis allows finding previously undefined or unknown patterns within the data and increases our understanding of the processes underlying successful problem -solving behavior even further. (shrink)

In this paper we propose the multiple complex systems approach for assessing domain-general complex problem-solving skills and its processes knowledge acquisition and knowledge application. After defining the construct and the formal frameworks for describing complex problems, we emphasise some of the measurement issues inherent in assessing CPS skills with single tasks. With examples of the MicroDYN test and the MicroFIN test, we show how to adequately score problem-solving skills by using multiple tasks. We discuss implications for problem-solving research and the (...) assessment of CPS skills in general. (shrink)

The goal of the present Research Topic is to provide a forum where research groups, investigating teaching and teachers from multiple perspectives involving multidisciplinary (i.e., different disciplines working on different aspects of a problem independently within their disciplinary boundaries), interdisciplinary (i.e., restructuring and integrating existing disciplinary approaches to address problems relevant for all participating disciplines) and ideally transdisciplinary (i.e., seeking to integrate different lines of work from contributing disciplines to create new approaches or even new scientific disciplines) approaches (Hall, 2018; (...) Klein, 2017), can present and discuss the opportunities and challenges of such endeavors. The articles published in the research topic can be broadly classified into three categories: Conceptual reviews of transdisciplinary research on teaching and teachers, the results of transdisciplinary research projects, and methodological challenges and innovations related to transdisciplinary cooperation. Conceptual reviews of transdisciplinary researchThe Research Topic is initiated by Pea and Linn, who provide their personal perspectives on the emergence of the learning sciences as a transdisciplinary research community from the early 1970s to today. In line with the Research Topic’s aim, the paper illustrates how the specific approach of the learning sciences integrated approaches from disciplines as diverse as science education, psychology, an... (shrink)