Abstract

Today, all important educational achievement studies, particularly large scale assessments, use item response theory (IRT) as the standard method for their analyses. An important and very basic assumption of IRT is on the given dimensionality of a test: In order to be interpreted unidimensional a test has to be unidimensional and hence cannot be interpreted multidimensional. In reality though, this basic assumption is very often neglected. The Programme for International Student Assessment (PISA), for example, applies a unidimensional IRT-Model for the analysis of the mathematics achievement and at the same time applies a multidimensional IRT-model for the analysis of the subscales Change and Relationships, Quantity, Space and Shape, and Uncertainty and Data. This contradiction to one of the basic assumptions of IRT is also included in other well known large scale assessments. Strangely enough, non of these studies discuss the issue. This work discusses the advantage and disadvantages of the currently used approaches for a concurrent uni- and multidimensional analysis, and a new approach is presented, which bases on an IRT model, the generalized subdimension model (GSM), which allows the calculation of a weighted mean score within the IRT framework. Besides the demonstration of different applications for the GSM and its predecessor, the subdimension model, the model’s characteristics are compared to those of other models, such as hierarchical models. Beyond the comparison of the model fit, that is, the reliability of the results, the discussion particularly focuses on the difference in the interpretation, that is, on the validity of the results.