Raven, J., Prieler, J., & Benesch, M. (June 15, 2005). A Replication and Extension of the Item-Analysis of the Standard Progressive Matrices Plus, Together With a Comparison of the Results of Applying Three Variants of Item Response Theory. WebPsychEmpiricist. Retrieved (date) from http://wpe.info/papers_table.html

In 2003 Raven’s Standard Progressive Matrices Plus (SPM+) test was standardised on a nationally representative sample of 2,755 Romanians, aged 6 to 80. Using this data set it was possible to replicate and extend the Item Response Theory (IRT) based item analysis that had been conducted while developing the test. The correlation between the 1-parameter item difficulties (in logits) from the two studies was.96. More importantly, however, when the effects of applying different variants of IRT were compared, two striking conclusions emerged: (i) adoption of a one-parameter model - ie the most commonly employed variant of IRT - to data that really require a 3-parameter model can lead to seriously misleading conclusions. And, interestingly, as much or more can be learned by using the “unsophisticated” methods deployed by Raven in 1935 than by more recent statistical packages. (ii) The Figures displaying the Item Characteristic Curves for all 60 items of the SPM+ yield remarkable evidence of the scientific “existence” and scalability of Eductive (meaning-making) Ability. While these results are not new in an absolute sense, they will be new to many psychometricians, especially those steeped in classical test theory.

PDF version of article