Meyer, G. J., Hilsenroth, M. J., Baxter, D., Exner, J. E., Jr., Fowler, J. C., Piers, C. C. & Resnick, J. (2002). An examination of interrater reliability for scoring the Rorschach comprehensive system in eight data sets. Journal of Personality Assessment, 78, 219-274.
Described interrater reliability for the Comprehensive System (CS) in 8 relatively large samples, including (1) students, (2) experienced researchers, (3) clinicians, (4) clinicians and then researchers, (5) a composite clinical sample (i.e., 1 to 4), and 3 samples in which randomly generated erroneous scores were substituted for (6) 10%, (7) 20%, or (8) 30% of the original responses. Across samples, 133 to 143 statistically stable CS scores had excellent reliability. The authors also demonstrate reliability findings from this study that closely match the results derived from a synthesis of prior research. Results show that CS summary scores are more reliable than scores assigned to individual responses, small samples are more likely to generate unstable and lower reliability estimates, and G. J. Meyer's (1997) procedures for estimating response segment reliability were accurate. It is concluded that the CS can be scored reliably, but because scoring is the result of coder skills clinicians must conscientiously monitor their accuracy. (PsycINFO Database Record (c) 2002 APA, all rights reserved)