The reliability and factor validity of the SPARO questionnaire
Abstract
In 2009, the SPARO questionnaire was one of the most used assessment methods in the Czech Republic (Urbánek, 2010), especially between psychologists in human resources, prisons, police, or the army. However, empirical evidence of its reliability and validity is missing. Using a significant sample of police officers and applicants to police, we researched the questionnaire's reliability and factor validity. About half of the scales do not have adequate internal consistency; the majority of them are not unidimensional. Factor validity is questionable, especially in the main components of the questionnaire, which are highly correlated. These correlations were also present in the original standardized samples as our and original correlation matrices do not differ. The factor structure of the two general factors items is not clear and two dimensional. Our results do not support the use of the questionnaire in practice. It is necessary to confirm its’ predictive validity; the criterion validity is evaluated in our follow up study (Cígler & Rudá, 2021).
https://doi.org/10.29364/epsy.391
(Fulltext in Czech)
Keywords
internal consistency, factor validity, SPARO, Police of the Czech Republic, personal selectionLiterature
Bentler, P. M., & Woodward, J. A. (1980). Inequalities among lower bounds to reliability: With applications to test construction and factor analysis. Psychometrika, 45(2), 249–267. https://doi.org/…7/BF02294079
Bentler, Peter M. (2009). Alpha, dimension-free, and model-based internal consistency reliability. Psychometrika, 74(1), 137–143. https://doi.org/…6-008-9100-1
Benton, T. (2015). An empirical assessment of Guttman’s Lambda 4 reliability coefficient. In R. Millsap, D. Bolt, L. van der Ark, & W. C. Wang (Eds.), Quantitative Psychology Research. Springer Proceedings in Mathematics & Statistics, vol. 89 (pp. 301–310). Springer. https://doi.org/…-319-07503-7
Cígler, H., & Rudá, A. (2021). Souběžná validita dotazníku SPARO a MMPI 2. E-psychologie, 15(1), 40–68. https://doi.org/…64/epsy.392/
Corr, P. J., & Matthews, G. (2009). The Cambridge handbook of personality psychology. Cambridge University Press.
Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests. Psychometrika, 16(3), 297–334. https://doi.org/…7/BF02310555
Guttman, L. (1945). A basis for analyzing test-retest reliability. Psychometrika, 10(4), 255–282. https://doi.org/…7/BF02288892
Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. https://doi.org/…519909540118
Jorgensen, T. (2020). Function lavTestLRT() for Model Comparison. [Online forum post]. Google Groups. Available on groups.google.com/forum/#!topic/lavaan/nNFcxgzodAs
Jorgensen, T. D., Pornprasertmanit, S., Schoemann, A. M., & Rosseel, Y. (2020). semTools: Useful tools for structural equation modeling (0.5–3). https://cran.r-project.org/…age=semTools
Jurišová, E., & Sarmány-Schuller, I. (2013). Structure of basal psychical self-regulation and personality integration in relation to coping strategies in decision-making in paramedics. Studia Psychologica, 55(1), 3–18. https://doi.org/….2013.01.617
Kenny, D. A., Kaniskan, B., & McCoach, D. B. (2014). The performance of RMSEA in models with small degrees of freedom. Sociological Methods & Research, 44(3), 486–507. https://doi.org/…124114543236
Kuder, G. F., & Richardson, M. W. (1937). The theory of the estimation of test reliability. Psychometrika, 2(3), 151–160. https://doi.org/…7/BF02288391
Lakens, D. (2017). Equivalence tests. Social Psychological and Personality Science, 8(4), 355–362. https://doi.org/…550617697177
Leary, M. R., & Hoyle, R. H. (Eds.). (2009). Handbook of individual differences in social behavior. The Guilford Press.
MacCallum, R. C., Browne, M. W., & Sugawara, H. M. (1996). Power analysis and determination of sample size for covariance structure modeling. Psychological Methods, 1(2), 130–149. https://doi.org/…989X.1.2.130
Mangiafico, S. (2020). rcompanion: Functions to Support Extension Education Program Evaluation (R package version 2.3.21). https://cran.r-project.org/…e=rcompanion
Marko, M. (2016). Využitie a zneužitie Cronbachovej alfy pri hodnotení psychodiagnostických nástrojov. TESTFÓRUM, 5(7). https://doi.org/…/TF2016-7-90
McDonald, R. P. (1999). Test theory: A unified treatment. Lawrence Erlbaum Associates, Inc.
Mikšík, O. (1992). IHAVEZ – SPIDO – VAROS (příručka). Psychodiagnostika.
Mikšík, O. (2001). Zjišťování bazální struktury a dynamiky autoregulace, integrovanosti a psychické odolnosti osobnosti dotazníky SPARO, BAROM, IHATRANS (manuál). Diaros.
Mikšík, O. (2003). Psychologické teorie osobnosti. Karolinum.
Mikšík, O. (2004). Dotazník SPARO (příručka). Psychodiagnostika.
Mikšík, O. (2007). Psychologická charakteristika osobnosti. Karolinum.
Mikšík, O. (2009). Psychika osobnosti v období závažných životních a společenských změn. Karolinum.
Phillips, N. (2017). yarrr: A Companion to the e-Book “YaRrr!: The Pirate’s Guide to R” (R package version 0.1.5). https://cran.r-project.org/package=yarrr
Preiss, J., & Haas, T. (1997). Některé psychometrické charakteristiky české verze Washingtonského psychosociálního dotazníku pro záchvatová onemocnění (WPSI). Československá psychologie, 41(4), 334–346.
R Core Team. (2020). R: A language and environment for statistical computing. R Foundation for Statistical Computing.
Raykov, T. (1997). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21(2), 173–184. https://doi.org/…216970212006
Revelle, W. (1978). Hierarchical cluster analysis and the internal structure of tests. Multivariate Behavioral Research, 14(1), 57–74. https://doi.org/…906mbr1401_4
Revelle, W. (2019). psych: Procedures for Personality and Psychological Research (1.9.12). Northwestern University. https://cran.r-project.org/package=psych
Revelle, W., & Zinbarg, R. E. (2009). Coefficients Alpha, Beta, Omega, and the glb: Comments on Sijtsma. Psychometrika, 74(1), 145–154. https://doi.org/…6-008-9102-z
Rosseel, Y. (2012). lavaan: An R Package for structural equation modeling. Journal of Statistical Software, 48(2), 1–36. http://www.jstatsoft.org/v48/i02/
Satorra, A. (2000). Scaled and Adjusted Restricted Tests in Multi-Sample Analysis of Moment Structures (pp. 233–247). Springer US. https://doi.org/…15-4603-0_17
Signorell, A., & et al. (2020). DescTools: Tools for descriptive statistics (R package version 0.99.32). https://cran.r-project.org/…ge=DescTools
Sijtsma, K. (2009). On the use, the misuse, and the very limited usefulness of Cronbach’s Alpha. Psychometrika, 74(1), 107–120. https://doi.org/…6-008-9101-0
Šucha, M., Šťastná, L., & Zámečník, P. (2017). Osobnostní vlastnosti řidičů řídících pod vlivem alkoholu. Adiktologie, 17(1), 34–44.
Svoboda, M., Řehan, V., Vtípil, Z., Klimusová, H., Humpolíček, P., Urbánek, T., & Kouhoutek, T. (2004). Aplikovaná psychodiagnostika v České republice. MSD.
Torchiano, M. (2019). effsize: Efficient Effect Size Computation (R package version 0.7.6). https://doi.org/…nodo.1480624
Urbánek, T. (2010). Nejpoužívanější psychodiagnostické metody v České republice. TESTFÓRUM, 1(1), 6–9. https://doi.org/…7/TF2010-1-3
Wickham, H., & Bryan, J. (2019). readxl: Read Excel Files (R package version 1.3.1). https://cran.r-project.org/package=readxl