Procedimentos para teste da unidimensionalidade

Transcrição

1 Procedimentos para teste da unidimensionalidade Prof. Dr. Ricardo Primi USF

2 Sumário Definição Modelos (uni, multi, bi-fatorial) Modelo operacional : unidimensionalidade essencial Procedimentos AFE scree plot AFE análise paralela AFE com itens dicotômicos Full information factor analysis (FIFA) Comparação Bi-Factor com Unidimensionalidade Análise fatorial dos resíduos AFC Programas SPSS: scree plot e análise paralela TESTFACT: AFE e AFC bi-factor e FIFA MPLUS AFE, AFC bi-factor e FIFA WINSTEPS

3 O que é unidimensionalidade? Uma dimensão controlando o escore no item Discriminação e carga fatorial

4 Construtos(latentes((exemplo(da(educação)( Domínio(de(saberes((MAT):(equações,(sistemas(de( equações,(funções((etc..)((( hcp:// Causa:(raciocínio(e(conhecimento(matemáJco( Transmissão((como(a(causa(se(estrutura):(via(educação( ensinolaprendizagem( Causa((latente)( Solução(de(problemas(matemáJcos((observáveis)( Gq(e( RQ((

5 Construtos(latentes((exemplo(da(medicina)( Roséola:(febre,(nariz(escorrendo,(tosse(e(pintas( vermelhas(na(pele( Transmissão((como(a(causa(se(estrutura):(via(saliva(( Causa((latente)( Sintomas((observáveis)(

6 Análise(Fatorial( Por(exemplo,(como(sabemos(se(um(conjunto( de(sintomas(tem(uma(causa(comum(?( Associação/correlação:(( Causa((latente)( Sintomas((observáveis)(?(

7 Análise(Fatorial:(psicologia( Estudo(fatorial(da(inteligência( Aplicam8se(vários(testes(de(conteúdos(diferentes( (raciocínio(lógico,(memória,(cria?vidade,(solução( de(problemas,(etc..)(( Calcula8se(uma(matriz(de(associações(entre(os( vários(testes((matriz(de(correlação)( Agrupam8se(os(testes(pelas(correlações( Analisam8se(os(testes(e(se(inferem(os(fatores( (causas(dos(desempenhos).(

8

9

10

11 Eigen values of original and simulated factors and components Análise(paralela( Eigen values of tetrachoric/polychoric matrix Factor Number PC Actual Data PC Simulated Data FA Actual Data FA Simulated Data Foram(extráidos(grupos(de(itens((fatores)(de(1(a(5(e(examinados( para(ver(se(faziam(sen;do( O(modelo(de(dois(fatores(foi(o(mais(coerente((fatores( correlacionados)( Eigen values of original and simulated factors and components Eigen values of tetrachoric/polychoric matrix Factor Number PC Actual Data PC Simulated Data FA Actual Data FA Simulated Data

12 Representação gráfica dos modelos SEM Qual Life Res (2007) 16: Fig. 1 Four possible latent variable models MODEL A MODEL B MODEL C MODEL D the researcher is hoping to measure versus how much is due to secondary dimensions? This is exactly the question five-factor extractions, with oblimin rotations. The five factor solution represents the a priori domains while the

13 Tipos&de&modelos&mul,dimensionais&

14 O que é multidimesnionalidade? 10 Chapter 16 ITEMS 1 LATENT DIMENSIONS ITEMS 1 LATENT DIMENSIONS Graphical Illustration 9 Between Item Multi-Dimensionality 9 Within Item Multi-Dimensionality a j1 = 1.8, a j2 =.3, d j =.3, D j = Figure A Graphic Depiction of Within and Between Item Multi-dimensionality 294 Assessment 18(3) 5. FIRST ISSUE: WITHIN VERSUS BETWEEN MULTIDIMENSIONALITY 1.0 To assist in the discussion of different types of multidimensional models 0.8 and tests we have introduced the notions of within and between item multidimensionality (Adams, Wilson & Wang, 1997; Wang, 1994; Wang & 0.6 Wilson, in 1996). A test is regarded as multidimensional between item if it is made up of several unidimensional sub-scales. A test is considered multidimensional within item at least one of the items relate to more than one 0.4 latent dimension The Multidimensional Between-Item Model. Tests that 0contain several 0.0 sub-scales each measuring related, but supposedly distinct, latent dimensions are very commonly encountered 2 in practice. In such tests each 2 item belongs 0 to only one particular sub-scale and there are no items in common across the 2 sub-scales. In the past, item response modelling of such tests has proceeded by either (a) applying a unidimensional model to each of the scales Anxiety Figure 2. Item characteristic surface for a multidimensional item response model Anxiety and depression are used as examples of two distinct latent variables that both influence the probability of item endorsement. Depression see Ackerman (19 to the standard ICC in order to account for the second latent segunda-feira, variable. 16 de setembro de 13 scales measuring psychological syndromes. Nevertheless, because of the beneficial properties of the model, fit of a EDMS 724 Modern Measurement Theories (Spring 2008, Dr. André A. Rupp)

15 Unidimensionalidade essencial Pressuposto de quase todas as análises clássicas! Um fator dominante e fatores secundários, embora existam, são negligenciáveis 10 Chapter 16 ITEMS 1 LATENT DIMENSIONS ITEMS 1 LATENT DIMENSIONS Between Item Multi-Dimensionality 9 Within Item Multi-Dimensionality Figure A Graphic Depiction of Within and Between Item Multi-dimensionality 5. FIRST ISSUE: WITHIN VERSUS BETWEEN MULTIDIMENSIONALITY To assist in the discussion of different types of multidimensional models

16 Procedimentos SPSS: scree plot Eigenvalue X ordem Eigen 1 / Eigen 2 > 5 SPSS: RanEingen (Enzmann, 1997) Cria V variáveis para N sujeitos com distribuição aleatória. r v1 vs rv2 = 0. (dados paralelos) Extrai os fatores dessa matriz Repete i vezes Calcula a média dos primeiros, segundos, terceiros... fatores Enzmann, D. (1997). RanEigen: a program to determine the parallel analysis criterion for the number of principal components. Applied Psychological Measurement, 21, 232. ( Mitarbeiter/Enzmann/Software/Enzmann_Software.html)

17 !

18 Exemplo1 ENEM 2006 Resultados Dois fatores 20,6% e 2,9% da variância (Eig1/Eig2 = 4,54, Rotação Promax r (f1 vs f2)=0,67 Modelagem Rasch e Extração de contrastes r =0,50 quinta-feira, 2 de junho de 2011

19 Exemplo 2: QFCP SPSS...

20 Análise Fatorial Exploratória (AFE) com itens dicotômicos FERGUSON, G. A. (1941). The factorial interpretation of test dificulty. Psychometrika, 6 (5):

21

22

23 Full information factor analysis e análise fatorial dos FIFA (Testfact) Análise fatorial via TRI resíduos Estima-se os parâmetros (geralmente o modelo de 3 parâmetros, c é informado) e deriva-se as cargas a partir dos parâmetros a. TRI multidimensional! Análise fatorial dos resíduos (WINSTEPS) Aplica-se o modelo unidimensional (1o fator) todos os itens Calcula-se os resíduos Roda-se a análise de componentes principais nos resíduos (2o, 3o, 4o fatores) sem rotação Dividi-se os itens que formam grupos nos contrastes Verifica-se se os escores estimados a partir desses itens estão correlacionados (avalia-se a interferência das dimensões secundárias)

24 Comparação Bi-Factor com Unidimensionalidade Em essência uma AFC Passo 1: análise unidimensional tradicional Passo 2: análise bifatorial Compara-se as cargas no fator geral das duas soluções Qual Life Res (2007) 16: Fig. 1 Four possible latent variable models MODEL A MODEL B Se a os fatores específicos não distorcem a medida as cargas no fator geral de ambos os modelos serão similares. Isso é interpretado que, mesmo com a presença de fatores específicos a medida do fator geral não é distorcida MODEL C MODEL D the researcher is hoping to measure versus how much is due to secondary dimensions? This is exactly the question the bifactor model can address. In most IRT applications, researchers explore whether their item-level data are sufficient for Model A as opposed to Models B or C. We argue that the more interesting comparisons are between Model A (unidimensional) versus Model D (bifactor), and the choice between Model C (multiple correlated dimensions) versus Model D. For the remainder, we will explore these comparisons using the Ó five-factor extractions, with oblimin rotations. The five factor solution represents the a priori domains while the two factor solution represents a plausible alternative [15]. MICROFACT [39] was used to conduct these analyses. The first column in Table 3 displays the loadings under Model A. Although the items vary widely in their loadings, an argument can be made that there is a strong general dimension here. All items load reasonably well (>.40) on the first factor; the first five eigenvalues for this matrix are 6.8, 1.4,.9,.8, and,.7 and thus the ratio of the first to

25 The role of the bifactor model in resolving dimensionality issues in health outcomes measures Steven P. Reise Æ Julien Morizot Æ Ron D. Hays Received: 25 August 2006 / Accepted: 30 January 2007 / Published online: 4 May 2007 Ó Springer Science+Business Media B.V Abstract Objectives We propose the application of a bifactor model for exploring the dimensional structure of an item response matrix, and for handling multidimensionality. Background We argue that a bifactor analysis can complement traditional dimensionality investigations by: (a) providing an evaluation of the distortion that may occur when unidimensional models are fit to multidimensional data, (b) allowing researchers to examine the utility of forming subscales, and, (c) providing an alternative to nonhierarchical multidimensional models for scaling individual differences. Method To demonstrate our arguments, we use responses (N = 1,000 Medicaid recipients) to 16 items in the Consumer Assessment of Healthcare Providers and Systems (CAHPS Ó 2.0) survey. Analyses Exploratory and confirmatory factor analytic and item response theory models (unidimensional, multidimensional, and bifactor) were estimated. Results CAHPS Ó items are consistent with both unidimensional and multidimensional solutions. However, the bifactor model revealed that the overwhelming majority of common variance was due to a general factor. After controlling for the general factor, subscales provided little measurement precision. Conclusion The bifactor model provides a valuable tool for exploring dimensionality related questions. In the Discussion, we describe contexts where a bifactor analysis is most productively used, and we contrast bifactor with multidimensional IRT models (MIRT). We also describe S. P. Reise (&) J. Morizot R. D. Hays Department of Psychology, University of California, Franz Hall, Los Angeles, CA , USA reise@psych.ucla.edu implications of bifactor models for IRT applications, and raise some limitations. Keywords Bifactor model Unidimensionality assumption Item response theory Multidimensional item response model Health outcomes measurement Item response theory (IRT) [1] methods were developed in the context of large-scale assessment to more efficiently and accurately measure broadband dimensional constructs such as verbal and quantitative aptitude. In recent years, IRT methods have been applied in a wider variety of substantive contexts, especially health outcomes research [2 6]. This article is not an introductory or didactic review of IRT methods as applied in the health outcomes domain. Such reports are available in a variety of sources [7, 8]. Rather, this article is aimed toward researchers who are familiar with IRT methods, and who are actively working on IRT applications. We assume the reader is familiar with the literature that relates factor analytic and IRT models [9 13]. We discuss two related topics in the application of IRT models to health outcomes measures: (a) (uni)dimensionality assessment, and (b) application of hierarchical versus non-hierarchical multidimensional models. We first draw a distinction between narrow and broad constructs/measures. We then argue that for broader measures with diverse indicators, researchers should consider use of the bifactor model for representing the (multi)dimensional structure of their data. We argue that a bifactor model: (a) allows for the examination of the distortion that may occur when unidimensional IRT models are fit to multidimensional data, (b) allows researchers to empirically examine the utility of forming subscales, and, (c) provides an alternative 123

26 La utilización del modelo bifactorial para testar la unidimensionalidad de una batería de pruebas de raciocinio.! 3 The use of the bi-factor model to test the uni-dimensionality of a battery of reasoning tests Abstract Running Head: Item factor analysis of a Battery of Reasoning Tests Ricardo Primi, Marjorie Cristina Rocha da Silva and Priscila Rodrigues Santana, Graduate Program in Psychology, University of San Francisco, Brazil Monalisa Muniz, University of Vale do Sapucaí, Brazil and Leandro S. Almeida Institute of Education, University of Minho, Portugal. The Battery of Reasoning Tests 5 (BPR-5) aims to assess the reasoning ability of individuals using sub-tests with different formats and contents that require basic processes of inductive and deductive reasoning in their resolution. The BPR has three sequential forms: BPR-5i (for children from first to fifth grade), BPR-5 Form A (for children from sixth to eight grade) and BPR-5 form B (for high school and undergraduate students). The present study analysed 412 questionnaires concerning BPR-5i, 603 questionnaires concerning BPR-5 Form A and 1748 questionnaires concerning BPR-5 Form B. The main goal was to test the uni-dimensionality of the battery and its tests in relation to items using the bi-factor model. Results have indicated that the assumption of a general reasoning factor underlying different contents items is supported. Key-words: Battery of reasoning tests, factorial validity, item response theory, bi-factor model Please address correspondence to: Ricardo Primi Universidade São Francisco Laboratório de Avaliação Psicológica e Educacional (LabAPE) Rua Alexandre Rodrigues Barbosa, 45 CEP , Itatiba São Paulo, Brazil. rprimi@mac.com Web: The research activities of the first author, which resulted in this article, are financed by the Brazilian National Council for Scientific Research (CNPq) and the São Paulo Research Foundation (FAPESP). Resumén La Batería de Pruebas de Raciocinio (BPR-5) tiene como objetivo evaluar la capacidad de razonamiento de las personas utilizando pruebas menores con diferentes ítems y contenidos, pero que presentan relaciones en lo referente a la inducción y la deducción que intervienen en su resolución de la tarea. La BPR tiene una organización secuencial: BPR-5i (para niños de 1º a 5º grado), BPR-5 versión A (del 6º al 8º grado) y BPR-5 versión B (enseñanza secundaria y terciaria). El presente estudio evaluó los datos de 289 protocolos de la BPR-5i, 603 de la BPR-5 versión A y 1748 de la BPR-5 versión B. El objetivo principal fue poner a prueba la unidimensionalidad de la batería y de las pruebas que la componen. Los resultados confirmaron la existencia de un factor único relatado con el razonamiento independientemente del contenido de las tareas. Palabras clave: Batería de Pruebas de Raciocinio, validez factorial, desarrollo cognitivo, teoría de respuesta al ítem, calibración de la prueba. 1!

27 ! 22 Exemplo BPR-5 A, B e i Figure 1. Item examples of BPR sub-tests Abstract Reasoning A B C D E? Verbal Reasoning Day : Night is as Bright : A. Light B. Energy C. Dark D. Clarity E. Cloud Numerical Reasoning ?? Spatial Reasoning A B C D E Mechanical Reasoning What level (A, B, C) allows a person to reach a greater depth after jumping? If equal mark D. Practical Reasoning (Only in BPR-5i) John's house is nearby Anthony s home. One house is white and the other is grey. Anthony's house is not white. State what is the colour of the house of each of these two men.

28 ! 18 Table 1. Summary results of bi-factor and full information factor models BPR-5i BPR-5 A BPR-5 B Bi-factor model g bi (%) s AR (%) s VR (%) s MR (%) s SR (%) s NR (%) s PR (%) Uniqueness Reliability (of g-factor) χ / / /1443 CFI TLI RMSEA Uni-dimensional model (Full information) Eig 1 / Eig /6.4 (5.6) 37.7/5.15 (7.32) 39.37/6.41 (6.14) g full Reliability (of g-factor) χ / / /1558 CFI TLI RMSEA Corrected Chi-Square difference test for the weighted least squares estimator (WLSMV) χ df p <.0001 <.0001 <.0001

29 Figure 2. Scatter plots of factor loadings in g factor with the bi-dimensional model for the youngest children (top), A (lower left) and B Forms (lower right).