# Full text of "DTIC ADA144869: An Investigation of the Effect of Correlated Abilities on Observed Test Characteristics."

## See other formats

^flD-fll44 869 UNCLASSIFIED AN INVESTIGATION OF THE EFFECT OF CORRELATED ABILITIES 1/1 ON OBSERVED TEST C. . <U) AHERICAN COLL TESTING PROGRAH IONA CITV lA R L HCKINLEV ET AL. HAV 84 RR-84-1-0NR N8eei4-81-i('8817 F/G 12/1 NL AD-A144 869 An Investigation of the Effect of Correlated Abilities on Observed Test Characteristics Robert L McKinley and Mark D. Reckase Research Report ONR84-1 May 1984 The Ar , >rican College Testing Program Assessment Programs Area T est Development Division Iowa City, Iowa 52243 Prepared under Contract No. N00014-81-K0817 with the Personnel and Training Research Programs Psychological Sciences Division Office of Naval Research Approved for public release, distribution unlimited. Reproduction in whole or in pari is permitted for any purpose of the United States Government. 84 08 27 175 Y Cu ASSi FiC AT'OS Of "^HIS PAGE 'When Dai. REPORT DOCUMENTATION PAGE j KtAD INSTRl' 'TIONS BEKORE COMPLETING FORM 1 report number 2 GOVT ACCESSION NO 1 3 RECiPiEn'^’S CA-'A.O: number 4 TiTlE 'and Subtitle' j 5 Tvp£ or REPOR”’’ 4 PEROD COVERED An Investigation of the Effect of Correlated | Abilities on Observed Test Characteristics Technical Report 1 6 PERfORMiNG ORG REPORT NUMBER 7 author^.; B contract or GRAn*^ NUMBER'*,) Robert L. McKinley Mark D. Reckase \'00014-81-K08 17 9 PEREORMING ORCANI Z ATiON name ano address The American College Testing Program P.O. Box 168 Iowa City, lA 522A3 10 PROGRAM ELEMENT, project task AREA 5 WORK UNIT NUMBERS P.E.: 6115:5N Proj . : RRf r.A. : 042-04-01 W.U.: NRl50-499 I' CON'^ROLLING OEEICE NAME AND ADDRESS 12 report DATE Personnel and Training Research Programs May, 1984 Office of Naval Research <3 number of pages Arlington, VA 22217 48 14 monitoring agency name 6 AODRZSSrif dllfarant from Controlling Otfica) IS security CLASS, (O/ IM« r.port) unclassified IS* declassification downgrading schedule 16 Distribution statement ro/ thi$ Rapoet) Approved for public release; distribution unlimited. Reproduction in whole or part is permitted for any purpose of the United States government. n Distribution statement fof tha mbatfmct antarad in Block 20. It dlHarant Irotr Raport) 19 KEY WORDS ^Continua on ravarsa alda if nacaaamiy and Identify by block numbar) Multidimensional Item Response Theory Item Response Theory Dichotomous Data Est Lmation 20 abstract rContInua on ravaraa alda If nacaaamry mnd Idantlfy by block numbar) A study was conducted to assess the effects of correlated abilities on test characteristics, and to explore the effects of correlated abilities on the use of a multidimensional item response theory model which fioes not explicitly account for such a correlation. Two tests were constructed. One test had two relatively unidimensional subsets of items, while the other had items that were all two-dimensional. For each te‘,t response data ,, 1473 eoi'’iONOF ■ MOV 6S IS ossoLere SEClIRlTv CCASSiriC.TlON This P*Gf 'IWi.n D.l. tnl»r»dl SeCUHITV CLASSIFICATION OF THIS PACE (Whtn Oala Entmtmd) were generated according to a multidimensional two-parameter logistic model using four groups of examinees. The groups of examinees differed in the degree of inter-dimension ability correlation. To evaluate the effects of correlated abilities on observed test characteristics, the simulated response data were analyzed using item analysis and factor analysis techniques. To assess the effects of correlated abilities on the use of the multidimensional model, the parameters of the model were estimated, and the estimates were compared to the true parameters. The results of this study indicated that the presence of correlated abilities has important Implications for the characteristics of test data, and for the application of multidimensional item response theory models. It was concluded that it is necessary to consider latent item structure as well as latent ability structure in test construction and analysis. It was also concluded that use of multidimensional item response theory models that do not explicitly account for correlated abilities may result in misinterpretation of the underlying dimensions. It was suggested that research should be conducted to determine the nature of the misinterpretation and to perhaps develop an item response theory analogue to factor rotation. /\ S/N 0102- LF. 014- 6601 SECUPITV classification OF THIS PACEIlFhcn Dmim CONTENTS Page Introduction. 1 Method . 1 MIRT Model. 2 Design.2 Datasets. 3 Analyses. 5 Results . 5 Test I Analyses. ....5 Dataset 1.5 Dataset 2. 9 Dataset 3. 13 Dataset 4.... . 17 Test 2 Analyses.. 21 Dataset 5.21 Dataset 6. 25 Dataset 7.29 Dataset 8. 33 Discussion .. 37 Test 1 Analyses. 38 Item Analysis Results.... 38 Principal Component Analysis Results.38 MIRT Analysis Results. 40 Test 2 Analyses. .........42 Item Analysis Results. 42 Principal Component Analysis Results. ......42 MIRT Analysis Results. .......44 Overall Results. 45 Summary and Conclusions.............46 48 References An Investigation of the Effect of Correlated Abilities on Observed Test Characteristics Because of the required assumption of unidimensionaiity, much of the item response theory (IRT) methodology that has been developed is inappropriate for a wide range of applications. In such applications, either unidimensional sets of items cannot be constructed, or they are not desired. Until recently, in such circumstances the practitioner has been forced to abandon IRT and adopt more traditional test analysis procedures, or to inappropriately apply IRT methods and hope the procedures are robust to violations of the unidimensionality assumption. Unfortunately, such robustness has not been demonstrated. In recent years, researchers have begun grappling with the dimensionality problem. Several IRT models have been proposed for the multidimensional case, and recently some theory and procedures have been developed for applying such models (Reckase and McKinley, 1982; McKinley and Reckase, 1983a, 1983b). The work that has been done in this area indicates that it has great promise for dealing with the dimensionality problem. In multidimensional item response theory (MIRT), one of the most important questions that has not yet been addressed focuses on the effect of correlated abilities on the interpretation of model parameters. Logically, it seems desirable to construct different, homogeneous (unidimensionai) sets of items to measure each ability or trait of interest. In the case of unrelated abilities, such as math computation ability and vocabulary ability, this is a practical approach. However, if the abilities of Interest are related, such as in the case of reading comprehension and vocabulary, constructing an item set that measures only one of these two abilities is more difficult. Developing a unidimensional set of vocabulary items seems easy, but how does one construct reading comprehension items that do not also include at least a small vocabulary component? The purpose of this study is to investigate the effects of varying degrees of correlation between abilities on observed test characteristics. This research has two primary objectives. The first objective is to identify the characteristics of response data yielded in the case of correlated abilities. If unique characteristics can be identified and used to distinguish the multidimensional data from those produced in the unidimensional case, then it should be possible to identify real test situations in which a MIRT model is appropriate. The second objective of this research is to determine the effect of varying degrees of correlation between abilities on estimates of parameters from a MIRT model which does not explicitly account for such a correlation. Method MIRT methodology is relatively new and probably unfamiliar to many. Therefore, before continuing with a discussion of this research, a brief discussion of the MIRT model selected for this study will be presented. For a more detailed discussion of this model, see McKinley and Reckase, 1983a. -f.' The MIRT Model The MIRT model selected for this study is an extension of the two- parameter logistic (2PL) model proposed by Birnbaum (1968). The multidimensional 2PL model, or M2PL model, is given by exp(d. + a.' 0.) _ i —1—1 1 + exp(d^ + where is a vector of ability parameters for examinee j, is a vector of discrimination parameters for Item 1, Is a scalar Item parameter related to item difficulty, and P£(_0_j) is the probability of a correct response to item 1 by an examinee having ability . The discrimination and ability vectors in Equation 1 are both of order m, where m is the number of dimensions comprising the complete latent space. The a^' ^ term in Equation 1 can be written as ^ ®jk * where is the item discrimination parameter for dimension k and 0^^^ is the examinee ability parameter for dimension k. In the unidimensional case Equation 1 simplifies to the 2PL model (without the D*1.7 term usually used in the 2PL model) with where bj^ is the difficulty parameter for item 1 from the 2PL model. Design The basic design of this study involved the simulation and analysis of response data generated for examinees having varying levels of correlation between their abilities on different dimensions. The generated data were then analyzed using traditional test analysis techniques to determine the effects of the correlated abilities on the test characteristics. Afterward, the data were analyzed using the M2PL model to determine the effects of correlated abilities on the estimates of the parameters of the model. Datasets Simulated test data were generated for two different types of test. The first test type measured two dimensions, with half of the items on the test measuring predominantly one dimension, and other half measuring predominantly the second dimension. The second test type also measured two dimensions, but for this test each item measured both dimensions. Four datasets were generated for each test using Interdlmenslonal ability correlations of .7, .5 .35, and 0. Table 1 summarizes the eight datasets that were created. Table 1 Dataset Simulated Datasets Test - V-V.' .- 1 1 0.70 2 1 0.50 3 1 0.35 4 1 0.00 5 2 0.70 6 2 0.50 7 2 0.35 8 2 0.00 Table 2 shows the true item parameters used to simulate the two tests. Each test had 50 items, and the same set of d-parameters was used for both tests. As can be seen, for test 1 the items generally have high discriminations on one dimension or the other, but not both. For test 2 the items tend to have high discriminations on both dimensions. Table 2 V. True Item Parameters Used To Simulate Both Tests V . . .v '-’.V * _ • H i Table 2(Continued) True Item Parameters Used To Simulate Both Tests Item d Test 1 Test 2 ®1 ®2 ^1 32 11 -0.51 1.38 0.30 0.98 1.02 12 0.27 1.39 0.27 0.93 1.07 13 -0.14 1.21 0.73 1.20 0.75 14 -1.71 1.30 0.55 0.68 1.24 15 0.41 1.39 0.27 0.68 1.24 16 1.15 1.37 0.37 1.02 0.98 17 -0.52 1.37 0.35 1.02 0.98 18 0.40 1.39 0.28 0.83 1.15 19 -0.28 1.41 0.11 1.25 0.67 20 0.59 1.34 0.46 1.06 0.94 21 -2.77 1.35 0.41 0.61 1.27 22 -0.17 1.36 0.38 0.90 1.09 23 -0.58 1.35 0.44 0.74 1.21 24 -0.11 1.39 0.26 0.72 1.22 25 -0.61 1.38 0.30 0.94 1.06 26 1.20 0.33 1.38 0.57 1.30 27 0.14 0.35 1.37 0.73 1.21 28 1.40 0.07 1.41 0.99 1.01 29 0.34 0.21 1.40 0.75 1.20 30 -1.02 0.32 1.38 0.86 1.12 31 1.05 0.34 1.37 0.83 1.14 32 0.31 0.55 1.30 0.74 1.21 33 -0.83 0.45 1.34 0.61 1.27 34 0.55 0.33 1.37 0.99 1.01 35 1.46 0.42 1.35 0.79 1.17 36 1.05 0.33 1.38 1.07 0.92 37 -1.05 0.37 1.37 1.11 0.88 38 0.34 0.54 1.31 0.80 1.17 39 0.20 0.37 1.37 0.68 1.24 40 -0.45 0.40 1.36 1.17 0.79 41 1.67 0.34 1.37 0.72 1.22 42 -0.20 0.27 1.39 0.93 1.07 43 -0.69 0.30 1.38 1.17 0.79 44 0.35 0.28 1.39 0.93 1.07 45 0.93 0.47 1.34 1.15 0.82 46 0.86 0.67 1.25 1.10 0.89 47 -1.70 0.41 1.35 1.14 0.83 48 -1.54 0.40 1.36 1.04 0.95 49 -0.57 0.33 1.38 1.06 0.94 50 -0.78 0.28 1.39 0.95 1.05 Mean -0.06 0.86 0.86 0.89 1.07 S.D. 1.04 0.51 0.51 0.18 0.16 Each group of examinees consisted of 2000 simulated subjects with true abilities selected at random from a bivariate normal distribution having both means equal to 0.0, both standard deviations equal to (;.50, and having the appropriate correlation. Each group was used for only one of the two tests. Thus, there were eight sets of item responses generated. Each set of responses was stored in the appropriate dataset, depending on the test and the correlation between abilities. The same set of analyses were then run on each dataset. Analyses There were four types of analyses run on each dataset in Table 1. The first type was an item analysis. This consisted of computing item proportion- correct difficulty indices, item point blserial discrimination indices, and a KR-20 test reliability coefficient. The second type of analysis performed on each dataset was a principal components analysis. More specifically, a principal components analysis of tetrachorlc correlations was performed on each dataset. Using the results of the principal components analyses, both a varimax rotated and an oblique rotated factor solution were obtained for each dataset. In each case two factors were rotated. The third type of analysis performed was the application of the M2PL model to the data. This consisted of estimating item and person parameters for the M2PL model for each dataset. Parameter estimation for the M2PL model was performed using the MAXLOG program (McKinley and Reckase, 1983c). The final type of analysis performed on these data consisted of correlational analyses. Correlations were computed among true and estimated item parameters, item statistics (traditional difficulty and discrimination), and factor loadings. Correlations between true and estimated ability parameters were also computed. Results As was discussed above, there were four types of analyses performed on the eight datasets created for this study - item analyses, principal component analyses, MIRT analyses, and correlation analyses. The results obtained from all four of these sets of analyses will be presented for each dataset separately, beginning with the four datasets based on the first test. Remember that the first test contained two subsets of items, each of which was relatively unidimensional. After presenting these results, the results for the second test will be presented. The second test had items that each measured two dimensions. Test 1 Analyses Dataset 1 . Table 3 shows the results of the item analysis, principal components analysis, and MIRT analysis of the first dataset. This dataset was created using test 1 and a group of examinees having an inter-dimension ability correlation of 0.70. The columns headed 'Item Parameter Estimates' are the results obtained from the MIRT analysis, and are estimates of the item parameters of the M2PL model. The columns headed 'Item Statistics' are the 6 ( proportion-correct item difficulties (p) and item point biserial discrimination indexes (pbis) obtained from the item analysis of the first dataset. Columns 6 and 7 are the varimax rotated factor loadings for the first two factors of the principal components analysis of tetrachoric correlations. The last two columns are oblique rotated factor loadings for the first two factors from the principal components analysis. Table 3 Item Parameter Estimates, Item Statistics, and Factor Loadings for Dataset 1 Item Parameter Item Factor Loadings Estimates Statistics Orthogonal Oblique aj .43 0.88 .69 0.73 .72 0.89 .80 0.44 .46 0.91 .64 0.59 .72 0.77 .51 0.74 .38 1.02 .43 0.97 .41 1.11 .45 0.68 .59 1.00 .66 0.78 .46 0.73 .46 0.91 .45 1.03 .45 1.02 .06 1.33 .60 0.81 .37 0.73 .51 0.89 .51 0.95 .60 0.67 .0 2.00 .01 0.48 .88 0.4 6 .78 0.2 8 .92 0.3 4 .95 0.4 0 .62 0.2 6 .70 0.4 4 .70 .34 0.5 0.2 0 9 f / \ Item Parameter Estimates Statistics Factor Loadings Orthogonal Oblique The mean score on test 1 for this group of examinees was 24,14, and the standard deviation was 8.13. The KR-20 reliability for these data was 0.86. The correlation between the factors, obtained from the oblique solution, was 0.64. Table 4 shows the intercorrelation matrix for the true and estimated item parameters for the first dataset. As can be seen, the correlations of the true and estimated item parameters were 0.996 for the d-parameter, 0.731 for the true a on the first dimension and the estimated a for the second dimension, and 0.768 for the true a for the second dimension and the estimated a for the first dimension. Thus, the d-parameter was very well estimated, and the a-parameters were only moderately well estimated. " » * K * • ' ■*. % , Variable True Estimated Table 5 shows the Intercorrelatlon matrix of the true and estimated ability parameters obtained for the first dataset. As can be seen, the ability on dimension 1 had a correlation of 0.670 with the ability estimate on the second dimension, while there was a correlation of 0.704 between the true ability for dimension 2 and the ability estimate on dimension 1. Despite the correlation of 0.685 obtained for the true abilities, the estimated abilities were not correlated (r=-0.140). Thus, while the abilities for this group were moderately well estimated, the correlation between the dimensions was not recovered by the estimation process. Table 5 Intercorrelatlon Matrix for True and Estimated Ability Parameters for Dataset 1 1.000 0.685 1.000 0.444 0.704 1.000 0.67 0.39 -0.14 1.00 The correlation of the proportion-correct difficulty Index and the d- parameter was 0.995 (for true and estimated d-values), which is about what was expected. The point blserlal discrimination index had a correlation of -0.131 with the true a-parameter for dimension 1 and 0.166 for the second dimension. Using the a-value estimates the correlation was 0.258 for dimension 1 and 0.059 for dimension 2. This, too, was much as was expected. Since the point biserial is strongly affected by the dimensionality of the Items, It should not have a strong relationship to discrimination on a single dimension for two-dimensional data. Table 6 shows the Intercorrelatlon matrix for the true and estimated item parameters and the varlmax and oblique rotated factor loadings for the first dataset. As can be seen, there was a strong relationship between the factor loadings (both varlmax and oblique rotated) and the Item parameters (both true and estimated). The first four eigenvalues obtained from the principal components analysis of these data were 10.01, 1.50, 1.31, and 1.27. There appeared to be a strong first factor and a much smaller second factor. This Is consistent with the high Inter-dlmenslon ability correlation for these data. Table 6 Intercorrelatlon Matrix for True and Estimated Item Parameters and Factor Loadings for Dataset 1 Variable Item Parameters Estimated Factor Loadings Orthogonal Oblique True d 1.000 -0.172 0. .159 0. .996 0. .317 -0. .272 0. .330 -0. ,308 0.329 -0.319 1.000 -0, .987 -0. .189 -0. .751 0, .731 -0, .852 0. .860 -0.868 0.872 ^2 1, .000 0. .177 0. .768 -0, .730 0, .862 -0, .842 0.871 -0.862 Estimated d^ 1, .000 0, .358 -0, .323 0, .362 -0, .341 0.362 -0.352 1, .000 -0, .841 0, .953 -0, .852 0.942 -0.894 ^2 1, .000 -0, .852 0. .90S -0.880 0.908 Orthogonal I 1, .000 -0. ,918 0.995 -0.956 II 1, .000 -0.954 0.994 Oblique I 1.000 -0.981 The correlation of the point blserlal Index and the factor loadings was 0.269 and 0.091 for the two varlmax rotated factors, and 0.180 and -0.009 for the two oblique rotated factors. The proportion-correct difficulty Index had correlations of 0.329 and -0.323 with the varlmax rotated loadings, and 0.332 and -0.330 with the oblique rotated loadings. The proportion-correct and point blserlal indexes had a correlation of 0.086. Dataset 2 . Table 7 shows the Item parameter estimates. Item statistics, and factor loadings obtained for dataset 2. These data were generated using test 1 and a group of examinees with an inter-dimension ability correlation of 0.50. The mean score on test 1 for this group was 24.49, and the standard deviation was 7.73. The KR-20 reliability was 0.84, which Is slightly lower than the KR-20 for dataset 1. The correlation between the factors, obtained from the oblique rotation, was -0.59, which is slightly lower than for dataset 1, and opposite In sign. M ■•'■■I* -■■-■. •r.^: 10 Table 7 Item Parameter Estitr tea, Item Statistics, and Factor Loadings for Dataset 2 1% * *• fc.w: i ■ •. i'"' «^. w. "•' J fe r.: Item Item Parameter Estimates Item Statistics Factor Loadings Orthogonal Oblique d «1 32 P pbls I II I II 1 -1.41 0.55 0.91 0.23 0.28 0.23 0.41 0.11 -0.40 2 2.23 0.62 0.80 0.87 0.24 0.27 0.36 0.18 -0.32 3 -0.05 0.29 1.07 0.49 0.29 0.10 0.50 -0.07 -0.55 4 2.15 0.13 1.19 0.85 0.22 0.06 0.51 -0.12 -0.57 5 -0.81 0.37 1.06 0.34 0.29 0.14 0.48 -0.02 -0.50 6 -1.31 0.33 0.83 0.24 0.24 0.12 0.41 -0.01 -0.43 7 0.49 0.61 0.75 0.60 0.31 0.30 0.35 0.22 -0.30 8 -0.47 0.42 0.96 0.40 0.30 0.19 0.43 0.06 -0.43 9 -0.52 0.46 0.69 0.39 0.27 0.19 0.37 0.08 -0.37 10 -1.69 0.44 1.16 0.21 0.28 0.16 0.48 0.01 -0.50 11 -0.57 0.32 0.96 0.38 0.28 0.12 0.47 -0.04 -0.50 12 0.30 0.41 1.03 0.56 0.31 0.18 0.46 0.04 -0.47 13 -0.12 0.72 0.70 0.47 0.33 0.33 0.34 0.26 -0.27 14 -1.89 0.65 1.02 0.18 0.29 0.26 0.42 0.14 -0.40 15 0.41 0.53 0.90 0.59 0.32 0.23 0.43 0.10 -0.42 16 1.22 0.49 0.85 0.74 0.28 0.22 0.40 0.11 -0.39 17 -0.56 0.43 0.84 0.38 0.29 0.20 0.39 0.09 -0.38 18 0.35 0.49 0.83 0.57 0.31 0.21 0.41 0.10 -0.40 19 -0.40 0.0 1.86 0.44 0.27 0.02 0.54 -0.19 -0.63 20 0.68 0.66 0.95 0.64 0.36 0.29 0.44 0.17 -0.41 21 -2.74 0.31 0.94 0.08 0.17 0.10 0.41 -0.04 -0.45 22 -0.19 0.42 0.95 0.46 0.31 0.19 0.43 0.06 -0.44 23 -0.63 0.56 0.80 0.37 0.31 0.22 0.42 0.10 -0.41 24 -0.20 0.39 0.92 0.46 0.30 0.18 0.43 0.06 -0.43 25 -0.72 0.40 0.81 0.35 0.27 0.18 0.40 0.06 -0.40 26 1.29 0.80 0.27 0.76 0.24 0.37 0.16 0.38 -0.04 27 0.18 0.80 0.55 0.54 0.32 0.38 0.27 0.35 -0.17 28 1.47 0.71 0.28 0.79 0.21 0.37 0.10 0.40 29 0.40 0.93 0.23 0.58 0.27 0.43 0.12 0.46 30 -1.02 0.84 0.45 0.29 0.28 0.40 0.20 0.39 -0.08 31 1.31 1.32 0.25 0.73 0.30 0.53 0.11 0.59 0.08 32 0.28 1.00 0.50 0.56 0.33 0.46 0.21 0.46 -0.07 33 -0.79 0.88 0.29 0.34 0.26 0.43 0.12 0.46 34 0.63 1.11 0.28 0.63 0.30 0.49 0.12 0.53 0.05 35 1.46 0.77 0.42 0.78 0.25 0.38 0.19 0.38 -0.08 36 1.05 0.79 0.29 0.72 0.24 0.40 0.12 0.43 0.01 37 -1.21 1.23 0.18 0.28 0.27 0.50 0.08 0.56 0.10 38 0.43 1.14 0.23 0.59 0.29 0.50 0.10 0.55 0.07 39 0.19 0.93 0.42 0.54 0.32 0.42 0.22 0.42 -0.09 40 -0.53 0.92 0.30 0.39 0.28 0.44 0.14 0.46 0.01 41 1.69 0.73 0.40 0.82 0.23 0.36 0.18 0.35 -0.08 42 -0.22 0.78 0.36 0.45 0.28 0.39 0.18 0.39 -0.06 ■ . • . I •■'.v •- ;>vtJ •M Table 7(Contlnued) Item Parameter Estimates, Item Statistics, and Factor Loadings for Dataset 2 item Parameter Item Estimates Item Statistics Factor Loadings Orthogonal Oblique d ^1 32 P phis I II I II 43 -0.74 0.79 0.58 0.35 0.31 0.36 0.28 0.32 -0.19 44 0.34 0.72 0.27 0.58 0.25 0.36 0.15 0.36 -0.04 45 0.93 1.16 0.35 0.68 0.32 0.49 0.16 0.52 -0.00 46 0.88 0.90 0.51 0.68 0.31 0.42 0.23 0.41 -0.11 47 -1.74 0.90 0.46 0.18 0.25 0.38 0.21 0.37 -0.10 48 -1.76 1.28 0.27 0.20 0.27 0.48 0.14 0.52 0.03 49 -0.55 0.69 0.39 0.38 0.26 0.35 0.19 0.34 -0.09 50 -0.77 0.77 0.40 0.34 0.27 0.39 0.18 0.39 -0.07 Table 8 shows the Intercorrelation matrix for the true and estimated item parameters for dataset 2. The true and estimated d-parameters had a correlation of 0.998, indicating that the d-parameter was once again very well estimated. The correlation between the true and estimated a-values was 0.866 for the dimension 1 true value and the dimension 2 estimated value, and 0.834 for the dimension 2 true value and dimension 1 estimated value. The a-values, then, were better estimated for dataset 2 than for dataset 1. Table 8 Intercorrelation Matrix for True and Estimated Item Parameters for Dataset 2 True Estimated Variable d 32 d ®1 32 True d 1.000 -0.172 0.159 0.998 0.112 -0.182 ®1 32 d 1.000 -0.987 -0.180 -0.784 0.866 1.000 0.167 0.834 -0.886 Estimated 1.000 0.115 -0.189 ®1 32 1.000 -0.865 1.000 Table 9 shows the Intercorrelation matrix of the true and estimated ability parameters obtained for the second dataset. The true ability on dimension 1 had a correlation of 0.716 with the estimated ability on dimension 2, while there was a correlation of 0.743 for the true ability for dimension 2 and the dimension 1 estimated ability. The inter-dimension ability correlation was 0.494 for the true values, and -0.150 for the estimated values. These ability parameters were estimated slightly better than were the parameters for dataset 1, but once again the Inter-dlmenslon ability correlation was not recovered during the estimation process. Table 9 Intercorrelation Matrix for True and Estimated Ability Parameters for Dataset 2 True Estimated Variable ®1 ®2 ®1 ®2 True 1.000 0.494 0.321 0.716 1.000 0.743 -0.279 Estimated ®2 1.000 -0.150 1.000 The correlation of the proportion-correct difficulty index and the d- parameter was 0.993 (0.995 for the estimated d-parameter). The point biserlal index had a correlation with the true a-parameters of 0.187 for the first dimension and -0.134 for dimension 2. Those correlations were 0.166 and 0.123 when computed with the estimated a-values. Table 10 shows the intercorrelation matrix for the true and estimated item parameters and the factor loadings for dataset 2. Again, the relationship between the true and estimated item parameters and the varlmax and oblique rotated factor loadings were quite strong. The first four eigenvalues from the principal components analysis were 9.09, 1.79, 1.30, and 1.28. The first factor for these data was slightly smaller than for the first dataset, and the second factor slightly larger. This is consistent with the lower inter-dlmenslon ability correlation for this group of examinees. Table 10 Intercorrelation Matrix for True and Estimated Item Parameters and Factor Loadings for Dataset 2 Variable Item Parameters Factor ! Loadings True Estimated Orthogonal Oblique d ^1 ^2 d ®1 ®2 I II I II True d 1 .000 -0.172 0.159 0.998 0.112 -0.182 0.201 -0.191 0.202 0.196 1.000 -0.987 -0.180 -0.784 0.866 -0.859 0.936 -0.890 -0.930 ao 1.000 0.167 0.834 -0.886 0.899 -0.943 0.923 0.945 Estimated d^ 1.000 0.115 -0.189 0.204 -0.197 0.205 0.201 Si 1.000 -0.865 0.976 -0.876 0.963 0.911 aj 1.000 -0.907 0.954 -0.931 -0.955 Orthogonal I 1.000 -0.930 0.996 0.959 11 1.000 -0.961 -0.996 Oblique I 1.000 0.981 11 1.000 The correlation of the point blserial index and the factor loadings was 0.138 and 0.185 for the varlmax rotated loadings and 0.057 and -0.109 for the oblique rotated loadings. The proportion-correct difficulty index had correlations of 0.212 and -0.208 with the varimax rotated loadings and 0.214 and 0.212 with the oblique rotated loadings. The proportion-correct difficulty index and the point biserlal index had a correlation of 0.008. Dataset 3 . Table 11 shows the item parameter estimates, item statistics, and factor loadings for dataset 3. These data were generated using test 1 and a group of examinees with an inter-dimension ability correlation of 0.35. The mean score on test 1 for this group was 24.66 and the standard deviation was 7.25. The KR-20 reliability was 0.82. This was slightly lower than for dataset 2. The correlation between the factors, obtained from the oblique rotation, was 0.52, which is slightly lower than for dataset 2, and opposite in sign. P} 'S' .IV ;• ' • • • • ■'V ■* • Item Parameter Estimates illll Statistics Factor Loadings Orthogonal Oblique p pbis .23 0.22 .90 0.16 .51 0.27 .85 0.23 .33 0.24 .24 0.26 .63 0.29 .42 0.24 .40 0.25 .18 0.24 .39 0.27 .55 0.29 .47 0.31 .19 0.22 .59 0.25 ,75 0.25 .38 0.26 .58 0.23 .44 0.24 .63 0.27 ,07 0.17 .46 0.27 .39 0.28 .48 0.25 .39 0.29 .75 0.23 .54 0.26 .78 0.20 .56 0.28 .29 0.26 .71 0.25 .57 0.31 .33 0.30 .62 0.28 .79 0.21 34 0 37 -0 37 0 49 -0 40 0 Table 11(Continued) Item Parameter Estimates, Item Statistics, and Factor Loadings for Dataset 3 Item Item Parameter Estimates Item Statistics Factor Loadings Orthogonal Oblique d ^1 ^2 P phis I II I II 36 1.05 0.80 0.34 0.71 0.55 0.39 0.16 0.39 0.06 37 -1.08 0.88 0.34 0.28 0.26 0.41 0.17 0.40 0.07 38 0.39 0.96 0.34 0.58 0.30 0.45 0.17 0.45 -0.00 39 0.16 0.90 0.24 0.53 0.27 0.43 0.12 0.45 -0.00 40 -0.46 0.89 0.26 0.40 0.27 0.44 0.12 0.45 0.00 41 1.83 1.08 0.14 0.82 0.22 0.47 0.06 0.51 -0.08 42 -0.19 0.97 0.18 0.46 0.26 0.45 0.09 0.48 -0.03 43 -0.68 0.73 0.33 0.35 0.25 0.38 0.15 0.38 0.05 44 0.37 0.75 0.30 0.58 0.26 0.37 0.17 0.37 0.07 45 1.04 0.80 0.32 0.71 0.25 0.40 0.15 0.41 0.05 46 0.92 0.71 0.54 0.69 0.28 0.34 0.26 0.30 0.19 47 -1.68 0.86 0.39 0.19 0.24 0.39 0.18 0.38 0.08 48 -1.59 0.80 0.41 0.20 0.24 0.38 0.19 0.37 0.10 49 -0.58 0.98 0.39 0.38 0.30 0.45 0.18 0.45 0.06 -0.74 0.74 0.54 0.34 0.29 0.37 0.26 0.33 0.18 Table 12 shows the Intercorrelation matrix for the true and estimated item parameters for dataset 3. The true and estimated d-parameters had a correlation of 0.999, indicating that the d-parameter was very well estimated. The dimension 1 true a-values had a correlation of 0.921 with the dimension 2 estimated a-values, while there was a correlation of 0.937 between the dimension 2 true a-values and the dimension 1 estimated a-values. The a- values, then, were fairly well estimated for these data. Variable Table 13 shows the intercorrelation matrix of the true and estimated ability parameters obtained for dataset 3. The true abilities for dimension 1 had a correlation of 0.772 with dimension 2 of the estimated abilities. The dimension 2 true abilities had a correlation of 0.779 with the dimension 1 estimated abilities. The inter-dimension ability correlation was 0.345 for the true abilities and -0.087 for the estimated abilities. Table 13 Intercorrelation Matrix for True and Estimated Ability Parameters for Dataset 3 Variable True Estimated True 9 0 Estimated 6 0 1 1 1.000 0.345 1.000 0.231 0.779 1.000 0.772 0.229 -0.087 1.000 The correlation between the d-pararaeter and the proportion-correct difficulty index was 0.995 (0.993 for the d-parameter estimates). The correlation of the point blserlal discrimination index with the true a- parameters was -0.134 for dimension 1 and 0.171 for dimension 2. When estimated a-values were used, these correlations were 0.261 and -0.079. Table 14 shows the Intercorrelation matrix for the true and estimated item parameters and the rotated factor loadings for dataset 3. As was the case previously, there was a strong relationship between the true and estimated item parameters and both sets of rotated loadings. The first four eigenvalues from the principal components analysis of these data were 7.92, 17 2,08, 1,43, and 1,29, The first factor for these data was smaller than for the previous datasets, and the second factor was somewhat larger. Table 14 Intercorrelation Matrix for True and Estimated Item Parameters and Factor Loadings for Dataset 3 Item Parameter Factor Loadings Variable True Estimated Orthogonal Oblique d ®1 ^2 d ®1 ^2 I II I II True d 1 .000 -0.172 0.159 0.999 0.131 -0.207 0.149 -0.215 0.164 -0.204 ^1 1.000 -0.987 -0.164 -0.929 0.921 -0.946 0.949 -0.953 0.955 ®2 1.000 0.153 0.937 -0.915 0.959 -0.942 0.961 -0.952 Estimated d" 1.000 0.127 -0.201 0.145 -0.200 0.160 -0.199 1.000 -0.936 0.988 -0.949 0.987 -0.965 a-, 1.000 -0.934 0.989 -0.957 0.984 Orthogonal I 1.000 -0.954 0.997 -0.972 11 1.000 -0.972 0.997 Oblique I 1.000 -0.986 II 1.000 The correlation of the point blserlal index and the factor loadings was 0,299 and -0.050 for the varimax rotation, and 0.247 and -0,106 for the oblique rotation. The proportion-correct difficulty Index had correlations of 0.147 and -0.214 with the varimax rotated loadings, and 0.162 and -0.203 with the oblique rotated loadings. The proportion-correct difficulty and point blserlal discrimination Indexes had a correlation of -0.101, Dataset 4 . Table 15 shows the Item parameter estimates, Item statistics, and factor loadings for dataset 4. These data were generated using test 1 and a group of examinees with an Inter-dlmension ability correlation of 0.00. The mean score on test 1 for this group was 24,61 and the standard deviation was 6.52. The KR-20 reliability was 0.77, which Is somewhat lower than for dataset 3. The correlation between the factors, obtained from the oblique rotation, was 0.36, which Is slightly lower than was the case for dataset 3. Item Parameter Item Factor Loadings Estimates Statistics Orthogonal Oblique p pbls .23 0.20 .90 0.13 .51 0.23 .85 0.18 .33 0.21 .24 0.23 .63 0.24 .41 0.20 .39 0.20 .17 0.21 .39 0.24 .55 0.24 .47 0.26 .18 0.18 .59 0.20 .75 0.21 .38 0.22 .59 0.19 .44 0.21 .63 0.23 .07 0.15 .46 0.24 .39 0.23 .48 0.21 .39 0.23 .75 0.18 .54 0.22 .78 0.17 .56 0.25 .29 0.23 .72 0.21 .57 0.26 .32 0.28 .62 0.23 .80 0.18 15 0.33 0.10 04 0.35 -0.03 11 0.39 0.04 03 0.47 -0.06 08 0.40 O.Oi 12 0.43 0.04 09 0.46 0.01 04 0.41 -0.04 02 0.45 -0.07 08 0.48 -0.01 07 0.46 -0.02 05 0.48 -0.04 25 0.32 0.20 13 0.33 0.07 06 0.40 -0.01 12 0.39 0.04 10 0.40 0.02 03 0.41 -0.05 01 0.48 -0.10 12 0.40 0.04 08 0.40 0.00 17 0.36 0.11 12 0.40 0.05 09 0.37 0.02 12 0.40 0.04 43 -0.00 0.45 39 0.09 0.39 42 -0.00 0.44 48 0.06 0.49 47 0.05 0.49 38 0.09 0.38 39 0.17 0.38 46 0.16 0.45 44 0.06 0.46 94 69 1.000 0.007 0.0 1.000 0.8 l.Oi The proportion-correct difficulty index had a correlation of 0.995 with the true d-parameter, and a correlation of 0.993 with the d-parameter estimates. The correlation between the point biserial index and the true a- parameters was -0.1A6 for dimension 1 and 0.183 for dimension 2. These values were 0.274 and -0.097 for the estimated a values. Table 18 shows the Intercorrelation matrix for the true and estimated item parameters and the rotated factor loadings for dataset 4. As has been the case all along, there was a strong relationship between the item parameters and estimates and both sets of rotated factor loadings. The first four eigenvalues from the principal components analysis for dataset 4 were 6.32, 2.74, 1.42, and 1.32. Table 18 Intercorrelation Matrix for True and Estimated Item Parameters and Factor Loadings for Dataset 4 Item Parameters Factor Loadings Variable True Estimated Orthogonal Oblique d ®1 ®2 d ®1 &2 I II I II True d 1 .000 -0.172 0.159 0.999 0.133 -0.205 0.\hl -0.217 0.163 -0.203 ^1 1.000 -0.987 -0.165 -0.929 0.920 -0.945 0.949 ■ -0.953 0.976 ®2 1.000 0.153 0.937 -0.914 0.958 -0.942 0.962 -0.953 Estimated d 1.000 0.129 -0.199 0.144 -0.211 0.159 -0.198 1.000 -0.937 0.988 -0.949 0.987 -0.965 ^2 1.000 -0.939 0.989 ■ -0.957 0.985 Orthogonal r 1.000 -0.955 0.998 -0.973 II 1.000 ■ -0.972 0.998 Oblique I 1.000 -0.986 II 1.000 The point biserial index had correlations of 0,312 and -0.066 with the varlmax rotated factor loadings. The correlations between the point blserlals and the oblique rotated factor loadings were 0.262 and -0.121. The correlations between the proportion-correct index and the varlmax rotated factor loadings were 0.146 and -0.217, while correlations of 0.163 and -0.203 were obtained between the point biserials and the oblique rotated factor loadings. The correlation between the proportion-correct difficulty index and the point blserlal discrimination index was -0.110, Test 2 Analyses Dataset 5 . Table 19 shows the item parameter estimates, item statistics, and rotated factor loadings for dataset 5. These data were generated using test 2 and a group of examinees having an inter-dimension ability correlation of 0.70. The mean score on test 2 for these examinees was 24.19, while the standard deviation was 9.00. The KR-20 reliability was 0.89. The correlation between factors, obtained from the oblique rotation, was 0.62. 22 Table 19 Item Parameter Estimates, Item Statistics, and Factor Loadings for Dataset 5 Item Parameter Estimates Factor Loadings Statistics Orthogonal Oblique Item Parameter Item Factor Loadings Item Estimates Statistics Orthogonal Oblique d ^1 ®2 P pbls I II I II 39 0.19 0.89 0.57 0.54 0.35 0.45 0.22 0.50 0.01 40 -0.49 1.03 0.60 0.40 0.38 0.49 0.23 0.55 -0.01 41 1.70 1.16 0.48 0.79 0.32 0.50 0.19 0.57 -0.06 42 -0.36 0.07 0.45 0.39 0.57 0.15 0.67 -0.14 43 -0.91 0.94 0.74 0.32 0.37 0.47 0.28 0.50 0.06 44 0.32 0.99 0.56 0.57 0.37 0.44 0.26 0.48 0.05 45 0.91 0.98 0.54 0.68 0.35 0.49 0.20 0.55 -0.04 46 0.94 0.51 1.16 0.68 0.33 0.20 0.50 0.11 0.46 47 -1.86 0.77 0.85 0.17 0.31 0.29 0.42 0.24 0.32 -1.60 0.93 0.49 0.20 0.29 0.44 0.19 0.49 -0.03 49 0.77 0.84 0.38 0.37 0.34 0.39 0.31 0.26 50 -0.84 0.76 0.99 0.34 0.37 0.32 0.43 0.28 0.31 Table 20 shows the intercorrelation matrix obtained for the true and estimated Item parameters for dataset 5 . The correlation between the true and estimated d-parameters was 0.997 The correlations of the true and estimated a-parameters for these data were 0.198 for dimension 1 and 0.167 for dimension 2. These values are In marked contrast to the high values obtained for test 1. Here, there is no significant correlation between the true and estimated Item discrimination parameters. Table 20 Intercorrelation Matrix for True and Estimated Item Parameters for Dataset 5 True Variable _ d ®1 32 d ®1 32 True d 1.000 -0.048 0.076 0.997 0.029 0.088 ®1 1.000 -0.985 -0.036 0.198 -0.176 ®2 1.000 0.067 -0.159 0.167 Estimated d 1.000 0.012 0.096 ®l 1.000 -0.837 ®2 1.000 Table 21 shows the intercorrelation matrix for the true and estimated ability parameters for dataset 5. As can be seen, the true ability values on the two dimensions had about equal correlations with the dimension 1 estimated abilities (0.592 for dimension 1, 0.597 for dimension 2). The correlations of the true abilities with the dimension 2 ability estimates were also almost equal (0.468 for dimension 1, 0.507 for dimension 2) and both were lower than the correlations with the dimension 1 estimates. The inter-dimension ability correlation was 0.687 for the true values and -0.203 for the estimated values. Table 21 Intercorrelation Matrix for True and Estimated Ability Parameters for Dataset 5 1.000 0.687 1.000 Estimated 6, 0.592 0.597 1.000 The correlation between the proportion-correct index and the d-parameter was 0.994 (0.992 with the d-parameter estimates). The correlation between the point blserlal index and the true a-parameters was 0.186 for dimension 1 and -0.142 for dimension 2. Wlien the a-parameter estimates were used, these correlations were 0.474 and -0.252. Table 22 shows the intercorrelation matrix for the true and estimated item parameters and the rotated factor loadings for dataset 5. Interestingly, the factor loadings (both rotations) are strongly related to the estimated a- parameters, but not to the true a-parameters. The first four eigenvalues from the principal components analysis of dataset 5 were 12.32, 1.23, 1.21, and 1.18. The first factor for this dataset is larger than for the corresponding dataset from the test 1 analyses (dataset 1). Variable Item Parameters Estimated Factor Loadings Orthogonal Oblique I II I II True d 1.000 -0.048 0. .076 0. .997 0. .029 0. .088 0. .113 -0.061 0.105 -0.079 ®1 1.000 -0, .985 -0, .036 0. .198 -0. .176 0. .244 -0.195 0.238 -0.215 ®2 1, .000 0. .067 -0. .159 0. .167 -0. .215 0.181 -0.211 0.196 Estimated 4 1, .000 0, .012 0, .096 0, .116 -0.067 0.108 -0.085 ^1 1. .000 -0, .837 0, .860 -0.760 0.851 -0.806 ®2 1, .000 -0, .869 0.905 -0.885 0.908 Orthogonal I 1. .000 -0.925 0.998 -0.966 II 1.000 -0.949 0.992 Oblique I 1.000 -0.982 The point blserlal index had correlations of 0.509 (dimension 1) and -0.186 (dimension 2) with the varlmax rotated factor loadings» and 0.456 (dimension 1) and -0.298 (dimension 2) with the oblique rotated factor loadings. The proportion-correct index had correlations of 0.085 and -0.042 with the varlmax rotated loadings, and 0.078 and -0.057 with the oblique rotated factor loadings. The correlation between the proportion-correct and point biserial indexes was 0.120. Dataset 6 . Table 23 shows the item parameter estimates, item statistics, and rotated factor loadings for dataset 6. These data were generated using test 2 and a group of examinees with an inter-dimension ability correlation of 0.50. The mean score on test 2 for this group was 24.01, and the standard deviation was 8.54. The KR-20 reliability was 0.87. The correlation between factors, obtained from the oblique rotation, was -0.19, Table 23 Item Parameter Estimates, Item Statistics, and Factor Loadings for Dataset 6 Item Parameter Item Factor Loadings Estimates Statistics Orthogonal Oblique 1 -1.39 0 2 2.22 0 3 -0.12 0 4 2.78 0 5 -0.86 0 6 -1.36 0 7 0.44 0 8 -0.44 0 9 -0.49 0 10 -1.76 0 11 -0.58 0 12 0.29 0 13 -0.21 0 14 -2.74 2 15 0.40 0 16 1.09 0 17 -0.58 1 18 0.68 0 19 -0.38 0 20 0.65 0 21 -3.04 1 22 -0.24 0 23 -0.69 0 24 -0.21 1 25 -0.73 0 26 1.19 0 27 0.05 1 28 1.54 1 29 0.33 0 30 -1.02 0 31 0.92 0 32 0.28 0 33 -1.02 0 34 0.49 1 35 1.45 0 36 1.06 1 37 -1.21 1 38 0.29 0 39 0.13 0 Dll] p pbls .23 0.26 .87 0.26 .48 0.34 .84 0.27 .33 0.31 .24 0.29 .59 0.36 .40 0.30 .40 0.32 .18 0.26 .38 0.36 .56 0.38 .46 0.33 .17 0.28 .59 0.33 .71 0.32 .38 0.36 .58 0.35 .42 0.33 .64 0.32 .07 0.21 .45 0.36 .36 0.33 .46 0.36 .35 0.31 .73 0.31 .51 0.35 .78 0.29 .57 0.36 .30 0.33 .69 0.30 .56 0.34 .29 0.30 .60 0.35 .78 0.29 .70 0.32 .28 0.34 .56 0.34 .53 0.36 -0.05 -0.13 -0.18 -0.42 0.10 0.07 11 04 15 04 06 16 01 42 12 22 05 45 01 00 02 02 02 17 05 02 17 10 12 13 00 05 08 17 07 15 22 10 13 > Table 24 shows the Intercorrelation matrix for the true and estimated item parameters for dataset 6. The correlation between the true and estimated d-parameters was 0.986, which is somewhat lower than was the case with dataset 5. The true a-parameters for dimension 1 had a correlation of 0.109 with the dimension 2 estimated a-values, while the dimension 2 true a-values had a correlation of 0.056 with the dimension 1 estimated a-values. Again, there is no significant correlation between the true and estimated a-parameters. Table 24 Intercorrelation Matrix for True and Estimated Item Parameters for Dataset 6 d l.OOO -0.048 0.076 0.989 -0.379 1.000 -0.985 -0.021 -0.065 ®2 1.000 0.054 0.056 d 1.000 -0.469 1.000 a2 Table 25 shows the intercorrelation matrix for the true and estimated abilities for dataset 6. As can be seen, the true abilities had slightly higher correlations with the dimension 2 ability estimates than with the dimension 1 ability estimates. Each true ability parameter had about equal correlations with the two sets of ability parameter estimates. The inter- dimension ability correlations was 0.493 for the true abilities and -0.378 for the estimated abilities. Table 25 Intercorrelation Matrix for True and Estimated Ability Parameters for Dataset 6 Estimated 6^ 1.000 0.493 l.OOO 0.424 0.446 1.000 There was a correlation of 0,993 between the proportion-correct difficulty index and the true d-parameter. The correlation was 0.983 for the estimated d-parameter. The point blserlal discrimination index had correlations of 0.158 and -0.096 with the true a-parameters and correlations of 0.032 and 0.137 with the estimated a-parameters. Table 26 shows the intercorrelation matrix for the true and estimated item parameters and the rotated factor loadings for dataset 6. As was the case with dataset 5, there was a strong relationship between the estimated a- parameters and both sets of rotated factor loadings, but no significant relationship between the true a-parameters and the factor loadings. The first four eigenvalues from the principal components analysis of dataset 6 were 11.18, 1.28, 1.25, and 1.22. There is a large first factor, and the second factor is almost nonexistent. w*. .S .%• •.* •/ • a-•*». i", Item Parameters Factor Loadings Variable Estimated Orthogonal Oblique d ^2 ^ ®2 I II I II True d 1 .000 -0.048 0.076 0.989 -0.379 0.335 -0.370 0.382 -0.306 -0.382 1.000 -0.985 -0.021 -0.065 0.109 -0.008 0.045 0.036 -0.030 1.000 0.054 0.056 -0.076 -0.003 -0.012 -0.021 0.005 Estimated d^ 1.000 -0.469 0.420 -0.439 0.449 -0.366 -0.450 1.000 -0.840 0.953 -0.886 0.897 0.928 1.000 -0.852 0.918 -0.659 -0.902 Orthogonal I 1.000 -0.944 0.926 0.982 II 1.000 -0.750 -0.990 Oblique I 1.000 0.837 The correlations of the point biserlals with the varlmax rotated loadings were 0.074 and 0.194. With the oblique rotated loadings, the correlations were 0.371 and -0.079. The proportion-correct difficulty Index had correlations of -0.367 and 0.379 with the varlmax rotated loadings, and correlations of -0.303 and -0.379 with the oblique rotated loadings. The correlation of the proportion-correct difficulty and point biserial discrimination Indexes was 0.094. Dataset 7 . Table 27 shows the Item parameter estimates, item statistics, and rotated factor loadings for dataset 7. These data were generated using test 2 and a group of examinees having an inter-dimension ability correlation of 0.35. The mean score on test 2 for this group was 24.07, and the standard deviation was 8.20. The KR-20 reliability was 0.86. The correlation between factors, obtained from the oblique rotation, was 0.23. 30 ' » Tj- V— V w -^-j^ T- -» ■:r»^ Table 27 Item Parameter Estimates, Item Statistics, and Factor Loadings for Dataset 7 Item Parameter Estimates Factor Loadings Statistics Orthogonal .22 0.25 0.18 .88 0.24 0.25 .47 0.34 0.36 .84 0.27 0.11 .32 0.30 0.34 .24 0.27 0.34 .59 0.35 0.34 .41 0.28 0.32 .40 0.29 0.27 .18 0.25 0.29 .39 0.34 0.38 .56 0.36 0.28 .46 0.31 0.37 .17 0.26 0.57 .59 0.32 0.30 .72 0.30 0.16 .38 0.34 0.37 .59 0.34 0.10 .42 0.31 0.28 .64 0.31 0.32 .07 0.20 0.17 .45 0.34 0.38 .36 0.32 0.33 .46 0.35 0.38 .35 0.29 0.38 .74 0.29 0.45 .52 0.34 0.54 .79 0.27 0.36 .57 0.34 0.26 .30 0.30 0.28 .70 0.29 0.36 .56 0.32 0.46 .29 0.28 0.34 .59 0.33 0.48 .78 0.26 0.32 .71 0.30 0.45 .28 0.33 0.55 .56 0.32 0.28 .53 0.34 0.28 .38 0.31 0.33 .82 0.26 0.02 .44 0.33 0.32 Oblique V W V > ^ Item Parameter Item Factor Loadings Estimates Statistics Orthogonal Oblique 43 -0.82 0.54 1.10 0.34 0.32 0.16 0.49 0.36 44 0.34 0.85 0.96 0.57 0.36 0.36 0.36 0.48 45 0.97 0.96 0.73 0.69 0.31 0.41 0.25 0.49 46 0.81 0.56 0.86 0.67 0.29 0.21 0.39 0.36 47 -1.81 0.80 0.95 0.18 0.29 0.35 0.32 0.46 48 -1.98 1.39 0.51 0.18 0.28 0.56 0.09 0.56 49 -0.70 0.74 1.13 0.36 0.36 0.30 0.42 0.46 '50 -0.78 0.88 0.86 0.34 0.34 0.40 0.30 0.50 Table 28 contains the Intercorrelatlon matrix for the true and estimated Item parameters for dataset 7. The correlation between the true and estimated d-parameters was 0.988, which Is about the same as was obtained for dataset 6. The true a-parameters for dimension 1 had a correlation of 0.049 with the dimension 2 estimated a-values, and the dimension 2 true a-parameters had a correlation of 0.061 with the dimension I estimated a-values. Table 28 Intercorrelatlon Matrix for True and Estimated Item Parameters for Dataset 7 Table 29 contains the Intercorrelatlon matrix for the true and estimated ability parameters for dataset 7. The values In this table follow a pattern much like what was found for dataset 6. Each true ability had about equal correlations with the two sets of estimates, and both had slightly higher correlations with the dimension 2 estimates than with the dimension 1 estimates. The inter-dimension ability correlation was 0.334 for the true abilities and -0.380 for the estimates. Table 29 Intercorrelation Matrix for True and Estimated Ability Parameters for Dataset 7 Variable True Estimated 01 02 ®1 ®2 True 1.000 0.334 0.381 0.430 1.000 0.411 0.450 Estimated ®2 1.000 -0.380 1.000 There was a correlation of 0.993 between the proportion-correct difficulty index and the true d-parameter. The correlation was 0.983 for the estimated d-parameter. The point biserlal discrimination index had correlations of 0.170 and -0.107 with the true a-parameters and correlations of 0.010 and 0.051 with the estimated a-parameters. Table 30 shows the Intercorrelation matrix for the true and estimated item parameters and the rotated factor loadings for dataset 7. As was the case with datasets 5 and 6, there was a strong relationship between the estimated a-parameters and both sets of rotated factor loadings» but no significant relationship between the true a-parameters and the factor loadings. The first four eigenvalues from the principal components analysis of dataset 7 were 10.33, 1.31, 1.26, and 1.23. There is a large first factor, and the second factor is almost nonexistent. Item Parameters Factor Loadings Variable True Estimated Orthogonal Oblique True d 1.000 -0.048 0.076 0.987 -0.323 0.364 -0.228 0.248 -0.192 0.243 ®1 1.000 -0.985 -0.043 -0.069 0.049 0.006 0.040 0.043 0.021 ^2 1.000 0.075 0.061 -0.018 0.003 -0.027 -0.018 -0.017 Estimated d 1.000 -0.433 0.468 -0.322 0.339 -0.279 0.337 ®1 1.000 -0.837 0.924 -0.872 0.885 -0.908 4 1.000 -0.832 0.910 -0.695 0.890 Orthogonal I 1.000 -0.940 0.961 -0.980 II 1.000 -0.808 0.989 Oblique I 1.000 -0.887 The correlations of the point biserials with the varimax rotated loadings were 0.2A6 with 0.030. With the oblique rotated loadings, the correlations were 0.448 and -0.089. The proportion-correct difficulty index had correlations of -0.241 and 0.262 with the varimax rotated loadings, and correlations of -0.204 and 0.257 with the oblique rotated loadings. The correlation of the proportion-correct difficulty and point biserial discrimination indexes was 0.097. Dataset 8 . Table 31 shows the item parameter estimates, item statistics, and rotated factor loadings for dataset 8. These data were generated using test 2 and a group having an inter-dimension ability correlation of 0.00. The mean score on test 2 for this group was 24.18, and the standard deviation was 7.32. The KR-20 reliability was 0.82. The correlation between factors, obtained from the oblique rotation, was -0.57. b u Item Parameter Estimates Factor Loadings Statistics Orthogonal Oblique 26 1.24 0.96 0.56 0.75 0.24 27 0.10 1.18 0.48 0.52 0.28 28 1.54 0.67 0.72 0.80 0.22 29 0.34 0.76 0.95 0.57 0.31 30 -1.02 0.71 0.81 0.29 0.27 31 0.98 0.70 0.74 0.71 0.25 32 0.30 0.97 0.54 0.56 0.27 33 -0.96 0.73 0.54 0.30 0.24 34 0.49 1.06 0.66 0.60 0.30 35 1.47 0.86 0.56 0.79 0.23 36 1.00 0.70 0.73 0.71 0.25 37 -1.68 2.00 0.17 0.27 0.28 38 0.33 0.53 0.99 0.57 0.28 39 0.17 0.63 0.95 0.54 0.30 40 -0.56 0.68 0.71 0.38 0.26 41 2.66 0.0 2.00 0.82 0.22 42 -0.28 0.77 0.63 0.44 0.27 43 -0.81 0.68 0.88 0.33 0.28 40 0.14 0.43 0.02 48 0.08 0.54 0.11 23 0.31 0.17 -0.25 25 0.41 0.16 -0.37 29 0.30 0.25 -0.22 31 0.23 0.30 -0.13 43 0.12 0.47 0.05 36 0.15 0.38 -0.02 43 0.19 0.45 -0.03 34 0.18 0.34 -0.06 33 0.23 0.32 -0.12 54 0.06 0.62 0.16 21 0.37 0.13 -0.34 20 0.42 0.11 -0.40 28 0.28 0.25 -0.20 06 0.60 -0.25 -0.71 31 0.25 0.29 -0.16 20 0.41 0.11 -0.38 35 0.29 0.32 -0.19 37 0.19 0.37 -0.06 06 0.47 -0.07 -0.51 47 0.28 0.29 -0.19 47 0.06 0.54 0.13 27 0.41 0.20 -0.35 42 0.19 0.44 -0.03 Table 32 contains the Intercorrelatlon matrix for the true and estimated Item parameters for dataset 8. The correlation between the true and estimated d-parameters was 0.989. The true and estimated a-values for dimension 1 had a correlation of 0.11, while for dimension 2 the correlation was 0.037. 1.000 “1 ^2 Estimated d -0.048 1.000 0.076 -0.985 1.000 0.989 -0.074 0.104 1.000 -0.331 0.011 -0.017 -0.428 1.000 0.370 -0.005 0.037 0.464 -0.784 1.000 Table 33 shows the Intercorrelation matrix for the true and estimated ability parameters for dataset 8. The same pattern is present as was found for the other test 2 datasets. Both sets of true abilities had about equal correlations with the two sets of estimates. The inter-dimension ability correlation was -0.036 for the true values and -0.466 for the estimates. Table 33 Intercorrelation Matrix for True and Estimated Ability Parameters for Dataset 8 1.000 -0.036 0.288 0.309 1.000 0.343 0.344 e? 1.000 -0.466 «2 1.000 The correlation between the true d-parameter and the proportion-correct index was 0.993. The correlation between the proportion-correct index and the estimated d-values was 0.986. The correlations of the point biserlal index and the a-values were 0.140 and -0.083 for the true values, and 0.203 and 0.084 for the estimates. Table 34 shows the Intercorrelation matrix for the true and estimated item parameters and the two sets of rotated factor loadings obtained for dataset 8. As has been the pattern with the test 2 datasets, there is a strong relationship between the estimated a-values and both sets of loadings, but no correlation between the true a-values and the factor loadings. The ».V Vk. .V . ..k'k A. k- first four eigenvalues from the principal components analysis of dataset 8 were 8.19, 1.39, 1.30, and 1.29. Table 34 Intercorrelation Matrix for True and Estimated Item Parameters and Factor Loadings for Dataset 8 Variable Item Parameters Factor Loading s True Estimated Orthogonal Oblique d ai ^2 d aj 32 I II I II True d 1 .000 -0.048 0.076 0.989 -0.331 0.370 -0.308 0.308 -0.312 -0.312 1.000 -0.985 -0.074 0.011 -0.005 -0.035 0.072 -0.043 -0.064 ap 1.000 0.104 -0.017 0.037 0.037 -0.056 0.042 0.052 Estimated d 1.000 -0.428 0.464 -0.390 0.380 -0.392 -0.388 1.000 -0.784 0.887 -0.786 0.876 0.823 a2 1.000 -0.818 0.888 -0.844 -0.882 Orthogonal I 1.000 -0.929 0.997 0.960 II 1.000 -0.956 -0.995 Oblique I 1.000 0.980 II 1.000 The correlations of the point blserial Index with the rotated factor loadings were 0.243 and 0.074 for the varlmax rotation, and 0.175 and 0.006 for the oblique rotation. For the proportion-correct difficulty index the correlations were -0.324 and 0.330 with the varlmax rotated loadings, and -0.330 and -0.333 with the oblique rotated factor loadings. The correlation between the point biserlals and proportion-correct values was 0.073. Discussion The purpose of this study was to examine the effects of correlated abilities on observed test characteristics, and to explore the implications of correlated abilities for multidimensional item response theory, or MIRT, analysis when a model is used that does not explicitly account for such a correlation. The approach taken was to generate simulation data with known true parameters, using varying levels of correlation between abilities, and to analyze the data using a number of different test analysis procedures. The procedures selected were item analysis, principal component analysis, and MIRT analysis. In addition, correlational analyses were performed to explore the relationship of obtained statistics and parameter estimates to the true parameters, as well as the Interrelationships among the item parameter estimates and traditional item statistics. All of these analyses were performed for two different tests. One test was comprised of two relatively Independent dimensions (each item discriminating on only one of the dimensions), while the other test was comprised of two correlated dimensions (each item discriminating at least moderately on both dimensions). The analyses of these two tests will be discussed separately, and then an attempt will be made to Integrate the results of the two sets of analyses. 38 Test 1 Analyses In test 1, an attempt was made to use two relatively unidimensional subsets of items. One subset of items discriminated fairly highly on the first dimension, and very poorly on the second. The second subset of items discriminated fairly highly on the second dimension and relatively poorly on the first. Thus, the test had two relatively independent factors. In an attempt to evaluate the effects of correlated abilities for such a test, the three types of test analysis procedures were applied, and the results analyzed. The results for each type of test analysis procedure will now be discussed separately. Item Analysis Results . The one clear pattern which emerged from the item analyses performed on these data was the decline of the test KR-20 reliability with the decline of the correlation between ability dimensions. This trend is summarized in Table 35, which shows a drop in reliability from 0.86 to 0.77 when the inter-dimension ability correlation dropped from 0.70 to 0.00. This is an indication that an Increased correlation between ability dimensions results in more common variance, which in turn yields a higher KR-20. The fact that the items were constructed to have a relatively low inter- dimensional correlation may have somewhat mitigated this effect, but it did not eliminate it. Table 35 Relationship Between Ability Correlation and Test Reliability for Test 1 Dataset Ability Correlation KR-20 1 0.70 0.86 2 0.50 0.84 3 0.35 0.82 4 0.00 0.77 Principal Component Analysis Results. One pattern evident in the factor analysis results was the decline in factor correlation with the decline in ability correlation. This pattern was similar to that found for the KR-20 analyses, and is also indicative of the increased multidimensionality of the test data (decreased size of the common component). The pattern Is illustrated in Table 36. A similar sort of pattern was evident In the eigenvalues resulting from the principal components analyses. The first four eigenvalues for each set of response data are shown In Table 37. As can be seen, as the ability correlation decreased, so did the size of the first eigenvalue. At the same time, the size of the second eigenvalue Increased. Table 37 Relationship Between Ability Correlation and Eigenvalues for Test 1 Dataset Ability Eigenvalues Correlation E 2 E 3 E4 1 0.70 10.01 1.50 1.31 1.27 2 0.50 9.09 1.79 1.30 1.28 3 0.35 7.92 2.08 1.43 1.29 4 0.00 6.32 2.74 1.42 1.32 Another trend found in the results of these analyses was the tendency toward an increase in the correlations between the true item discriminations and both the orthogonal and oblique rotated factor loadings as the ability correlation decreased. This tendency is shown in Table 38. Bear in mind that the correlations shown are not always matched on dimensions. That is, in some cases the dimension 1 a-value correlation with the factor II loadings is shown. Ability Orthogonal Oblique Dataset elation I II I .70 0.86 0.86 0.87 .50 0.94 0.90 0.93 .35 0.96 0.95 0.96 .00 0.96 0.95 0.96 MIRT Analysis Results . As the correlation between ability dimensions decreased, there was a slight increase in the correlations between the true and estimated item parameters. That is to say, the estimation program was better able to recover the true parameters when the ability dimensions were leas correlated. This trend is shown in Table 39. Note that the dimensions did not always match. That is, some of the correlations reported in Table 39 for the discrimination values are actually correlations between the true a- values on one dimension and the estimated a-values for the other dimension. Table 39 Relationship Between Ability Correlation and the Correlation Between True and Estimated Item Parameters for Test 1 Ability Item Parameter Dataset Correlation d 1 0.70 0.996 0.73 2 0.50 0.998 0.87 3 0.35 0.999 0.92 4 0.00 0.999 0.92 Another interesting result from the MIRT analyses Involved the correlation between the discrimination parameters. For test I the correlation between the true dimension 1 and dimension 2 a-values was -0.987. Table 40 shows the a-value correlations for the estimated a-values for the four datasets for test 1. Note that the correlation is well below (in absolute value) the correlation for the true values for dataset 1, but as the ability correlation decreases the obtained a-value correlation more nearly approaches the true value. Dataset Ability Correlation a-Value Correlation 1 0.70 -0.841 2 0.50 -0.865 3 0.35 -0.936 4 0.00 -0.937 Table 41 summarizes the correlations between the true and estimated ability parameters for test 1. As can be seen, the correlations Increase as the inter-dimension ability correlation decreases. The estimation program was better able to recover the true examinee abilities when the ability dimensions were less correlated. This is consistent with the results obtained for the item parameters. Table 41 Relationship Between Ability Correlation and the Correlations Between the True and Estimated Ability Parameters for Test 1 Dataset Ability Ability Parameter Correlation ®2 1 0.70 0.70 0.67 2 0.50 0.74 0.72 3 0.35 0.78 0.77 4 0.00 0.81 Table 42 shows the correlations between the two ability dimensions for the ability parameter estimates for the four datasets for test 1. Also shown are the actual true ability correlations obtained for the four datasets (as opposed to the true correlations for the populations from which examinees were selected). As can be seen, the sample correlations for the true abilities were quite close to the true population values. In every case, however, the correlation for the ability parameter estimates is very near 0.0. Regardless of the true correlation between ability dimensions, the ability parameter estimate dimensions are forced to be uncorrelated. 42 Table 42 Relationship Between Ability Correlation and Inter-Dimension Ability Estimate Correlation for Test 1 Dataset Ability Correlation (Population) Ability Correlation (Sample) Ability Estimate Correlation 1 0.70 0.68 -0.14 2 0.50 0.49 -0.15 3 0.35 0.35 -0.09 4 0.00 0.01 -0.05 Test 2 Analyses In test 2 items were selected to have at least moderately high discriminations on both dimensions. Thus, the test was constructed to have somewhat correlated dimensions. The same analyses performed on test 1 were then run on test 2. Again, the results for each type of test analysis procedure will be discussed separately. Item Analysis Results. The only pattern discernible among the item analysis results for the test 2 datasets was a slight decline in KR-20 reliability with a decrease in the correlation between ability dimensions. This trend can be seen in the data shown in Table 43. The trend is less dramatic than was the case with the test 1 data, however. In the test 1 datasets the principal component was due primarily to the ability dimension correlation, while in the test 2 data the principal component was at least partially due to the nature of the items. Thus, a decline in ability correlation did not have as great an impact on its size. Table 43 Relationship Between Ability Correlation and Test Reliability for Test 2 Dataset Ability Correlation KR-20 5 0.70 0.89 6 0.50 0.87 7 0.35 0.86 8 0.00 0.82 Principal Component Analysis Results . The pattern of factor correlations was rather confusing for the test 2 datasets. These values are shown In Table 44. Dataset Ability Correlation Factor Correlation As can be seen, there is no systematic relationship between ability correlation and factor correlation. To be consistent with the previous results, the factor correlation should have declined slightly with the decrease in ability correlation. The results for datasets 5 and 8 are consistent with this, but the results for datasets 6 and 7 are quite Inconsistent with this. As yet, no satisfactory explanation for this phenomenon has been determined. Table 45 shows the trend in eigenvalues as ability correlation decreased for the test 2 data. As can be seen, these results are much more consistent with the item analysis results. There was a slight decrease in the size of the first eigenvalue with the decrease in ability correlation. The decrease is more marked from dataset 7 to 8 than between the other tests, but so was the decline in KR-20. There was a negligible increase in the size of the second eigenvalue as the ability correlation decreased. Table 45 Relationship Between Ability Correlation and Eigenvalues for Test 2 Dataset Ability Correlation Eigenvalues .70 12.32 1.23 1.21 .50 11.18 1.28 1.25 .35 10.33 1.31 1.26 .00 8.19 1.39 1.30 A pattern found for the test 2 analyses which is in marked contrast to the results for test 1 Involves the correlations between the true a-values and the factor loadings. For test 1 there was a tendency for an Increase in correlations between a-values and factor loadings as the ability correlation decreased. All of the a-value-factor loading correlations were relatively high, though. For the test 2 data, the true a-value-factor loading correlations were all around 0.0, regardless of the ability correlation. MIRT Analysis Results . As the correlation between ability dimensions decreased, the correlation between the true and estimated d-parameters decreased slightly, except for dataset 8, for which the correlation was the same as for dataset 6. The correlation between the true and estimated a-parameters was essentially 0.0 for all four test 2 datasets. As the correlation between ability dimensions decreased, the correlations between the two a-parameter estimates for the two dimensions diverged from the correlation between the true a-parameters. This trend is shown In Table 46. The true a-parameter correlation for test 2 was -0.985. Table 46 Relationship Between Ability Correlation and Inter-Dimension a-Value Correlation Dataset Ability Correlation a-Value Correlation -0.837 -0.840 -0.836 -0.784 Table 47 shows the relationship between true ability correlation and the correlation between the true and estimated ability parameters. As can be seen, as the true ability correlation decreased, so did the correlations between the true and estimated ability parameters. Table 47 Relationship Between Ability Correlation and the Correlations Between the True and Estimated Ability Parameters for Test 2 -■••• 5 0 .70 0.592 .> 6 0 .50 0.424 0 .35 0.381 8 r-TT— 0 .00 0.288 Table 48 shows a very interesting pattern involving the inter-dimension ability estimate correlations for test 2. As can be seen, the sample true ability dimension correlations are quite close to the target population valnes. For the ability estimates, however, the correlation differs substantially from the true ability correlation. In every case the correlation is nej^ative, and it becomes more negative as the true value approached 0.0. Table 48 Relationship Between Ability Correlation and Inter-Dimension Ability Estimate Correlation for Test 2 Dataset Ability Correlation (Population) Ability Correlation (Sample) Ability Estimate Correlation 5 0.70 0.69 -0.20 6 0.50 0.49 -0.38 7 0.35 0.33 -0.38 8 0.00 -0.04 -0.47 Overall Results It is quite clear that it is not enough to talk about the 'dimensionality of a test', or about whether the 'dimensions of a test' are correlated. There are two distinct concepts Involved, and they play quite different roles in determining the latent structure of response data. The first concept is latent item structure, and the second is latent ability structure. The latent structure of a test item refers to the number and interrelationships of the dimensions required for performance on the item. In this research, two types of latent item structure we^'e employed. For test 1, each item required basically only one dimension. The first half of the items required the first dimension, while the second half required the second dimension. Thus, there were two dimensions underlying the test, each of which operated relatively independently of the other. For test 2, each item required two dimensions. For some items the first dimension was more dominant than the second, while for the remaining items the second dimension was dominant. Since all items required both dimensions, the two dimensions did not operate as Independently as for test 1. The latent ability structure of an examinee refers to the number and interrelationships of the dimensions underlying the examinee's responses. In this study the latent ability space was always two-dimensional, but the correlation of the two examinee latent ability dimensions varied. It is the interaction of these two concepts which determines the latent structure of response data. When the item dimensions operate relatively Independently, the dimensionality of the response data depends to a great extent on the latent ability structure of the examinees. Correlated abilities 46 tend to yield response data with a single dominant component or, viewed differently, response data with correlated latent dimensions. Uncorrelated abilities tend to yield response data with relatively uncorrelated latent dimensions. When the latent item dimensions do not operate independently, the effect of the latent ability structure is less pronounced. The effect of the correlation between latent ability dimensions is the same, but less extreme. The effect of the interaction of latent ability and item structures has serious implications for the analysis of test data and, perhaps more Importantly, for test development. Clearly it is not sufficient to consider only item characteristics when constructing or analyzing a test. It is not appropriate to assume that the latent ability structure is determined by item characteristics. It is necessary to consider how the two interact to produce a latent structure for test data. The results of this stuiy also have important implications for the application of MIRT methodology. The presence of correlated abilities certainly had pronounced effects on the results obtained from the application of the MIRT model selected for this study. The most important finding of this study regarding the use of MIRT methodology involves the inability of the MIRT model estimation program to recover the true dimensions when the dimensions were correlated. When latent item dimensions are independent, the procedure works fairly well, even when the latent ability dimensions are correlated. However, an Increased correlation between latent ability dimensions does lower the correlations between the true and estimated abilities. This does not necessarily mean that the estimation process breaks down. It simply means that the nature of the ability dimensions recovered by estimation is somewhat different than for the true dimensions. It is entirely possible that the estimated dimensions are in some sense a rotation of the true dimensions due to the fact that the estimation procedure and/or model does not explicitly account for inter-dimensional correlations. When latent item dimensions are not independent, the recovered dimensions are different from the true dimensions regardless of whether or not the latent ability dimensions are correlated. Again, some type of rotation of the latent ability structure might be involved. Under the circumstances, if MIRT methodology is to be viable, research on this question must be conducted. If a rotation is involved, it is imperative that its nature be discovered. Moreover, if MIRT parameters are to be invariant and interpretable, it seems likely that it will be necessary to develop something analagous to factor rotations in factor analysis. Summary and Conclusions A study was conducted to assess the effects of correlated abilities on test characteristics, and to explore the effects of correlated abilities on the use of a multidimensional item response theory model which does not explicitly account for such a correlation. Two tests were constructed. One test had two relatively unidimensional subsets of items, while the other had items that were all two-dimensional. For each test response data were '.■vV-'.l •V. V •j‘. • generated according to a multidimensional two-parameter logistic model using four groups of examinees. The groups of examinees differed in the degree of Inter-dlmension ability correlation. To evaluate the effects of correlated abilities on test characteristics, the simulated response data were analyzed using Item analysis and principal component analysis techniques. To assess the effects of correlated abilities on the use of the multidimensional model, the parameters of the model were estimated, and the estimates were compared to the true parameters. The results of this study Indicated that the presence of correlated abilities has Important Implications for the characteristics of test data, and for the application of multidimensional Item response theory models. It was concluded that It Is necessary to consider latent Item structure as well as latent ability structure In test construction and analysis. It was also concluded that use of multidimensional Item response theory models that do not explicitly account for correlated abilities may not yield accurate information about the nature of underlying dimensions. It was suggested that research should be conducted to determine the relationship between the observed and true correlation between abilities, and to perhaps develop an Item response theory analogue to factor rotation. References Blrnbaum, A. (1968). Some latent trait models and chelr use In inferring an examinees ability. In F.M. Lord and M.R. No/lck, Statistical theories of mental test scores. Reading, MA: Addlson-Wesley. McKinley, R.L. and Reckase, M.D. (August 1983). An extension of the two- parameter logistic model to the multidimensional latent space (Research Report ONR 83-2). Iowa City, lA; The American College Testing Program (a) McKinley, R.L. and Reckase, M.D. (August 1983). An application of a multidimensional extension of the two-parameter logistic latent trait model Research Report ONR83-3). Iowa City, lA: The American College Testing Program, (b) McKinley, R.L. and Reckase, M.D. (1983). MAXLOG: A computer program for the estimation of the parameters of a multidimensional logistic model. Behavior Research Methods and Instrumentation, 389-390. (c) Reckase, M.D. and McKinley, R.L. (1982). Some latent trait theory in a multidimensional latent space. Paper presented at the 1982 Invitational IRT/CAT Conference, Wyzata, Minnesota. * • * - * 4 . * ^ V •. •; 1 Ir. lick lond 04<tci o4 Itvil ItWirch Liiiion Of Met, Far East IPO San Franclico, Cl fiS03 1 Ir. lobart Irtaux -V MVTRAEOUIPCEI Coda I-095R V Orlando, FL 32813 • • 1 Dr. Stanlay Collytr , Off;c» of Naval Technology I 60C N. Ouincy Street ftflmqton, VP 2221'’ 1 CDF Hike Cc'rar. Office of Na«al Research BOO H. Quincy St. Code 27!: Arl;-.gten, VP 21217 1 Dr. Joh- E'.lis Na<> Personnel RVD Cente' Sar. Diego, CP 92252 1 Or. Richard Elster Dtpa rtaent of Adainistrative Sciences ■aval Postgraduate School Monterey, Cl 9'.940 1 N. PAT FEDERICO Code P13 VROC San Diego, CA 92132 I Hr. Dick Hoshae MVOP-135 Arlington Annex Rooa 2834 Mashington , DC 20350 1 Dr. Norean J. Kerr Chief of Naval Technical Training Naval Pir Station Neaphis (75) Millington, TN 38034 1 Dr. Leonard hroeker lav> Personnel RkD Center San Diego, CP 92152 1 Dar.i; Lang Na<y Personnel RID Center San Diegc, CA 92152 1 Dr. Nilliaa L. Maloy (02i Chief of Naval Education and Training Naval Air Station Pensacola, FL 32308 1 Dr. Jues Hciride Navy Personnel RED Center San Diego, CP 92152 I Dr Nilliaa Montague NPRDC Code 13 San Diego, CP 92152 1 Library, Code P201L Navy Personnel RED Center San Diego, CA 92132 1 Technical Director Navy Personnel RED Center San Diego, CA 92152 6 Personnel E Training Resea'ch Sroup Code 442PT Office of Naval Research Arlington, VP 22217 1 DR. MARTIN F. HISKOFF NAVY PERSONNEL RE D CENTER SAN DIE80, CA 92152 1 Dr. Carl Ross CNET-PDCD luilding 90 Brcat Lakes NTC, IL 40088 1 Nr. Drew Sands NPRDC Code 62 San Diego, CA 92152 1 Nary Schrat: Navy Personnel RED Center San Diego, CA 92152 1 Dr. Robert 6. Seith Office of Chief of Naval Operations 0F-9B7ri Hashingtcn, DC 20350 I 2r. Alfred SecSe, Director Departiient N-" Naval Yraining Equipeent Center Orlando, FL 328:7 1 Ir. Richard Smw LMton Scitntift 04{ice o4 Raval Rttcarch Iranch Office, London Box 39 FPO Hc« York, RY 09S10 1 Dr. Richard Sorenoon Na>y Personnel RID Cente' Sar Diego, CA 92152 1 Dr. Frede'^icl Steinheise' CNC - OPli' Naw Anne> Ar;ir.gton, VA 20370 1 hr. Brad Syopson Nav, Personnel RID Center Sa* Diego, CA 92152 I Dr. Jaoes Toeeddale Tec^nkal Director Navy Personnel RLC Ceme' Sa'. Diego, CA 92152 1 Dr. Frank Vicino Navy Personnel RkD Center San Diego, CA 92152 1 Dr. Eduard Negoan Office of Naval Rotearch (Code 411StP) 800 North Duincy Street Arlington, VA 22217 1 Dr. Ronald Hcitiaan Naval Postgraduate School Departoent of Adoinistrativi Sciences Honterey, CA 93.940 1 Dr. Douglas Hetzel Code 12 Navy Personnel RtO Center San Diego, CA 92152 1 DR. MARTIN F. NISKOFF NAVv PERSONNEL Rt D CENTER SAN D1E60. CA 92152 1 Hr John H. No’.fe Nav) Personnel Rtl' Center San Diego, CA ^2152 1 Dr. Hallace Nulfeck, III Navy Personnel RtD Center San Diego, CA 92152 Narine Corps 1 Jerry Lehnus CAT Project Office H6 Narine Corps Nashington , DC 20360 1 Headquarters, U. S. Narine Corps Code NPI-20 Nashington, DC 20360 1 Spec lal Assistant for Narine Corps Natters Code 1001 Of'Ke of Na.al Research BOC N. Cuincy St. Arlington, VA 22217 1 Najor Frank Yohannan, USNC Headquarters, Narine Corps (Code NPl-20) Hasringtc.*, DC 203BC Aray 1 Dr. Kent Eaton Aroy Research Institute 5001 Eisenhouer Blvd. Alexandria , VA 22333 1 Dr. Nyron Fischl U.S. Aroy Research Institute for the Social and Behavioral Sciences 5001 Eisenhouer Avenue Alevandna, VA 22333 I Dr. Clesser Nartm Aroy Research Institute 5001 Eisenhouer Blvd. Alexandria, VA 22333 1 Dr. Nilliai E. Nordtrock FHC-ADCO Box 25 APO. NY 0971C 1 Dr, Ha-oli F. C’Neil. Jr. Director, Training Research Lab Ariy Research Institute 5001 Eisenhouer Avenue Alexandria, VA 22333 r S’ % ] CMundtr, O.S. Ariy RttHrtk Inftitute for tht Miavioril t Social Scitncis ATTN: PERI-IR (Dr. Judith Orasanu) 9001 EistnhoMT Dvtnut Oliiandria, VA 22333 I Nr. Robtrt Ross U.S. Aroy Rosearch Institute for the Social and letiavioral Sciences 300! EistnhoMr Avenue Alciandria, VA 22333 1 Dr. Robert Sasoor U. S. Aroy Research Institute for the Dehavioral and Social Sciences 3001 Eisenbouer Avenue Alexandria. VA 22333 1 Dr. Joyce Shields Arty Research Institute for the Bshav'oral and Social Sciences 5001 Eisenhover Avenue Alexandria. VA 2233: 1 Dr. Hilda Ring Arty Research Institute SOCl Eisenhouer Ave. Alexandria, VA 22333 Air Force 1 Dr. Earl A. Alluisi HR, AFHRL (AFSC) Drooks AFi, TI 78235 1 Nr. Rayoond E. Christal RFHRL/NOE Drooks AFD, Tl 78235 I Dr. Alfred R. Fregly AFOSR/NL Dolling AFB, DC 20332 1 Dr. Senevieve Haddad ^rcgrat Harager ;.;(e Sciences Drectorate AFCSR Selling AFB, DC 20332 1 Dr. Patrick Ryllonen AFHRL/HOE 1 Nr. Randolph Park RFNRL/NOAM Drooks AFB, TI 78235 1 Dr. Roger Pennell Air Force Huaan Resources Laboratory LOM-y AFB, CO 80230 1 Dr. Nalcola Ree AFHRL/HP Brooks AFB, TI 78235 1 Hajor John Nelsh AFHRL/NOAN Drooks AFD , TI 78223 Departrent of Defense 12 Defense Technical Inforoation Center Cateron Station, Bldg 5 Alexandria, VA 22314 Attn: TC 1 Nilitary Assistant for Training and frsonnel Technology Office of the Under Secretary of Oefens for Research I Engineering Root 33129, The Pentagon Mashington, DC 2C30I i Dr. H. Steve Selloan Office of the Assistant Secretary of Defense (HPA I L) 2B269 The Pentagon Mashington, DC 2030! 1 Dr. Robert A. Nisher OISDRE (ELS The Pentagon, Rooa 3D129 Mashington, DC 20301 Drooks AFB, Tl 78235 ^ - kv . • - ■ « ,1 J . I,-H >i-:- CivlllM AgtOClca 1 Or. Virn H. Urry Pirsoflntl OtO Ctnttr IHfict o4 Ptrsonntl Nanagmnt IfOO E Strtft m tlat^in|ton, K 20413 1 Hr. TkOMi A. Mart U. S. Coait Suard Inatituta P. 0. Substation IB Oklahoaa City, OK 73169 1 Dr. Jostph L. Young, Director Heaory I Cognitive Processes National Science Foundation Hashington, DC 20550 f?' Private Sector 1 Br. Jaaes Algina university of Florida Bainesville, FL 32605 1 Br. Erling B. Andersen • Beparteent of Statistics Itudiestraede 6 1433 Cepenhagen J BENHARK . 1 -J 1 Or. .iaac leja'- Educational Testing Service Princeton, NJ 0fl43C _.v3 1 Dr. Henucha Birenbaua ■ vj *• School of Education ■ Tel Aviv University • Tel Aviv, Raeat Aviv 69976 Israel 1 Or. Nerner Sirke •W • . ’ Personalstaeaaat der Bundesuer' D-300C Koelr- 90 NEST 6ERNANY iii 1 Dr. R. Darrell Ixk . »• • Beparteent of Education University of Chicago Chicago, IL 60637 •J I te ]■■- '*■ V-iN* ^4 I Nr. Arnold lobrer SKtion oF Psychological Research Caserne Petits Chateau CRE 1000 Brussels Bilgiua 1 Dr. Robert Brennan Aeerican College Testing Prograes P. 0. lo« 168 loea City. lA 32243 1 Dr. Blenn Bryan 6206 Pee Road Bethesda, HD 20B17 1 Dr. Ernest R. Cadotte 307 stokely University of Tennessee Knoxville. ’N 3’?16 1 Dr. John B. Ca"’ol! 409 Elliott 'd. Chapel Hill, RC * • " j. » 1 Ir. Notmi) Cliff ••ft. of Ptycholofy Itoiv. of So. California llRivirfity Pa. k Ui UnftlM, Cil V0007 > Ir. Mint Croib*; Elucation Rtatarcb Ctnttr Onivarsity of Ltyftn loorbaavdaan 2 2334 EH Layden Tha NETHEPLAHDS 1 laa Cronbact IS Laburnua Road Atharton, CA 9^205 I CTI/Hc6rat-Hill Library 2300 Gardan Road Honta'’ay, CA ?j940 1 Dr. Haltar Cunningbai Uni varsity of Hiaoi ••partoant of Psychology Sainesvilla, FL 32611 1 Dr. Dattpradad Divgi Syracusa Uni varsity ••partoant of PsycAoiogr Syracusa, ME 33210 I Ir. Eooanual lonchin •apartaont of Psychology (Riivorflty of Illinois OMapaign, IL *1820 1 Ir. Hai-Xi long •all Foundation Rooa 314, luilding I •00 Roosevalt Road Gian Ellyn, IL *0137 1 Or. Frit: Drasgoa Oapa'tKnt of Psychology Univarsity of Illinois 6{)3 E. Danial St. Chaapaign, IL 61S20 1 Dr. SuSi' Eafcarlscn fsrcHGLOS'- rEP4'‘:PE^' 1 ERIC Facility-Acquisitions 4833 Rugby Avanua •atbasda, HD 20014 1 Or. 8anjaaia A. FairAank, ir. McFann-lray t Asaociatas, Inc. 923 Callaghan Suita 225 San Antonio, Tl 78228 1 Dr. Laorard Faldt Lindquist Carta' ir Baasuraant Univarsity o* Ion loaa City. lA 32241 1 Ur;v. Prof. Or. Sarhard Fischar Liabiggassa 3/3 A 1010 Vianna AUSTRIA 1 Professor Honald Fitigarald Univarsity of Ha* England Araidala, Rau SOkth Halas 2351 AUSTRALIA 1 Or. Daitar Flatcbar University of Oregon Oapartaant of Co^utar Science Eugene, DR P7403 1 Or. John R. Fradariksan •olt laranak * Rauaan SO Houlton Street Caabridga, HA 02138 1 Dr. Janice Gifford Univarsity of Hassachusatts School of Education Aaharst, HA 01002 1 Dr. Robert Slasar Learning Research * Oevalopaent Center University of Pittsburgt 3'»39 O’Hara Street PITTSIURSN, PA 15260 1 Dr. Harvin D. Slock 217 Stone Hall Cornell University Ithaca, MY 14853 IL! J.' L' »m I'J WA' '-F/V WM 'W*»■* *.» •}■ *.i'' 1 tr. Itrt Srtcn Johns Hopkins University Departoont of Psychology CbarlH I 34th Street laitioore, HS 21218 1 M. JMES 6. 6REEM LHK UHIVERSITY OF P1TTSIUR6H 3439 O’HARA STREET PIT^SBURSH, PA 15213 1 Dr. Ron Haobleton School of Education University of Hassachusetts Aeherst, HA 01002 1 Or. Paul Horst i77 6 Street, 1184 Chwli Vista. CA 9001C 1 Dr. Llovd Huipbrevs Depa''taent cf hsychslog, University of Illinois 807 East Daniel Street Chaepaign, IL 81020 1 Dr. Steven Hunka Departoent of Education University of Alberta Edoonton, Alberta CANADA 1 Ir. Earl Hunt Dept, of Psychology University of Nashington Seattle, M 9S10S 1 Dr. Jack Hunter 2122 Coohdge St. Lansing, HI 48908 1 Dr, Huynh Huynh College of Education University of South Carolina Coluebia, SC 29208 1 Dr. Douglas H. Jones Advanced Statistical Technologies Corporation 10 Trafalgar Cou''t Lau'enceville, HI 08:46 1 Dr. Harcel Just Departoent of Psychology Carnegie-Hellon University Pittsburgh, PA 1S213 1 Dr. Doeetrios Karis Departoent of Psycho!ogy University of Illinois 803 E. Daniel Street Chaepaign, IL 81820 1 Professor John A. Keats Departaent of Psychology The University of Neucastle N.S.M. 2308 AUSTRALIA 1 Or. Milliao Koch University of Tevas-Austin Neasureoent and Evaluation Center Austin, Tt 78703 1 Dr. Alan Lesgold Learning R80 Center University of Pittsburgh 3939 O’Hara Street Pittsburgh, PA 15280 1 Or. Hichael Levine Departoent of Educational Psychology 210 Education Dldg. University of Illinois Chaepaign, IL 81801 1 Dr. Charles Leeis FKolteit Sociale Hetenschappen Rijksuniversiteit Broningen Oude Boteringestraat 23 97126C Broningen Netherlands 1 Dr. RcRiert Linn College of Education University of Illinois Urbana, IL 81801 1 Hr. Phillip Livingston Systees and Applied Sciences Corpcratic 861: Keniluorth Avenue Riverdale. HD 2084C 1 Dr. Robtrt Lockun Center tor Naval Analysis 200 North Beauregard St. Alexandria, VA 22311 1 Dr. Frederic R. Lord Educational Testing Service Princeton, NJ 08341 1 Dr. Jaees Luasder Defarteent of Psychology University of Nesterr Aust'il.a Nedlands H.A. iOOP AUSTRALIA 1 Or. Don Lyor, P. 0. Bo> 4< Higley , AZ 8523f> 1 Dr. Gary Harco Stop 31-E Educational ^estin; Ser.ice Princeton, NJ 0843! 1 Dr, Scott Kaxtell Departeent of Psychology University of Notre Oaee Notre Oaae, IN WZb 1 Dr. Saauel T. Nayc Loyola University of Chicago 120 North Hichigan Avenue Chicago, IL Will 1 Nr. Robert HcKlnley Noerican College Testing Prograos P.O. Dox la8 Iowa City, lA 32243 1 Dr. Barbara Heans Huoan Resources Research Organization 300 North Nashington Alexandria, VA 22314 1 Processor Jason Nil lean Departeent c' Education Stone Hall Corbel; lir.iversit* If ica. Nv 1403: 1 Dr. N. Alan Niceeander University of Otlahoea Departeent of Psychology Oklahoea City, OF 730i9 1 Dr. Donald A Noroan Cognitive Science, C-013 Univ. of California, San Diego La Jolla, CA 92093 1 Dr. Ne'.vin R. Novic> 336 Lindquist Cente' for Heasureent Uni.ersity of loea loea City, lA 32242 1 Dr. Janes Olson NICAT, Inc. 1873 Scuth State Street Orea, UT 84037 1 Hayne N. Patience Anerican Cccnci'. on Education . BED Testing Service, Suite 20 One Dupont Cirle, NN Nashington, DC 20036 1 Dr. Janes Paulson Dept, of Psychology Portland State UniveSrsity P.O. Box 731 Portland, OR 97207 1 Or. Janes H. Pellogrino University of California, Santa Barbara Dept. of Psychology Santa Darabara , CA 93106 1 Dr. Douglas H. Jones Advanced Statistical Technologies Corporation 10 Trafalgar Court Laerenceville, NJ 08148 1 Dr. Steven E. FoltrocL Bell Laboratories 2D-444 60C Nountain Ave. Hur'-av Hii;, NJ 07974 1 Dr. Na-'k D. Reciase ACT P. 0. Bor 168 love City, lA 32243 1 D'. Robert Hislew 711 lilincis Street Geneva. IL 6C!!4 I Ir. Thoui Rtynolds Univtriity of Tnat-Oillti lUrketing ItpirtMnt P. 0. loi 686 Rickariton, TX 79080 1 8r. LawMCt Radntr 403 Elt Ovinut T*koM Park, m 20012 1 Dr. J, Ryan Dtpartaent of Education University of South Carolina Coluabia, SC 29208 1 PROF. FUHIKO SANEJIHA DEPT. OF PSYCHOLOBY UNIVERSITY OF TENNESSEE KNOIVILLE, TN 3^916 1 Frank L. Schaidt Departient of Psychology Rldg. 66 George Mashington University Nashington, DC 200C2 1 Or. Halter Schneider Psychology Departaent 603 E. Daniel Chaapaign, IL 61820 1 Leaell Schoer Psychological 6 Quantitative Foundations College of Education University of loaa loaa City, lA S2242 I Dr. Eaaanuel Donchin Departaent of Psychology University of Illincis Chaapaign, IL 61820 I Dr. Kaiuo Shigenasu 7*9'24 Kugenuaa-F.aigan Fujusana 251 JAPAN 1 Cr. Hilliaa Sias Cente*" for Na«a! final vs; s 200 North Beaurec.rd Street filekandna, VA 21!!! 1 Dr. H. Nallace Sinaiko Prograa Director Nanpoaer Research and Advisory Services Saithsonian Institution 801 North Pitt Street Alexandria, VA 22314 1 Martha Stocking Educational Testing Service Princeton, NJ 08541 1 Dr. Peter Stolof* Center for Naval Analysis 200 North Beauregard Street Alexandria, VA 22311 1 Dr. Hilliaa Stout University of Illinois Departaent of Natheaatics Urbana, IL 61801 1 DR. PATRICK SUPPES INSTITUTE FOR HATHEHATICA. STUOIES IN THE SOCIAL SCIENCES STANFORD UNIVERSITY STANFORD, CA 94305 1 Dr. Hariharan Saaainathan Laboratory of Psychoaetric and Evaluation RHearch School of Education UnivKsity of Massachusetts Aaherst, NA 01003 1 Dr. Kikuai Tatsuoka Coaputar Based Education Research Lab 252 Engineering Research Laboratory Urbana, IL 61801 1 Or. Maurice Tatsuoka 220 Education Bldg 1310 S. Sixth St. Chaapaign, IL 61820 I Dr. David Thissen Departient of Psychology University of Kansas Laurence, KS 66044 1 Dr. Douglas Toune Univ. of Sc. Cali^cnia Behavioral TethnoJc^, ljijs 1845 S. Elena Avt. Redondc Bea:“. Cfi 91277 c .’v/^ 1 #r. Robtrt T«ttt*ki>«i v oi Stitiitic* 0»iv#r»ity of Hitiouri • ColuabK, HO 45201 ^ 1 If. LHyifd Tocktr UBivoriity of Illinoi* Otpirtoont of Piycholooy 603 E. 0«i*l Strut Chaopiign, IL 61B2C 1 Or. V. R. R. Uppuluri Union Carbide Corporation Hue 1 ear Diviiion P. 0. lo> Y Oak Ridge, TN 37830 1 Or. David Vale Assesownt Systees Corporation 2233 University Avenue Suite 310 St. Paul, HH SSIM 1 Dr. Houard Maine' Division of Psychological Studies Educational Testing Service Princeton, M 08540 1 Or. HicAael T. Haller Departoent of Educational Psychology University of Hisconsin—Hiluaukee UilMukK, Ml 53201 1 Dr. Irian Haters HuaRM 300 Morth Hashington Aletandria, VA ^314 1 Or. David J. Heiss H660 Elliott Mall University of Minnesota 75 E. River Road Minneapolis, HN 5S4SS 1 Or. Rand R. ll;lco« University of Southern Califc'nia Decartnent of Psychology Lcs Angeles, CA P0CC7 1 lerean Military Representative RTTH: Hoifgang Hildegrube Strcitkraefteaet I-S300 knn 2 4400 Irandyuine Itreet, MH Uasbington . K 20016 1 Dr. Iruce HilliaH Departeent of Educational Psychology University of Illinois Urbana, IL 61801 1 Ms. Marilyn Hingersky Educational Testing Service Princeton, NJ 0854; 1 Or. Hendy Yen CTI/Mc6ra« Mill Del Monte Research Park Monterey, CA 93940 '2V Cw0*ifIVlV lV^.A