Skip to main content

Full text of "DTIC ADA144869: An Investigation of the Effect of Correlated Abilities on Observed Test Characteristics."

See other formats


^flD-fll44 869 
UNCLASSIFIED 


AN INVESTIGATION OF THE EFFECT OF CORRELATED ABILITIES 1/1 
ON OBSERVED TEST C. . <U) AHERICAN COLL TESTING PROGRAH 
IONA CITV lA R L HCKINLEV ET AL. HAV 84 RR-84-1-0NR 
N8eei4-81-i('8817 F/G 12/1 NL 














































AD-A144 869 


An Investigation of the Effect of Correlated 
Abilities on Observed Test Characteristics 


Robert L McKinley 
and 

Mark D. Reckase 


Research Report ONR84-1 
May 1984 





The Ar , >rican College Testing Program 
Assessment Programs Area 
T est Development Division 
Iowa City, Iowa 52243 



Prepared under Contract No. N00014-81-K0817 
with the Personnel and Training Research Programs 
Psychological Sciences Division 
Office of Naval Research 

Approved for public release, distribution unlimited. 

Reproduction in whole or in pari is permitted for 
any purpose of the United States Government. 

84 08 27 175 













Y Cu ASSi FiC AT'OS Of "^HIS PAGE 'When Dai. 


REPORT DOCUMENTATION PAGE j 

KtAD INSTRl' 'TIONS 

BEKORE COMPLETING FORM 

1 report number 2 GOVT ACCESSION NO 1 

3 RECiPiEn'^’S CA-'A.O: number 



4 TiTlE 'and Subtitle' j 

5 Tvp£ or REPOR”’’ 4 PEROD COVERED 

An Investigation of the Effect of Correlated | 

Abilities on Observed Test Characteristics 

Technical Report 

1 6 PERfORMiNG ORG REPORT NUMBER 

7 author^.; 

B contract or GRAn*^ NUMBER'*,) 

Robert L. McKinley 

Mark D. Reckase 

\'00014-81-K08 17 

9 PEREORMING ORCANI Z ATiON name ano address 

The American College Testing Program 

P.O. Box 168 

Iowa City, lA 522A3 

10 PROGRAM ELEMENT, project task 
AREA 5 WORK UNIT NUMBERS 

P.E.: 6115:5N Proj . : RRf 

r.A. : 042-04-01 

W.U.: NRl50-499 

I' CON'^ROLLING OEEICE NAME AND ADDRESS 

12 report DATE 

Personnel and Training Research Programs 

May, 1984 

Office of Naval Research 

<3 number of pages 

Arlington, VA 22217 

48 

14 monitoring agency name 6 AODRZSSrif dllfarant from Controlling Otfica) 

IS security CLASS, (O/ IM« r.port) 


unclassified 


IS* declassification downgrading 
schedule 


16 Distribution statement ro/ thi$ Rapoet) 


Approved for public release; distribution unlimited. Reproduction in whole 
or part is permitted for any purpose of the United States government. 


n Distribution statement fof tha mbatfmct antarad in Block 20. It dlHarant Irotr Raport) 



19 KEY WORDS ^Continua on ravarsa alda if nacaaamiy and Identify by block numbar) 


Multidimensional Item Response Theory 
Item Response Theory 


Dichotomous Data 
Est Lmation 


20 abstract rContInua on ravaraa alda If nacaaamry mnd Idantlfy by block numbar) 

A study was conducted to assess the effects of correlated abilities on 
test characteristics, and to explore the effects of correlated abilities on 
the use of a multidimensional item response theory model which fioes not 
explicitly account for such a correlation. Two tests were constructed. 

One test had two relatively unidimensional subsets of items, while the 
other had items that were all two-dimensional. For each te‘,t response data 


,, 1473 eoi'’iONOF ■ MOV 6S IS ossoLere 


SEClIRlTv CCASSiriC.TlON This P*Gf 'IWi.n D.l. tnl»r»dl 
































































SeCUHITV CLASSIFICATION OF THIS PACE (Whtn Oala Entmtmd) 


were generated according to a multidimensional two-parameter logistic 
model using four groups of examinees. The groups of examinees differed 
in the degree of inter-dimension ability correlation. 

To evaluate the effects of correlated abilities on observed test 
characteristics, the simulated response data were analyzed using item 
analysis and factor analysis techniques. To assess the effects of 
correlated abilities on the use of the multidimensional model, the 
parameters of the model were estimated, and the estimates were compared 
to the true parameters. 

The results of this study indicated that the presence of correlated 
abilities has important Implications for the characteristics of test data, 
and for the application of multidimensional item response theory models. 

It was concluded that it is necessary to consider latent item structure 
as well as latent ability structure in test construction and analysis. 

It was also concluded that use of multidimensional item response theory 
models that do not explicitly account for correlated abilities may result 
in misinterpretation of the underlying dimensions. It was suggested 
that research should be conducted to determine the nature of the 
misinterpretation and to perhaps develop an item response theory analogue 
to factor rotation. 

/\ 


S/N 0102- LF. 014- 6601 




SECUPITV classification OF THIS PACEIlFhcn Dmim 











































CONTENTS 


Page 


Introduction. 1 

Method . 1 

MIRT Model. 2 

Design.2 

Datasets. 3 

Analyses. 5 

Results . 5 

Test I Analyses. ....5 

Dataset 1.5 

Dataset 2. 9 

Dataset 3. 13 

Dataset 4.... . 17 

Test 2 Analyses.. 21 

Dataset 5.21 

Dataset 6. 25 

Dataset 7.29 

Dataset 8. 33 

Discussion .. 37 

Test 1 Analyses. 38 

Item Analysis Results.... 38 

Principal Component Analysis Results.38 

MIRT Analysis Results. 40 

Test 2 Analyses. .........42 

Item Analysis Results. 42 

Principal Component Analysis Results. ......42 

MIRT Analysis Results. .......44 

Overall Results. 45 

Summary and Conclusions.............46 


48 



References 











































An Investigation of the Effect of Correlated Abilities 
on Observed Test Characteristics 


Because of the required assumption of unidimensionaiity, much of the item 
response theory (IRT) methodology that has been developed is inappropriate for 
a wide range of applications. In such applications, either unidimensional 
sets of items cannot be constructed, or they are not desired. Until recently, 
in such circumstances the practitioner has been forced to abandon IRT and 
adopt more traditional test analysis procedures, or to inappropriately apply 
IRT methods and hope the procedures are robust to violations of the 
unidimensionality assumption. Unfortunately, such robustness has not been 
demonstrated. 

In recent years, researchers have begun grappling with the dimensionality 
problem. Several IRT models have been proposed for the multidimensional case, 
and recently some theory and procedures have been developed for applying such 
models (Reckase and McKinley, 1982; McKinley and Reckase, 1983a, 1983b). The 
work that has been done in this area indicates that it has great promise for 
dealing with the dimensionality problem. 

In multidimensional item response theory (MIRT), one of the most 
important questions that has not yet been addressed focuses on the effect of 
correlated abilities on the interpretation of model parameters. Logically, it 
seems desirable to construct different, homogeneous (unidimensionai) sets of 
items to measure each ability or trait of interest. In the case of unrelated 
abilities, such as math computation ability and vocabulary ability, this is a 
practical approach. However, if the abilities of Interest are related, such 
as in the case of reading comprehension and vocabulary, constructing an item 
set that measures only one of these two abilities is more difficult. 

Developing a unidimensional set of vocabulary items seems easy, but how does 
one construct reading comprehension items that do not also include at least a 
small vocabulary component? 

The purpose of this study is to investigate the effects of varying 
degrees of correlation between abilities on observed test characteristics. 

This research has two primary objectives. The first objective is to identify 
the characteristics of response data yielded in the case of correlated 
abilities. If unique characteristics can be identified and used to 
distinguish the multidimensional data from those produced in the 
unidimensional case, then it should be possible to identify real test 
situations in which a MIRT model is appropriate. The second objective of this 
research is to determine the effect of varying degrees of correlation between 
abilities on estimates of parameters from a MIRT model which does not 
explicitly account for such a correlation. 

Method 

MIRT methodology is relatively new and probably unfamiliar to many. 
Therefore, before continuing with a discussion of this research, a brief 
discussion of the MIRT model selected for this study will be presented. For a 
more detailed discussion of this model, see McKinley and Reckase, 1983a. 








-f.' 












The MIRT Model 


The MIRT model selected for this study is an extension of the two- 
parameter logistic (2PL) model proposed by Birnbaum (1968). The 
multidimensional 2PL model, or M2PL model, is given by 




exp(d. + a.' 0.) 
_ i —1—1 

1 + exp(d^ + 


where is a vector of ability parameters for examinee j, is a vector of 
discrimination parameters for Item 1, Is a scalar Item parameter related to 
item difficulty, and P£(_0_j) is the probability of a correct response to item 1 
by an examinee having ability . 

The discrimination and ability vectors in Equation 1 are both of order m, 
where m is the number of dimensions comprising the complete latent space. 

The a^' ^ term in Equation 1 can be written as 


^ ®jk * 


where is the item discrimination parameter for dimension k and 0^^^ is the 

examinee ability parameter for dimension k. In the unidimensional case 
Equation 1 simplifies to the 2PL model (without the D*1.7 term usually used in 
the 2PL model) with 




where bj^ is the difficulty parameter for item 1 from the 2PL model. 
Design 


The basic design of this study involved the simulation and analysis of 
response data generated for examinees having varying levels of correlation 
between their abilities on different dimensions. The generated data were then 
analyzed using traditional test analysis techniques to determine the effects 
of the correlated abilities on the test characteristics. Afterward, the data 
were analyzed using the M2PL model to determine the effects of correlated 
abilities on the estimates of the parameters of the model. 





Datasets 

Simulated test data were generated for two different types of test. The 
first test type measured two dimensions, with half of the items on the test 
measuring predominantly one dimension, and other half measuring predominantly 
the second dimension. The second test type also measured two dimensions, but 
for this test each item measured both dimensions. Four datasets were 
generated for each test using Interdlmenslonal ability correlations of .7, .5 
.35, and 0. Table 1 summarizes the eight datasets that were created. 


Table 1 


Dataset 


Simulated Datasets 


Test 


- V-V.' .- 


1 

1 

0.70 

2 

1 

0.50 

3 

1 

0.35 

4 

1 

0.00 

5 

2 

0.70 

6 

2 

0.50 

7 

2 

0.35 

8 

2 

0.00 


Table 2 shows the true item parameters used to simulate the two tests. 
Each test had 50 items, and the same set of d-parameters was used for both 
tests. As can be seen, for test 1 the items generally have high 
discriminations on one dimension or the other, but not both. For test 2 the 
items tend to have high discriminations on both dimensions. 


Table 2 


V. 


True Item Parameters Used 
To Simulate Both Tests 





V . . .v 

'-’.V 

* _ • 































H 

i 




Table 2(Continued) 

True Item Parameters Used 
To Simulate Both Tests 


Item 

d 

Test 1 


Test 2 


®1 

®2 

^1 

32 

11 

-0.51 

1.38 

0.30 

0.98 

1.02 

12 

0.27 

1.39 

0.27 

0.93 

1.07 

13 

-0.14 

1.21 

0.73 

1.20 

0.75 

14 

-1.71 

1.30 

0.55 

0.68 

1.24 

15 

0.41 

1.39 

0.27 

0.68 

1.24 

16 

1.15 

1.37 

0.37 

1.02 

0.98 

17 

-0.52 

1.37 

0.35 

1.02 

0.98 

18 

0.40 

1.39 

0.28 

0.83 

1.15 

19 

-0.28 

1.41 

0.11 

1.25 

0.67 

20 

0.59 

1.34 

0.46 

1.06 

0.94 

21 

-2.77 

1.35 

0.41 

0.61 

1.27 

22 

-0.17 

1.36 

0.38 

0.90 

1.09 

23 

-0.58 

1.35 

0.44 

0.74 

1.21 

24 

-0.11 

1.39 

0.26 

0.72 

1.22 

25 

-0.61 

1.38 

0.30 

0.94 

1.06 

26 

1.20 

0.33 

1.38 

0.57 

1.30 

27 

0.14 

0.35 

1.37 

0.73 

1.21 

28 

1.40 

0.07 

1.41 

0.99 

1.01 

29 

0.34 

0.21 

1.40 

0.75 

1.20 

30 

-1.02 

0.32 

1.38 

0.86 

1.12 

31 

1.05 

0.34 

1.37 

0.83 

1.14 

32 

0.31 

0.55 

1.30 

0.74 

1.21 

33 

-0.83 

0.45 

1.34 

0.61 

1.27 

34 

0.55 

0.33 

1.37 

0.99 

1.01 

35 

1.46 

0.42 

1.35 

0.79 

1.17 

36 

1.05 

0.33 

1.38 

1.07 

0.92 

37 

-1.05 

0.37 

1.37 

1.11 

0.88 

38 

0.34 

0.54 

1.31 

0.80 

1.17 

39 

0.20 

0.37 

1.37 

0.68 

1.24 

40 

-0.45 

0.40 

1.36 

1.17 

0.79 

41 

1.67 

0.34 

1.37 

0.72 

1.22 

42 

-0.20 

0.27 

1.39 

0.93 

1.07 

43 

-0.69 

0.30 

1.38 

1.17 

0.79 

44 

0.35 

0.28 

1.39 

0.93 

1.07 

45 

0.93 

0.47 

1.34 

1.15 

0.82 

46 

0.86 

0.67 

1.25 

1.10 

0.89 

47 

-1.70 

0.41 

1.35 

1.14 

0.83 

48 

-1.54 

0.40 

1.36 

1.04 

0.95 

49 

-0.57 

0.33 

1.38 

1.06 

0.94 

50 

-0.78 

0.28 

1.39 

0.95 

1.05 

Mean 

-0.06 

0.86 

0.86 

0.89 

1.07 

S.D. 

1.04 

0.51 

0.51 

0.18 

0.16 

























Each group of examinees consisted of 2000 simulated subjects with true 
abilities selected at random from a bivariate normal distribution having both 
means equal to 0.0, both standard deviations equal to (;.50, and having the 
appropriate correlation. Each group was used for only one of the two tests. 
Thus, there were eight sets of item responses generated. Each set of 
responses was stored in the appropriate dataset, depending on the test and the 
correlation between abilities. The same set of analyses were then run on each 
dataset. 

Analyses 

There were four types of analyses run on each dataset in Table 1. The 
first type was an item analysis. This consisted of computing item proportion- 
correct difficulty indices, item point blserial discrimination indices, and a 
KR-20 test reliability coefficient. 

The second type of analysis performed on each dataset was a principal 
components analysis. More specifically, a principal components analysis of 
tetrachorlc correlations was performed on each dataset. Using the results of 
the principal components analyses, both a varimax rotated and an oblique 
rotated factor solution were obtained for each dataset. In each case two 
factors were rotated. 

The third type of analysis performed was the application of the M2PL 
model to the data. This consisted of estimating item and person parameters 
for the M2PL model for each dataset. Parameter estimation for the M2PL model 
was performed using the MAXLOG program (McKinley and Reckase, 1983c). 

The final type of analysis performed on these data consisted of 
correlational analyses. Correlations were computed among true and estimated 
item parameters, item statistics (traditional difficulty and discrimination), 
and factor loadings. Correlations between true and estimated ability 
parameters were also computed. 

Results 

As was discussed above, there were four types of analyses performed on 
the eight datasets created for this study - item analyses, principal component 
analyses, MIRT analyses, and correlation analyses. The results obtained from 
all four of these sets of analyses will be presented for each dataset 
separately, beginning with the four datasets based on the first test. 

Remember that the first test contained two subsets of items, each of which was 
relatively unidimensional. After presenting these results, the results for 
the second test will be presented. The second test had items that each 
measured two dimensions. 


Test 1 Analyses 

Dataset 1 . Table 3 shows the results of the item analysis, principal 
components analysis, and MIRT analysis of the first dataset. This dataset was 
created using test 1 and a group of examinees having an inter-dimension 
ability correlation of 0.70. The columns headed 'Item Parameter Estimates' 
are the results obtained from the MIRT analysis, and are estimates of the item 
parameters of the M2PL model. The columns headed 'Item Statistics' are the 



































6 


( 


proportion-correct item difficulties (p) and item point biserial 
discrimination indexes (pbis) obtained from the item analysis of the first 
dataset. Columns 6 and 7 are the varimax rotated factor loadings for the 
first two factors of the principal components analysis of tetrachoric 
correlations. The last two columns are oblique rotated factor loadings for 
the first two factors from the principal components analysis. 


Table 3 

Item Parameter Estimates, Item Statistics, and Factor 
Loadings for Dataset 1 


Item Parameter Item 

Factor 

Loadings 

Estimates Statistics 

Orthogonal 

Oblique 


aj 

.43 

0.88 

.69 

0.73 

.72 

0.89 

.80 

0.44 

.46 

0.91 

.64 

0.59 

.72 

0.77 

.51 

0.74 

.38 

1.02 

.43 

0.97 

.41 

1.11 

.45 

0.68 

.59 

1.00 

.66 

0.78 

.46 

0.73 

.46 

0.91 

.45 

1.03 

.45 

1.02 

.06 

1.33 

.60 

0.81 

.37 

0.73 

.51 

0.89 

.51 

0.95 

.60 

0.67 

.0 

2.00 

.01 

0.48 

.88 

0.4 

6 

.78 

0.2 

8 

.92 

0.3 

4 

.95 

0.4 

0 

.62 

0.2 

6 

.70 

0.4 

4 

.70 

.34 

0.5 

0.2 

0 

9 
























































f / 


\ 


Item Parameter 


Estimates 


Statistics 


Factor Loadings 
Orthogonal Oblique 



The mean score on test 1 for this group of examinees was 24,14, and the 
standard deviation was 8.13. The KR-20 reliability for these data was 0.86. 
The correlation between the factors, obtained from the oblique solution, was 
0.64. 

Table 4 shows the intercorrelation matrix for the true and estimated item 
parameters for the first dataset. As can be seen, the correlations of the 
true and estimated item parameters were 0.996 for the d-parameter, 0.731 for 
the true a on the first dimension and the estimated a for the second 
dimension, and 0.768 for the true a for the second dimension and the estimated 
a for the first dimension. Thus, the d-parameter was very well estimated, and 
the a-parameters were only moderately well estimated. 






" » * K * • ' 




■*. % , 




































Variable 


True 


Estimated 



Table 5 shows the Intercorrelatlon matrix of the true and estimated 
ability parameters obtained for the first dataset. As can be seen, the 
ability on dimension 1 had a correlation of 0.670 with the ability estimate on 
the second dimension, while there was a correlation of 0.704 between the true 
ability for dimension 2 and the ability estimate on dimension 1. Despite the 
correlation of 0.685 obtained for the true abilities, the estimated abilities 
were not correlated (r=-0.140). Thus, while the abilities for this group were 
moderately well estimated, the correlation between the dimensions was not 
recovered by the estimation process. 


Table 5 

Intercorrelatlon Matrix for True and Estimated 
Ability Parameters for Dataset 1 



1.000 


0.685 

1.000 


0.444 

0.704 

1.000 


0.67 

0.39 

-0.14 

1.00 


The correlation of the proportion-correct difficulty Index and the d- 
parameter was 0.995 (for true and estimated d-values), which is about what was 
expected. The point blserlal discrimination index had a correlation of -0.131 
with the true a-parameter for dimension 1 and 0.166 for the second 
dimension. Using the a-value estimates the correlation was 0.258 for 
dimension 1 and 0.059 for dimension 2. This, too, was much as was expected. 
Since the point biserial is strongly affected by the dimensionality of the 







































Items, It should not have a strong relationship to discrimination on a single 
dimension for two-dimensional data. 


Table 6 shows the Intercorrelatlon matrix for the true and estimated item 
parameters and the varlmax and oblique rotated factor loadings for the first 
dataset. As can be seen, there was a strong relationship between the factor 
loadings (both varlmax and oblique rotated) and the Item parameters (both true 
and estimated). The first four eigenvalues obtained from the principal 
components analysis of these data were 10.01, 1.50, 1.31, and 1.27. There 
appeared to be a strong first factor and a much smaller second factor. This 
Is consistent with the high Inter-dlmenslon ability correlation for these 
data. 


Table 6 

Intercorrelatlon Matrix for True and Estimated Item 
Parameters and Factor Loadings for Dataset 1 


Variable 


Item Parameters 


Estimated 


Factor Loadings 
Orthogonal Oblique 


True 

d 1.000 

-0.172 

0. 

.159 

0. 

.996 

0. 

.317 

-0. 

.272 

0. 

.330 

-0. 

,308 

0.329 

-0.319 



1.000 

-0, 

.987 

-0. 

.189 

-0. 

.751 

0, 

.731 

-0, 

.852 

0. 

.860 

-0.868 

0.872 


^2 


1, 

.000 

0. 

.177 

0. 

.768 

-0, 

.730 

0, 

.862 

-0, 

.842 

0.871 

-0.862 

Estimated 

d^ 




1, 

.000 

0, 

.358 

-0, 

.323 

0, 

.362 

-0, 

.341 

0.362 

-0.352 








1, 

.000 

-0, 

.841 

0, 

.953 

-0, 

.852 

0.942 

-0.894 


^2 








1, 

.000 

-0, 

.852 

0. 

.90S 

-0.880 

0.908 

Orthogonal 

I 










1, 

.000 

-0. 

,918 

0.995 

-0.956 


II 












1, 

.000 

-0.954 

0.994 

Oblique 

I 














1.000 

-0.981 


The correlation of the point blserlal Index and the factor loadings was 
0.269 and 0.091 for the two varlmax rotated factors, and 0.180 and -0.009 for 
the two oblique rotated factors. The proportion-correct difficulty Index had 
correlations of 0.329 and -0.323 with the varlmax rotated loadings, and 0.332 
and -0.330 with the oblique rotated loadings. The proportion-correct and 
point blserlal indexes had a correlation of 0.086. 

Dataset 2 . Table 7 shows the Item parameter estimates. Item statistics, 
and factor loadings obtained for dataset 2. These data were generated using 
test 1 and a group of examinees with an inter-dimension ability correlation of 
0.50. The mean score on test 1 for this group was 24.49, and the standard 
deviation was 7.73. The KR-20 reliability was 0.84, which Is slightly lower 
than the KR-20 for dataset 1. The correlation between the factors, obtained 
from the oblique rotation, was -0.59, which is slightly lower than for dataset 
1, and opposite In sign. 













































M ■•'■■I* 


-■■-■. •r.^: 


10 








Table 7 


Item Parameter Estitr tea, Item Statistics, and Factor 
Loadings for Dataset 2 


1% 






* *• 

fc.w: 




i 


■ •. 

i'"' 
«^. 

w. 

"•' J 

fe 


r.: 




Item 

Item Parameter 

Estimates 

Item 

Statistics 

Factor Loadings 

Orthogonal Oblique 

d 

«1 

32 

P 

pbls 

I 

II 

I 

II 

1 

-1.41 

0.55 

0.91 

0.23 

0.28 

0.23 

0.41 

0.11 

-0.40 

2 

2.23 

0.62 

0.80 

0.87 

0.24 

0.27 

0.36 

0.18 

-0.32 

3 

-0.05 

0.29 

1.07 

0.49 

0.29 

0.10 

0.50 

-0.07 

-0.55 

4 

2.15 

0.13 

1.19 

0.85 

0.22 

0.06 

0.51 

-0.12 

-0.57 

5 

-0.81 

0.37 

1.06 

0.34 

0.29 

0.14 

0.48 

-0.02 

-0.50 

6 

-1.31 

0.33 

0.83 

0.24 

0.24 

0.12 

0.41 

-0.01 

-0.43 

7 

0.49 

0.61 

0.75 

0.60 

0.31 

0.30 

0.35 

0.22 

-0.30 

8 

-0.47 

0.42 

0.96 

0.40 

0.30 

0.19 

0.43 

0.06 

-0.43 

9 

-0.52 

0.46 

0.69 

0.39 

0.27 

0.19 

0.37 

0.08 

-0.37 

10 

-1.69 

0.44 

1.16 

0.21 

0.28 

0.16 

0.48 

0.01 

-0.50 

11 

-0.57 

0.32 

0.96 

0.38 

0.28 

0.12 

0.47 

-0.04 

-0.50 

12 

0.30 

0.41 

1.03 

0.56 

0.31 

0.18 

0.46 

0.04 

-0.47 

13 

-0.12 

0.72 

0.70 

0.47 

0.33 

0.33 

0.34 

0.26 

-0.27 

14 

-1.89 

0.65 

1.02 

0.18 

0.29 

0.26 

0.42 

0.14 

-0.40 

15 

0.41 

0.53 

0.90 

0.59 

0.32 

0.23 

0.43 

0.10 

-0.42 

16 

1.22 

0.49 

0.85 

0.74 

0.28 

0.22 

0.40 

0.11 

-0.39 

17 

-0.56 

0.43 

0.84 

0.38 

0.29 

0.20 

0.39 

0.09 

-0.38 

18 

0.35 

0.49 

0.83 

0.57 

0.31 

0.21 

0.41 

0.10 

-0.40 

19 

-0.40 

0.0 

1.86 

0.44 

0.27 

0.02 

0.54 

-0.19 

-0.63 

20 

0.68 

0.66 

0.95 

0.64 

0.36 

0.29 

0.44 

0.17 

-0.41 

21 

-2.74 

0.31 

0.94 

0.08 

0.17 

0.10 

0.41 

-0.04 

-0.45 

22 

-0.19 

0.42 

0.95 

0.46 

0.31 

0.19 

0.43 

0.06 

-0.44 

23 

-0.63 

0.56 

0.80 

0.37 

0.31 

0.22 

0.42 

0.10 

-0.41 

24 

-0.20 

0.39 

0.92 

0.46 

0.30 

0.18 

0.43 

0.06 

-0.43 

25 

-0.72 

0.40 

0.81 

0.35 

0.27 

0.18 

0.40 

0.06 

-0.40 

26 

1.29 

0.80 

0.27 

0.76 

0.24 

0.37 

0.16 

0.38 

-0.04 

27 

0.18 

0.80 

0.55 

0.54 

0.32 

0.38 

0.27 

0.35 

-0.17 

28 

1.47 

0.71 

0.28 

0.79 

0.21 

0.37 

0.10 

0.40 


29 

0.40 

0.93 

0.23 

0.58 

0.27 

0.43 

0.12 

0.46 


30 

-1.02 

0.84 

0.45 

0.29 

0.28 

0.40 

0.20 

0.39 

-0.08 

31 

1.31 

1.32 

0.25 

0.73 

0.30 

0.53 

0.11 

0.59 

0.08 

32 

0.28 

1.00 

0.50 

0.56 

0.33 

0.46 

0.21 

0.46 

-0.07 

33 

-0.79 

0.88 

0.29 

0.34 

0.26 

0.43 

0.12 

0.46 


34 

0.63 

1.11 

0.28 

0.63 

0.30 

0.49 

0.12 

0.53 

0.05 

35 

1.46 

0.77 

0.42 

0.78 

0.25 

0.38 

0.19 

0.38 

-0.08 

36 

1.05 

0.79 

0.29 

0.72 

0.24 

0.40 

0.12 

0.43 

0.01 

37 

-1.21 

1.23 

0.18 

0.28 

0.27 

0.50 

0.08 

0.56 

0.10 

38 

0.43 

1.14 

0.23 

0.59 

0.29 

0.50 

0.10 

0.55 

0.07 

39 

0.19 

0.93 

0.42 

0.54 

0.32 

0.42 

0.22 

0.42 

-0.09 

40 

-0.53 

0.92 

0.30 

0.39 

0.28 

0.44 

0.14 

0.46 

0.01 

41 

1.69 

0.73 

0.40 

0.82 

0.23 

0.36 

0.18 

0.35 

-0.08 

42 

-0.22 

0.78 

0.36 

0.45 

0.28 

0.39 

0.18 

0.39 

-0.06 




■ . • . I 

•■'.v •- 





;>vtJ 






•M 


















































Table 7(Contlnued) 


Item Parameter Estimates, Item Statistics, and Factor 
Loadings for Dataset 2 


item Parameter 
Item Estimates 


Item 

Statistics 


Factor Loadings 


Orthogonal Oblique 



d 

^1 

32 

P 

phis 

I 

II 

I 

II 

43 

-0.74 

0.79 

0.58 

0.35 

0.31 

0.36 

0.28 

0.32 

-0.19 

44 

0.34 

0.72 

0.27 

0.58 

0.25 

0.36 

0.15 

0.36 

-0.04 

45 

0.93 

1.16 

0.35 

0.68 

0.32 

0.49 

0.16 

0.52 

-0.00 

46 

0.88 

0.90 

0.51 

0.68 

0.31 

0.42 

0.23 

0.41 

-0.11 

47 

-1.74 

0.90 

0.46 

0.18 

0.25 

0.38 

0.21 

0.37 

-0.10 

48 

-1.76 

1.28 

0.27 

0.20 

0.27 

0.48 

0.14 

0.52 

0.03 

49 

-0.55 

0.69 

0.39 

0.38 

0.26 

0.35 

0.19 

0.34 

-0.09 

50 

-0.77 

0.77 

0.40 

0.34 

0.27 

0.39 

0.18 

0.39 

-0.07 


Table 8 

shows the 

Intercorrelation 

matrix 

for the 

true and 

estimated 

item 


parameters for dataset 2. The true and estimated d-parameters had a 
correlation of 0.998, indicating that the d-parameter was once again very well 
estimated. The correlation between the true and estimated a-values was 0.866 
for the dimension 1 true value and the dimension 2 estimated value, and 0.834 
for the dimension 2 true value and dimension 1 estimated value. The a-values, 
then, were better estimated for dataset 2 than for dataset 1. 


Table 8 


Intercorrelation Matrix for True and Estimated 
Item Parameters for Dataset 2 





True 



Estimated 


Variable 


d 


32 

d 

®1 

32 

True 

d 

1.000 

-0.172 

0.159 

0.998 

0.112 

-0.182 


®1 

32 

d 


1.000 

-0.987 

-0.180 

-0.784 

0.866 




1.000 

0.167 

0.834 

-0.886 

Estimated 




1.000 

0.115 

-0.189 


®1 

32 





1.000 

-0.865 







1.000 


Table 9 shows the Intercorrelation matrix of the true and estimated 
ability parameters obtained for the second dataset. The true ability on 
dimension 1 had a correlation of 0.716 with the estimated ability on dimension 
2, while there was a correlation of 0.743 for the true ability for dimension 2 
and the dimension 1 estimated ability. The inter-dimension ability 
correlation was 0.494 for the true values, and -0.150 for the estimated 
































values. These ability parameters were estimated slightly better than were the 
parameters for dataset 1, but once again the Inter-dlmenslon ability 
correlation was not recovered during the estimation process. 

Table 9 


Intercorrelation Matrix for True and Estimated 
Ability Parameters for Dataset 2 




True 


Estimated 

Variable 








®1 

®2 

®1 

®2 

True 


1.000 

0.494 

0.321 

0.716 



1.000 

0.743 

-0.279 

Estimated 

®2 



1.000 

-0.150 





1.000 


The correlation of the proportion-correct difficulty index and the d- 
parameter was 0.993 (0.995 for the estimated d-parameter). The point biserlal 
index had a correlation with the true a-parameters of 0.187 for the first 
dimension and -0.134 for dimension 2. Those correlations were 0.166 and 0.123 
when computed with the estimated a-values. 

Table 10 shows the intercorrelation matrix for the true and estimated 
item parameters and the factor loadings for dataset 2. Again, the 
relationship between the true and estimated item parameters and the varlmax 
and oblique rotated factor loadings were quite strong. The first four 
eigenvalues from the principal components analysis were 9.09, 1.79, 1.30, and 
1.28. The first factor for these data was slightly smaller than for the first 
dataset, and the second factor slightly larger. This is consistent with the 
lower inter-dlmenslon ability correlation for this group of examinees. 
































Table 10 










Intercorrelation Matrix for True and Estimated Item 
Parameters and Factor Loadings for Dataset 2 


Variable 



Item Parameters 



Factor ! 

Loadings 




True 


Estimated 


Orthogonal 

Oblique 



d 

^1 

^2 

d 

®1 

®2 

I II 

I 

II 

True 

d 1 

.000 

-0.172 

0.159 

0.998 

0.112 

-0.182 

0.201 -0.191 

0.202 

0.196 




1.000 

-0.987 

-0.180 

-0.784 

0.866 

-0.859 0.936 

-0.890 

-0.930 


ao 



1.000 

0.167 

0.834 

-0.886 

0.899 -0.943 

0.923 

0.945 

Estimated 

d^ 




1.000 

0.115 

-0.189 

0.204 -0.197 

0.205 

0.201 


Si 





1.000 

-0.865 

0.976 -0.876 

0.963 

0.911 


aj 






1.000 

-0.907 0.954 

-0.931 

-0.955 

Orthogonal 

I 







1.000 -0.930 

0.996 

0.959 


11 







1.000 

-0.961 

-0.996 

Oblique 

I 








1.000 

0.981 


11 









1.000 


The correlation of the point blserial index and the factor loadings was 
0.138 and 0.185 for the varlmax rotated loadings and 0.057 and -0.109 for the 
oblique rotated loadings. The proportion-correct difficulty index had 
correlations of 0.212 and -0.208 with the varimax rotated loadings and 0.214 
and 0.212 with the oblique rotated loadings. The proportion-correct 
difficulty index and the point biserlal index had a correlation of 0.008. 

Dataset 3 . Table 11 shows the item parameter estimates, item statistics, 
and factor loadings for dataset 3. These data were generated using test 1 and 
a group of examinees with an inter-dimension ability correlation of 0.35. The 
mean score on test 1 for this group was 24.66 and the standard deviation was 
7.25. The KR-20 reliability was 0.82. This was slightly lower than for 
dataset 2. The correlation between the factors, obtained from the oblique 
rotation, was 0.52, which is slightly lower than for dataset 2, and opposite 
in sign. 








P} 


'S' 

.IV 








;• 






' • • • • ■'V ■* • 
































Item Parameter 


Estimates 


illll 


Statistics 


Factor Loadings 
Orthogonal Oblique 


p 

pbis 

.23 

0.22 

.90 

0.16 

.51 

0.27 

.85 

0.23 

.33 

0.24 

.24 

0.26 

.63 

0.29 

.42 

0.24 

.40 

0.25 

.18 

0.24 

.39 

0.27 

.55 

0.29 

.47 

0.31 

.19 

0.22 

.59 

0.25 

,75 

0.25 

.38 

0.26 

.58 

0.23 

.44 

0.24 

.63 

0.27 

,07 

0.17 

.46 

0.27 

.39 

0.28 

.48 

0.25 

.39 

0.29 

.75 

0.23 

.54 

0.26 

.78 

0.20 

.56 

0.28 

.29 

0.26 

.71 

0.25 

.57 

0.31 

.33 

0.30 

.62 

0.28 

.79 

0.21 


34 

0 

37 

-0 

37 

0 

49 

-0 

40 

0 



































































Table 11(Continued) 


Item Parameter Estimates, Item Statistics, and Factor 
Loadings for Dataset 3 


Item 


Item Parameter 
Estimates 


Item 

Statistics 


Factor Loadings 


Orthogonal Oblique 



d 

^1 

^2 

P 

phis 

I 

II 

I 

II 

36 

1.05 

0.80 

0.34 

0.71 

0.55 

0.39 

0.16 

0.39 

0.06 

37 

-1.08 

0.88 

0.34 

0.28 

0.26 

0.41 

0.17 

0.40 

0.07 

38 

0.39 

0.96 

0.34 

0.58 

0.30 

0.45 

0.17 

0.45 

-0.00 

39 

0.16 

0.90 

0.24 

0.53 

0.27 

0.43 

0.12 

0.45 

-0.00 

40 

-0.46 

0.89 

0.26 

0.40 

0.27 

0.44 

0.12 

0.45 

0.00 

41 

1.83 

1.08 

0.14 

0.82 

0.22 

0.47 

0.06 

0.51 

-0.08 

42 

-0.19 

0.97 

0.18 

0.46 

0.26 

0.45 

0.09 

0.48 

-0.03 

43 

-0.68 

0.73 

0.33 

0.35 

0.25 

0.38 

0.15 

0.38 

0.05 

44 

0.37 

0.75 

0.30 

0.58 

0.26 

0.37 

0.17 

0.37 

0.07 

45 

1.04 

0.80 

0.32 

0.71 

0.25 

0.40 

0.15 

0.41 

0.05 

46 

0.92 

0.71 

0.54 

0.69 

0.28 

0.34 

0.26 

0.30 

0.19 

47 

-1.68 

0.86 

0.39 

0.19 

0.24 

0.39 

0.18 

0.38 

0.08 

48 

-1.59 

0.80 

0.41 

0.20 

0.24 

0.38 

0.19 

0.37 

0.10 

49 

-0.58 

0.98 

0.39 

0.38 

0.30 

0.45 

0.18 

0.45 

0.06 


-0.74 

0.74 

0.54 

0.34 

0.29 

0.37 

0.26 

0.33 

0.18 


Table 12 shows the Intercorrelation matrix for the true and estimated 
item parameters for dataset 3. The true and estimated d-parameters had a 
correlation of 0.999, indicating that the d-parameter was very well 
estimated. The dimension 1 true a-values had a correlation of 0.921 with the 
dimension 2 estimated a-values, while there was a correlation of 0.937 between 
the dimension 2 true a-values and the dimension 1 estimated a-values. The a- 
values, then, were fairly well estimated for these data. 








































Variable 



Table 13 shows the intercorrelation matrix of the true and estimated 
ability parameters obtained for dataset 3. The true abilities for dimension 1 
had a correlation of 0.772 with dimension 2 of the estimated abilities. The 
dimension 2 true abilities had a correlation of 0.779 with the dimension 1 
estimated abilities. The inter-dimension ability correlation was 0.345 for 
the true abilities and -0.087 for the estimated abilities. 


Table 13 

Intercorrelation Matrix for True and Estimated 
Ability Parameters for Dataset 3 


Variable 


True 


Estimated 


True 9 

0 

Estimated 6 
0 


1 


1 


1.000 


0.345 

1.000 


0.231 

0.779 

1.000 


0.772 

0.229 

-0.087 

1.000 


The correlation between the d-pararaeter and the proportion-correct 
difficulty index was 0.995 (0.993 for the d-parameter estimates). The 
correlation of the point blserlal discrimination index with the true a- 
parameters was -0.134 for dimension 1 and 0.171 for dimension 2. When 
estimated a-values were used, these correlations were 0.261 and -0.079. 

Table 14 shows the Intercorrelation matrix for the true and estimated 
item parameters and the rotated factor loadings for dataset 3. As was the 
case previously, there was a strong relationship between the true and 
estimated item parameters and both sets of rotated loadings. The first four 
eigenvalues from the principal components analysis of these data were 7.92, 





























17 



2,08, 1,43, and 1,29, The first factor for these data was smaller than for 
the previous datasets, and the second factor was somewhat larger. 


Table 14 


Intercorrelation Matrix for True and Estimated Item 
Parameters and Factor Loadings for Dataset 3 


Item Parameter Factor Loadings 


Variable True Estimated Orthogonal Oblique 




d 

®1 ^2 

d 

®1 ^2 

I 

II 

I 

II 

True 

d 1 

.000 

-0.172 0.159 

0.999 

0.131 -0.207 

0.149 

-0.215 

0.164 

-0.204 


^1 


1.000 -0.987 

-0.164 

-0.929 0.921 

-0.946 

0.949 

-0.953 

0.955 


®2 


1.000 

0.153 

0.937 -0.915 

0.959 

-0.942 

0.961 

-0.952 

Estimated 

d" 



1.000 

0.127 -0.201 

0.145 

-0.200 

0.160 

-0.199 






1.000 -0.936 

0.988 

-0.949 

0.987 

-0.965 


a-, 




1.000 

-0.934 

0.989 

-0.957 

0.984 

Orthogonal 

I 





1.000 

-0.954 

0.997 

-0.972 


11 






1.000 

-0.972 

0.997 

Oblique 

I 







1.000 

-0.986 


II 








1.000 


The correlation of the point blserlal index and the factor loadings was 
0,299 and -0.050 for the varimax rotation, and 0.247 and -0,106 for the 
oblique rotation. The proportion-correct difficulty Index had correlations of 
0.147 and -0.214 with the varimax rotated loadings, and 0.162 and -0.203 with 
the oblique rotated loadings. The proportion-correct difficulty and point 
blserlal discrimination Indexes had a correlation of -0.101, 

Dataset 4 . Table 15 shows the Item parameter estimates, Item statistics, 
and factor loadings for dataset 4. These data were generated using test 1 and 
a group of examinees with an Inter-dlmension ability correlation of 0.00. The 
mean score on test 1 for this group was 24,61 and the standard deviation was 
6.52. The KR-20 reliability was 0.77, which Is somewhat lower than for 
dataset 3. The correlation between the factors, obtained from the oblique 
rotation, was 0.36, which Is slightly lower than was the case for dataset 3. 
































Item Parameter 


Item 


Factor Loadings 


Estimates 


Statistics 


Orthogonal 


Oblique 


p 

pbls 

.23 

0.20 

.90 

0.13 

.51 

0.23 

.85 

0.18 

.33 

0.21 

.24 

0.23 

.63 

0.24 

.41 

0.20 

.39 

0.20 

.17 

0.21 

.39 

0.24 

.55 

0.24 

.47 

0.26 

.18 

0.18 

.59 

0.20 

.75 

0.21 

.38 

0.22 

.59 

0.19 

.44 

0.21 

.63 

0.23 

.07 

0.15 

.46 

0.24 

.39 

0.23 

.48 

0.21 

.39 

0.23 

.75 

0.18 

.54 

0.22 

.78 

0.17 

.56 

0.25 

.29 

0.23 

.72 

0.21 

.57 

0.26 

.32 

0.28 

.62 

0.23 

.80 

0.18 


15 

0.33 

0.10 

04 

0.35 

-0.03 

11 

0.39 

0.04 

03 

0.47 

-0.06 

08 

0.40 

O.Oi 

12 

0.43 

0.04 

09 

0.46 

0.01 

04 

0.41 

-0.04 

02 

0.45 

-0.07 

08 

0.48 

-0.01 

07 

0.46 

-0.02 

05 

0.48 

-0.04 

25 

0.32 

0.20 

13 

0.33 

0.07 

06 

0.40 

-0.01 

12 

0.39 

0.04 

10 

0.40 

0.02 

03 

0.41 

-0.05 

01 

0.48 

-0.10 

12 

0.40 

0.04 

08 

0.40 

0.00 

17 

0.36 

0.11 

12 

0.40 

0.05 

09 

0.37 

0.02 

12 

0.40 

0.04 

43 

-0.00 

0.45 

39 

0.09 

0.39 

42 

-0.00 

0.44 

48 

0.06 

0.49 

47 

0.05 

0.49 

38 

0.09 

0.38 

39 

0.17 

0.38 

46 

0.16 

0.45 

44 

0.06 

0.46 


94 

69 













































































































1.000 

0.007 

0.0 


1.000 

0.8 




l.Oi 


The proportion-correct difficulty index had a correlation of 0.995 with 
the true d-parameter, and a correlation of 0.993 with the d-parameter 
estimates. The correlation between the point biserial index and the true a- 
parameters was -0.1A6 for dimension 1 and 0.183 for dimension 2. These values 
were 0.274 and -0.097 for the estimated a values. 

Table 18 shows the Intercorrelation matrix for the true and estimated 
item parameters and the rotated factor loadings for dataset 4. As has been 
the case all along, there was a strong relationship between the item 
parameters and estimates and both sets of rotated factor loadings. The first 














































four eigenvalues from the principal components analysis for dataset 4 were 
6.32, 2.74, 1.42, and 1.32. 


Table 18 


Intercorrelation Matrix for True and Estimated Item 
Parameters and Factor Loadings for Dataset 4 





Item Parameters 




Factor 

Loadings 

Variable 



True 


Estimated 

Orthogonal 

Oblique 



d 

®1 

®2 

d 

®1 

&2 

I 

II 

I II 

True 

d 1 

.000 

-0.172 

0.159 

0.999 

0.133 

-0.205 

0.\hl 

-0.217 

0.163 -0.203 


^1 


1.000 

-0.987 

-0.165 

-0.929 

0.920 

-0.945 

0.949 ■ 

-0.953 0.976 


®2 



1.000 

0.153 

0.937 

-0.914 

0.958 

-0.942 

0.962 -0.953 

Estimated 

d 




1.000 

0.129 

-0.199 

0.144 

-0.211 

0.159 -0.198 







1.000 

-0.937 

0.988 

-0.949 

0.987 -0.965 


^2 






1.000 

-0.939 

0.989 ■ 

-0.957 0.985 

Orthogonal 

r 







1.000 

-0.955 

0.998 -0.973 


II 








1.000 ■ 

-0.972 0.998 

Oblique 

I 









1.000 -0.986 


II 









1.000 


The point biserial index had correlations of 0,312 and -0.066 with the 
varlmax rotated factor loadings. The correlations between the point blserlals 
and the oblique rotated factor loadings were 0.262 and -0.121. The 
correlations between the proportion-correct index and the varlmax rotated 
factor loadings were 0.146 and -0.217, while correlations of 0.163 and -0.203 
were obtained between the point biserials and the oblique rotated factor 
loadings. The correlation between the proportion-correct difficulty index and 
the point blserlal discrimination index was -0.110, 

Test 2 Analyses 

Dataset 5 . Table 19 shows the item parameter estimates, item statistics, 
and rotated factor loadings for dataset 5. These data were generated using 
test 2 and a group of examinees having an inter-dimension ability correlation 
of 0.70. The mean score on test 2 for these examinees was 24.19, while the 
standard deviation was 9.00. The KR-20 reliability was 0.89. The correlation 
between factors, obtained from the oblique rotation, was 0.62. 




















































22 




Table 19 


Item Parameter Estimates, Item Statistics, and Factor 
Loadings for Dataset 5 


Item Parameter 
Estimates 


Factor Loadings 


Statistics Orthogonal 


Oblique 






























































Item Parameter 


Item 


Factor Loadings 


Item 


Estimates Statistics Orthogonal Oblique 



d 

^1 

®2 

P 

pbls 

I 

II 

I 

II 

39 

0.19 

0.89 

0.57 

0.54 

0.35 

0.45 

0.22 

0.50 

0.01 

40 

-0.49 

1.03 

0.60 

0.40 

0.38 

0.49 

0.23 

0.55 

-0.01 

41 

1.70 

1.16 

0.48 

0.79 

0.32 

0.50 

0.19 

0.57 

-0.06 

42 

-0.36 


0.07 

0.45 

0.39 

0.57 

0.15 

0.67 

-0.14 

43 

-0.91 

0.94 

0.74 

0.32 

0.37 

0.47 

0.28 

0.50 

0.06 

44 

0.32 

0.99 

0.56 

0.57 

0.37 

0.44 

0.26 

0.48 

0.05 

45 

0.91 

0.98 

0.54 

0.68 

0.35 

0.49 

0.20 

0.55 

-0.04 

46 

0.94 

0.51 

1.16 

0.68 

0.33 

0.20 

0.50 

0.11 

0.46 

47 

-1.86 

0.77 

0.85 

0.17 

0.31 

0.29 

0.42 

0.24 

0.32 


-1.60 

0.93 

0.49 

0.20 

0.29 

0.44 

0.19 

0.49 

-0.03 

49 


0.77 

0.84 

0.38 

0.37 

0.34 

0.39 

0.31 

0.26 

50 

-0.84 

0.76 

0.99 

0.34 

0.37 

0.32 

0.43 

0.28 

0.31 


Table 20 shows the 

intercorrelation matrix obtained for the true 

and 

estimated Item parameters for dataset 5 

. The 

correlation between the 

true and 

estimated d-parameters 

was 0.997 

The 

correlations of 

the true 

and estimated 

a-parameters for 

these 

data were 

0.198 

for dimension 1 

and 0.167 for dimension 

2. 

These values 

are In 

marked contrast 

to the 

high values obtained for test 

1. 

Here, there 

is no significant correlation 

between 

the true 

and estimated 

Item discrimination parameters. 








Table 20 

Intercorrelation Matrix for True and Estimated 

Item Parameters for Dataset 5 



True 





Variable _ 








d 

®1 

32 

d 

®1 

32 

True d 

1.000 

-0.048 

0.076 

0.997 

0.029 

0.088 

®1 


1.000 

-0.985 

-0.036 

0.198 

-0.176 

®2 



1.000 

0.067 

-0.159 

0.167 

Estimated d 




1.000 

0.012 

0.096 

®l 





1.000 

-0.837 

®2 






1.000 






































Table 21 shows the intercorrelation matrix for the true and estimated 
ability parameters for dataset 5. As can be seen, the true ability values on 
the two dimensions had about equal correlations with the dimension 1 estimated 
abilities (0.592 for dimension 1, 0.597 for dimension 2). The correlations of 
the true abilities with the dimension 2 ability estimates were also almost 
equal (0.468 for dimension 1, 0.507 for dimension 2) and both were lower than 
the correlations with the dimension 1 estimates. The inter-dimension ability 
correlation was 0.687 for the true values and -0.203 for the estimated values. 


Table 21 

Intercorrelation Matrix for True and Estimated 
Ability Parameters for Dataset 5 




1.000 


0.687 

1.000 


Estimated 6, 


0.592 

0.597 

1.000 


The correlation between the proportion-correct index and the d-parameter 
was 0.994 (0.992 with the d-parameter estimates). The correlation between the 
point blserlal index and the true a-parameters was 0.186 for dimension 1 and 
-0.142 for dimension 2. Wlien the a-parameter estimates were used, these 
correlations were 0.474 and -0.252. 

Table 22 shows the intercorrelation matrix for the true and estimated 
item parameters and the rotated factor loadings for dataset 5. Interestingly, 
the factor loadings (both rotations) are strongly related to the estimated a- 
parameters, but not to the true a-parameters. The first four eigenvalues from 
the principal components analysis of dataset 5 were 12.32, 1.23, 1.21, and 
1.18. The first factor for this dataset is larger than for the corresponding 
dataset from the test 1 analyses (dataset 1). 








































Variable 


Item Parameters 


Estimated 


Factor Loadings 


Orthogonal Oblique 

I II I II 


True 

d 1.000 

-0.048 

0. 

.076 

0. 

.997 

0. 

.029 

0. 

.088 

0. 

.113 

-0.061 

0.105 

-0.079 


®1 

1.000 

-0, 

.985 

-0, 

.036 

0. 

.198 

-0. 

.176 

0. 

.244 

-0.195 

0.238 

-0.215 


®2 


1, 

.000 

0. 

.067 

-0. 

.159 

0. 

.167 

-0. 

.215 

0.181 

-0.211 

0.196 

Estimated 

4 




1, 

.000 

0, 

.012 

0, 

.096 

0, 

.116 

-0.067 

0.108 

-0.085 


^1 






1. 

.000 

-0, 

.837 

0, 

.860 

-0.760 

0.851 

-0.806 


®2 








1, 

.000 

-0, 

.869 

0.905 

-0.885 

0.908 

Orthogonal 

I 










1. 

.000 

-0.925 

0.998 

-0.966 


II 












1.000 

-0.949 

0.992 

Oblique 

I 













1.000 

-0.982 


The point blserlal index had correlations of 0.509 (dimension 1) and 
-0.186 (dimension 2) with the varlmax rotated factor loadings» and 0.456 
(dimension 1) and -0.298 (dimension 2) with the oblique rotated factor 
loadings. The proportion-correct index had correlations of 0.085 and -0.042 
with the varlmax rotated loadings, and 0.078 and -0.057 with the oblique 
rotated factor loadings. The correlation between the proportion-correct and 
point biserial indexes was 0.120. 

Dataset 6 . Table 23 shows the item parameter estimates, item statistics, 
and rotated factor loadings for dataset 6. These data were generated using 
test 2 and a group of examinees with an inter-dimension ability correlation of 
0.50. The mean score on test 2 for this group was 24.01, and the standard 
deviation was 8.54. The KR-20 reliability was 0.87. The correlation between 
factors, obtained from the oblique rotation, was -0.19, 





















































Table 23 

Item Parameter Estimates, Item Statistics, and Factor 
Loadings for Dataset 6 


Item Parameter 

Item 

Factor 

Loadings 

Estimates 

Statistics 

Orthogonal 

Oblique 


1 

-1.39 

0 

2 

2.22 

0 

3 

-0.12 

0 

4 

2.78 

0 

5 

-0.86 

0 

6 

-1.36 

0 

7 

0.44 

0 

8 

-0.44 

0 

9 

-0.49 

0 

10 

-1.76 

0 

11 

-0.58 

0 

12 

0.29 

0 

13 

-0.21 

0 

14 

-2.74 

2 

15 

0.40 

0 

16 

1.09 

0 

17 

-0.58 

1 

18 

0.68 

0 

19 

-0.38 

0 

20 

0.65 

0 

21 

-3.04 

1 

22 

-0.24 

0 

23 

-0.69 

0 

24 

-0.21 

1 

25 

-0.73 

0 

26 

1.19 

0 

27 

0.05 

1 

28 

1.54 

1 

29 

0.33 

0 

30 

-1.02 

0 

31 

0.92 

0 

32 

0.28 

0 

33 

-1.02 

0 

34 

0.49 

1 

35 

1.45 

0 

36 

1.06 

1 

37 

-1.21 

1 

38 

0.29 

0 

39 

0.13 

0 


Dll] 


p 

pbls 

.23 

0.26 

.87 

0.26 

.48 

0.34 

.84 

0.27 

.33 

0.31 

.24 

0.29 

.59 

0.36 

.40 

0.30 

.40 

0.32 

.18 

0.26 

.38 

0.36 

.56 

0.38 

.46 

0.33 

.17 

0.28 

.59 

0.33 

.71 

0.32 

.38 

0.36 

.58 

0.35 

.42 

0.33 

.64 

0.32 

.07 

0.21 

.45 

0.36 

.36 

0.33 

.46 

0.36 

.35 

0.31 

.73 

0.31 

.51 

0.35 

.78 

0.29 

.57 

0.36 

.30 

0.33 

.69 

0.30 

.56 

0.34 

.29 

0.30 

.60 

0.35 

.78 

0.29 

.70 

0.32 

.28 

0.34 

.56 

0.34 

.53 

0.36 


-0.05 
-0.13 
-0.18 
-0.42 
0.10 
0.07 
11 
04 

15 
04 
06 

16 
01 
42 
12 
22 
05 
45 
01 
00 
02 
02 
02 
17 
05 
02 
17 
10 
12 
13 
00 
05 
08 
17 
07 
15 
22 
10 
13 




















































































> 


Table 24 shows the Intercorrelation matrix for the true and estimated 
item parameters for dataset 6. The correlation between the true and estimated 
d-parameters was 0.986, which is somewhat lower than was the case with dataset 
5. The true a-parameters for dimension 1 had a correlation of 0.109 with the 
dimension 2 estimated a-values, while the dimension 2 true a-values had a 
correlation of 0.056 with the dimension 1 estimated a-values. Again, there is 
no significant correlation between the true and estimated a-parameters. 


Table 24 

Intercorrelation Matrix for True and Estimated 
Item Parameters for Dataset 6 


d 

l.OOO 

-0.048 

0.076 

0.989 

-0.379 



1.000 

-0.985 

-0.021 

-0.065 

®2 



1.000 

0.054 

0.056 

d 




1.000 

-0.469 






1.000 

a2 



































































Table 25 shows the intercorrelation matrix for the true and estimated 
abilities for dataset 6. As can be seen, the true abilities had slightly 
higher correlations with the dimension 2 ability estimates than with the 
dimension 1 ability estimates. Each true ability parameter had about equal 
correlations with the two sets of ability parameter estimates. The inter- 
dimension ability correlations was 0.493 for the true abilities and -0.378 for 
the estimated abilities. 


Table 25 

Intercorrelation Matrix for True and Estimated 
Ability Parameters for Dataset 6 



Estimated 6^ 


1.000 


0.493 

l.OOO 


0.424 

0.446 

1.000 


There was a correlation of 0,993 between the proportion-correct 
difficulty index and the true d-parameter. The correlation was 0.983 for the 
estimated d-parameter. The point blserlal discrimination index had 
correlations of 0.158 and -0.096 with the true a-parameters and correlations 
of 0.032 and 0.137 with the estimated a-parameters. 

Table 26 shows the intercorrelation matrix for the true and estimated 
item parameters and the rotated factor loadings for dataset 6. As was the 
case with dataset 5, there was a strong relationship between the estimated a- 
parameters and both sets of rotated factor loadings, but no significant 
relationship between the true a-parameters and the factor loadings. The first 
four eigenvalues from the principal components analysis of dataset 6 were 
11.18, 1.28, 1.25, and 1.22. There is a large first factor, and the second 
factor is almost nonexistent. 




w*. .S .%• •.* •/ • 

a-•*». i", 






































Item Parameters 


Factor Loadings 


Variable 


Estimated 


Orthogonal Oblique 




d 


^2 ^ 

®2 

I 

II I 

II 

True 

d 1 

.000 

-0.048 

0.076 0.989 -0.379 

0.335 

-0.370 

0.382 -0.306 

-0.382 




1.000 

-0.985 -0.021 -0.065 

0.109 

-0.008 

0.045 0.036 

-0.030 





1.000 0.054 0.056 

-0.076 

-0.003 

-0.012 -0.021 

0.005 

Estimated 

d^ 



1.000 -0.469 

0.420 

-0.439 

0.449 -0.366 

-0.450 





1.000 

-0.840 

0.953 

-0.886 0.897 

0.928 






1.000 

-0.852 

0.918 -0.659 

-0.902 

Orthogonal 

I 





1.000 

-0.944 0.926 

0.982 


II 






1.000 -0.750 

-0.990 

Oblique 

I 






1.000 

0.837 


The correlations of the point biserlals with the varlmax rotated loadings 
were 0.074 and 0.194. With the oblique rotated loadings, the correlations 
were 0.371 and -0.079. The proportion-correct difficulty Index had 
correlations of -0.367 and 0.379 with the varlmax rotated loadings, and 
correlations of -0.303 and -0.379 with the oblique rotated loadings. The 
correlation of the proportion-correct difficulty and point biserial 
discrimination Indexes was 0.094. 


Dataset 7 . Table 27 shows the Item parameter estimates, item statistics, 
and rotated factor loadings for dataset 7. These data were generated using 
test 2 and a group of examinees having an inter-dimension ability correlation 
of 0.35. The mean score on test 2 for this group was 24.07, and the standard 
deviation was 8.20. The KR-20 reliability was 0.86. The correlation between 
factors, obtained from the oblique rotation, was 0.23. 







































30 


' 




» Tj- V— V w -^-j^ T- -» ■:r»^ 


Table 27 

Item Parameter Estimates, Item Statistics, and Factor 
Loadings for Dataset 7 


Item Parameter 


Estimates 




Factor Loadings 


Statistics Orthogonal 


.22 

0.25 

0.18 

.88 

0.24 

0.25 

.47 

0.34 

0.36 

.84 

0.27 

0.11 

.32 

0.30 

0.34 

.24 

0.27 

0.34 

.59 

0.35 

0.34 

.41 

0.28 

0.32 

.40 

0.29 

0.27 

.18 

0.25 

0.29 

.39 

0.34 

0.38 

.56 

0.36 

0.28 

.46 

0.31 

0.37 

.17 

0.26 

0.57 

.59 

0.32 

0.30 

.72 

0.30 

0.16 

.38 

0.34 

0.37 

.59 

0.34 

0.10 

.42 

0.31 

0.28 

.64 

0.31 

0.32 

.07 

0.20 

0.17 

.45 

0.34 

0.38 

.36 

0.32 

0.33 

.46 

0.35 

0.38 

.35 

0.29 

0.38 

.74 

0.29 

0.45 

.52 

0.34 

0.54 

.79 

0.27 

0.36 

.57 

0.34 

0.26 

.30 

0.30 

0.28 

.70 

0.29 

0.36 

.56 

0.32 

0.46 

.29 

0.28 

0.34 

.59 

0.33 

0.48 

.78 

0.26 

0.32 

.71 

0.30 

0.45 

.28 

0.33 

0.55 

.56 

0.32 

0.28 

.53 

0.34 

0.28 

.38 

0.31 

0.33 

.82 

0.26 

0.02 

.44 

0.33 

0.32 


Oblique 



V W V > ^ 

























































Item Parameter 


Item 


Factor Loadings 


Estimates 


Statistics Orthogonal 


Oblique 


43 

-0.82 

0.54 

1.10 

0.34 

0.32 

0.16 

0.49 

0.36 

44 

0.34 

0.85 

0.96 

0.57 

0.36 

0.36 

0.36 

0.48 

45 

0.97 

0.96 

0.73 

0.69 

0.31 

0.41 

0.25 

0.49 

46 

0.81 

0.56 

0.86 

0.67 

0.29 

0.21 

0.39 

0.36 

47 

-1.81 

0.80 

0.95 

0.18 

0.29 

0.35 

0.32 

0.46 

48 

-1.98 

1.39 

0.51 

0.18 

0.28 

0.56 

0.09 

0.56 

49 

-0.70 

0.74 

1.13 

0.36 

0.36 

0.30 

0.42 

0.46 

'50 

-0.78 

0.88 

0.86 

0.34 

0.34 

0.40 

0.30 

0.50 


Table 28 contains the Intercorrelatlon matrix for the true and estimated 
Item parameters for dataset 7. The correlation between the true and estimated 
d-parameters was 0.988, which Is about the same as was obtained for dataset 
6. The true a-parameters for dimension 1 had a correlation of 0.049 with the 
dimension 2 estimated a-values, and the dimension 2 true a-parameters had a 
correlation of 0.061 with the dimension I estimated a-values. 


Table 28 


Intercorrelatlon Matrix for True and Estimated 
Item Parameters for Dataset 7 


Table 29 contains the Intercorrelatlon matrix for the true and estimated 
ability parameters for dataset 7. The values In this table follow a pattern 
much like what was found for dataset 6. Each true ability had about equal 
correlations with the two sets of estimates, and both had slightly higher 
correlations with the dimension 2 estimates than with the dimension 1 







































estimates. The inter-dimension ability correlation was 0.334 for the true 
abilities and -0.380 for the estimates. 


Table 29 


Intercorrelation Matrix for True and Estimated 
Ability Parameters for Dataset 7 


Variable 


True 

Estimated 



01 02 

®1 

®2 

True 


1.000 0.334 

0.381 

0.430 



1.000 

0.411 

0.450 

Estimated 

®2 


1.000 

-0.380 

1.000 


There was a correlation of 0.993 between the proportion-correct 
difficulty index and the true d-parameter. The correlation was 0.983 for the 
estimated d-parameter. The point biserlal discrimination index had 
correlations of 0.170 and -0.107 with the true a-parameters and correlations 
of 0.010 and 0.051 with the estimated a-parameters. 

Table 30 shows the Intercorrelation matrix for the true and estimated 
item parameters and the rotated factor loadings for dataset 7. As was the 
case with datasets 5 and 6, there was a strong relationship between the 
estimated a-parameters and both sets of rotated factor loadings» but no 
significant relationship between the true a-parameters and the factor 
loadings. The first four eigenvalues from the principal components analysis 
of dataset 7 were 10.33, 1.31, 1.26, and 1.23. There is a large first factor, 
and the second factor is almost nonexistent. 











Item Parameters 

Factor Loadings 

Variable 

True Estimated 

Orthogonal Oblique 


True 

d 1.000 

-0.048 0.076 

0.987 

-0.323 

0.364 

-0.228 

0.248 

-0.192 

0.243 


®1 

1.000 -0.985 

-0.043 

-0.069 

0.049 

0.006 

0.040 

0.043 

0.021 


^2 

1.000 

0.075 

0.061 

-0.018 

0.003 

-0.027 

-0.018 

-0.017 

Estimated 

d 


1.000 

-0.433 

0.468 

-0.322 

0.339 

-0.279 

0.337 


®1 



1.000 

-0.837 

0.924 

-0.872 

0.885 

-0.908 


4 




1.000 

-0.832 

0.910 

-0.695 

0.890 

Orthogonal 

I 





1.000 

-0.940 

0.961 

-0.980 


II 






1.000 

-0.808 

0.989 

Oblique 

I 







1.000 

-0.887 


The correlations of the point biserials with the varimax rotated loadings 
were 0.2A6 with 0.030. With the oblique rotated loadings, the correlations 
were 0.448 and -0.089. The proportion-correct difficulty index had 
correlations of -0.241 and 0.262 with the varimax rotated loadings, and 
correlations of -0.204 and 0.257 with the oblique rotated loadings. The 
correlation of the proportion-correct difficulty and point biserial 
discrimination indexes was 0.097. 


Dataset 8 . Table 31 shows the item parameter estimates, item statistics, 
and rotated factor loadings for dataset 8. These data were generated using 
test 2 and a group having an inter-dimension ability correlation of 0.00. The 
mean score on test 2 for this group was 24.18, and the standard deviation was 
7.32. The KR-20 reliability was 0.82. The correlation between factors, 
obtained from the oblique rotation, was -0.57. 
























































































































b 


u 


Item Parameter 


Estimates 


Factor Loadings 


Statistics Orthogonal 


Oblique 


26 

1.24 

0.96 

0.56 

0.75 

0.24 

27 

0.10 

1.18 

0.48 

0.52 

0.28 

28 

1.54 

0.67 

0.72 

0.80 

0.22 

29 

0.34 

0.76 

0.95 

0.57 

0.31 

30 

-1.02 

0.71 

0.81 

0.29 

0.27 

31 

0.98 

0.70 

0.74 

0.71 

0.25 

32 

0.30 

0.97 

0.54 

0.56 

0.27 

33 

-0.96 

0.73 

0.54 

0.30 

0.24 

34 

0.49 

1.06 

0.66 

0.60 

0.30 

35 

1.47 

0.86 

0.56 

0.79 

0.23 

36 

1.00 

0.70 

0.73 

0.71 

0.25 

37 

-1.68 

2.00 

0.17 

0.27 

0.28 

38 

0.33 

0.53 

0.99 

0.57 

0.28 

39 

0.17 

0.63 

0.95 

0.54 

0.30 

40 

-0.56 

0.68 

0.71 

0.38 

0.26 

41 

2.66 

0.0 

2.00 

0.82 

0.22 

42 

-0.28 

0.77 

0.63 

0.44 

0.27 

43 

-0.81 

0.68 

0.88 

0.33 

0.28 


40 

0.14 

0.43 

0.02 

48 

0.08 

0.54 

0.11 

23 

0.31 

0.17 

-0.25 

25 

0.41 

0.16 

-0.37 

29 

0.30 

0.25 

-0.22 

31 

0.23 

0.30 

-0.13 

43 

0.12 

0.47 

0.05 

36 

0.15 

0.38 

-0.02 

43 

0.19 

0.45 

-0.03 

34 

0.18 

0.34 

-0.06 

33 

0.23 

0.32 

-0.12 

54 

0.06 

0.62 

0.16 

21 

0.37 

0.13 

-0.34 

20 

0.42 

0.11 

-0.40 

28 

0.28 

0.25 

-0.20 

06 

0.60 

-0.25 

-0.71 

31 

0.25 

0.29 

-0.16 

20 

0.41 

0.11 

-0.38 

35 

0.29 

0.32 

-0.19 

37 

0.19 

0.37 

-0.06 

06 

0.47 

-0.07 

-0.51 

47 

0.28 

0.29 

-0.19 

47 

0.06 

0.54 

0.13 

27 

0.41 

0.20 

-0.35 

42 

0.19 

0.44 

-0.03 


Table 32 contains the Intercorrelatlon matrix for the true and estimated 
Item parameters for dataset 8. The correlation between the true and estimated 
d-parameters was 0.989. The true and estimated a-values for dimension 1 had a 
correlation of 0.11, while for dimension 2 the correlation was 0.037. 




























































1.000 


“1 

^2 

Estimated d 


-0.048 

1.000 


0.076 

-0.985 

1.000 


0.989 

-0.074 

0.104 

1.000 


-0.331 

0.011 

-0.017 

-0.428 

1.000 


0.370 

-0.005 

0.037 

0.464 

-0.784 

1.000 


Table 33 shows the Intercorrelation matrix for the true and estimated 
ability parameters for dataset 8. The same pattern is present as was found 
for the other test 2 datasets. Both sets of true abilities had about equal 
correlations with the two sets of estimates. The inter-dimension ability 
correlation was -0.036 for the true values and -0.466 for the estimates. 


Table 33 

Intercorrelation Matrix for True and Estimated 
Ability Parameters for Dataset 8 





1.000 

-0.036 

0.288 

0.309 



1.000 

0.343 

0.344 

e? 



1.000 

-0.466 

«2 




1.000 


The correlation between the true d-parameter and the proportion-correct 
index was 0.993. The correlation between the proportion-correct index and the 
estimated d-values was 0.986. The correlations of the point biserlal index 
and the a-values were 0.140 and -0.083 for the true values, and 0.203 and 
0.084 for the estimates. 

Table 34 shows the Intercorrelation matrix for the true and estimated 
item parameters and the two sets of rotated factor loadings obtained for 
dataset 8. As has been the pattern with the test 2 datasets, there is a 
strong relationship between the estimated a-values and both sets of loadings, 
but no correlation between the true a-values and the factor loadings. The 


».V Vk. .V . ..k'k A. k- 








































first four eigenvalues from the principal components analysis of dataset 8 
were 8.19, 1.39, 1.30, and 1.29. 


Table 34 


Intercorrelation Matrix for True and Estimated Item 
Parameters and Factor Loadings for Dataset 8 


Variable 



Item Parameters 



Factor Loading 

s 



True 

Estimated 

Orthogonal Oblique 


d 

ai ^2 

d aj 

32 

I 

II I 

II 

True 

d 1 

.000 

-0.048 0.076 

0.989 -0.331 

0.370 

-0.308 

0.308 -0.312 

-0.312 




1.000 -0.985 

-0.074 0.011 

-0.005 

-0.035 

0.072 -0.043 

-0.064 


ap 


1.000 

0.104 -0.017 

0.037 

0.037 

-0.056 0.042 

0.052 

Estimated 

d 



1.000 -0.428 

0.464 

-0.390 

0.380 -0.392 

-0.388 





1.000 

-0.784 

0.887 

-0.786 0.876 

0.823 


a2 




1.000 

-0.818 

0.888 -0.844 

-0.882 

Orthogonal 

I 





1.000 

-0.929 0.997 

0.960 


II 






1.000 -0.956 

-0.995 

Oblique 

I 






1.000 

0.980 


II 







1.000 


The correlations of the point blserial Index with the rotated factor 
loadings were 0.243 and 0.074 for the varlmax rotation, and 0.175 and 0.006 
for the oblique rotation. For the proportion-correct difficulty index the 
correlations were -0.324 and 0.330 with the varlmax rotated loadings, and 
-0.330 and -0.333 with the oblique rotated factor loadings. The correlation 
between the point biserlals and proportion-correct values was 0.073. 

Discussion 


The purpose of this study was to examine the effects of correlated 
abilities on observed test characteristics, and to explore the implications of 
correlated abilities for multidimensional item response theory, or MIRT, 
analysis when a model is used that does not explicitly account for such a 
correlation. The approach taken was to generate simulation data with known 
true parameters, using varying levels of correlation between abilities, and to 
analyze the data using a number of different test analysis procedures. The 
procedures selected were item analysis, principal component analysis, and MIRT 
analysis. In addition, correlational analyses were performed to explore the 
relationship of obtained statistics and parameter estimates to the true 
parameters, as well as the Interrelationships among the item parameter 
estimates and traditional item statistics. All of these analyses were 
performed for two different tests. One test was comprised of two relatively 
Independent dimensions (each item discriminating on only one of the 
dimensions), while the other test was comprised of two correlated dimensions 
(each item discriminating at least moderately on both dimensions). The 
analyses of these two tests will be discussed separately, and then an attempt 
will be made to Integrate the results of the two sets of analyses. 




















38 


Test 1 Analyses 

In test 1, an attempt was made to use two relatively unidimensional 
subsets of items. One subset of items discriminated fairly highly on the 
first dimension, and very poorly on the second. The second subset of items 
discriminated fairly highly on the second dimension and relatively poorly on 
the first. Thus, the test had two relatively independent factors. In an 
attempt to evaluate the effects of correlated abilities for such a test, the 
three types of test analysis procedures were applied, and the results 
analyzed. The results for each type of test analysis procedure will now be 
discussed separately. 

Item Analysis Results . The one clear pattern which emerged from the item 
analyses performed on these data was the decline of the test KR-20 reliability 
with the decline of the correlation between ability dimensions. This trend is 
summarized in Table 35, which shows a drop in reliability from 0.86 to 0.77 
when the inter-dimension ability correlation dropped from 0.70 to 0.00. This 
is an indication that an Increased correlation between ability dimensions 
results in more common variance, which in turn yields a higher KR-20. The 
fact that the items were constructed to have a relatively low inter- 
dimensional correlation may have somewhat mitigated this effect, but it did 
not eliminate it. 


Table 35 


Relationship Between Ability Correlation 
and Test Reliability for Test 1 


Dataset 

Ability Correlation 

KR-20 

1 

0.70 

0.86 

2 

0.50 

0.84 

3 

0.35 

0.82 

4 

0.00 

0.77 


Principal Component Analysis Results. One pattern evident in the factor 
analysis results was the decline in factor correlation with the decline in 
ability correlation. This pattern was similar to that found for the KR-20 
analyses, and is also indicative of the increased multidimensionality of the 
test data (decreased size of the common component). The pattern Is 
illustrated in Table 36. 































A similar sort of pattern was evident In the eigenvalues resulting from 
the principal components analyses. The first four eigenvalues for each set of 
response data are shown In Table 37. As can be seen, as the ability 
correlation decreased, so did the size of the first eigenvalue. At the same 
time, the size of the second eigenvalue Increased. 

Table 37 


Relationship Between Ability Correlation 
and Eigenvalues for Test 1 


Dataset 

Ability 


Eigenvalues 


Correlation 


E 2 

E 3 

E4 

1 

0.70 

10.01 

1.50 

1.31 

1.27 

2 

0.50 

9.09 

1.79 

1.30 

1.28 

3 

0.35 

7.92 

2.08 

1.43 

1.29 

4 

0.00 

6.32 

2.74 

1.42 

1.32 


Another trend found in the results of these analyses was the tendency 
toward an increase in the correlations between the true item discriminations 
and both the orthogonal and oblique rotated factor loadings as the ability 
correlation decreased. This tendency is shown in Table 38. Bear in mind that 
the correlations shown are not always matched on dimensions. That is, in some 
cases the dimension 1 a-value correlation with the factor II loadings is 
shown. 


























































Ability 


Orthogonal 


Oblique 


Dataset 


elation 

I 

II 

I 

.70 

0.86 

0.86 

0.87 

.50 

0.94 

0.90 

0.93 

.35 

0.96 

0.95 

0.96 

.00 

0.96 

0.95 

0.96 


MIRT Analysis Results . As the correlation between ability dimensions 
decreased, there was a slight increase in the correlations between the true 
and estimated item parameters. That is to say, the estimation program was 
better able to recover the true parameters when the ability dimensions were 
leas correlated. This trend is shown in Table 39. Note that the dimensions 
did not always match. That is, some of the correlations reported in Table 39 
for the discrimination values are actually correlations between the true a- 
values on one dimension and the estimated a-values for the other dimension. 


Table 39 

Relationship Between Ability Correlation 
and the Correlation Between True and Estimated 
Item Parameters for Test 1 


Ability 


Item Parameter 


Dataset 

Correlation 

d 


1 

0.70 

0.996 

0.73 

2 

0.50 

0.998 

0.87 

3 

0.35 

0.999 

0.92 

4 

0.00 

0.999 

0.92 



Another interesting result from the MIRT analyses Involved the 
correlation between the discrimination parameters. For test I the correlation 
between the true dimension 1 and dimension 2 a-values was -0.987. Table 40 
shows the a-value correlations for the estimated a-values for the four 
datasets for test 1. Note that the correlation is well below (in absolute 
value) the correlation for the true values for dataset 1, but as the ability 
correlation decreases the obtained a-value correlation more nearly approaches 
the true value. 





























Dataset 


Ability Correlation 


a-Value Correlation 


1 

0.70 

-0.841 

2 

0.50 

-0.865 

3 

0.35 

-0.936 

4 

0.00 

-0.937 


Table 41 summarizes the correlations between the true and estimated 
ability parameters for test 1. As can be seen, the correlations Increase as 
the inter-dimension ability correlation decreases. The estimation program was 
better able to recover the true examinee abilities when the ability dimensions 
were less correlated. This is consistent with the results obtained for the 
item parameters. 

Table 41 


Relationship Between Ability Correlation 
and the Correlations Between the True and Estimated 
Ability Parameters for Test 1 


Dataset 

Ability 

Ability 

Parameter 

Correlation 


®2 

1 

0.70 

0.70 

0.67 

2 

0.50 

0.74 

0.72 

3 

0.35 

0.78 

0.77 

4 

0.00 

0.81 



Table 42 shows the correlations between the two ability dimensions for 
the ability parameter estimates for the four datasets for test 1. Also shown 
are the actual true ability correlations obtained for the four datasets (as 
opposed to the true correlations for the populations from which examinees were 
selected). As can be seen, the sample correlations for the true abilities 
were quite close to the true population values. In every case, however, the 
correlation for the ability parameter estimates is very near 0.0. Regardless 
of the true correlation between ability dimensions, the ability parameter 
estimate dimensions are forced to be uncorrelated. 

































42 


Table 42 

Relationship Between Ability Correlation 
and Inter-Dimension Ability Estimate 
Correlation for Test 1 


Dataset 

Ability Correlation 

(Population) 

Ability Correlation 

(Sample) 

Ability Estimate 

Correlation 

1 

0.70 

0.68 

-0.14 

2 

0.50 

0.49 

-0.15 

3 

0.35 

0.35 

-0.09 

4 

0.00 

0.01 

-0.05 


Test 2 Analyses 

In test 2 items were selected to have at least moderately high 
discriminations on both dimensions. Thus, the test was constructed to have 
somewhat correlated dimensions. The same analyses performed on test 1 were 
then run on test 2. Again, the results for each type of test analysis 
procedure will be discussed separately. 

Item Analysis Results. The only pattern discernible among the item 
analysis results for the test 2 datasets was a slight decline in KR-20 
reliability with a decrease in the correlation between ability dimensions. 
This trend can be seen in the data shown in Table 43. The trend is less 
dramatic than was the case with the test 1 data, however. In the test 1 
datasets the principal component was due primarily to the ability dimension 
correlation, while in the test 2 data the principal component was at least 
partially due to the nature of the items. Thus, a decline in ability 
correlation did not have as great an impact on its size. 


Table 43 

Relationship Between Ability Correlation 
and Test Reliability for Test 2 


Dataset 

Ability Correlation 

KR-20 

5 

0.70 

0.89 

6 

0.50 

0.87 

7 

0.35 

0.86 

8 

0.00 

0.82 


Principal Component Analysis Results . The pattern of factor correlations 
was rather confusing for the test 2 datasets. These values are shown In Table 
44. 











Dataset 


Ability Correlation 


Factor Correlation 



As can be seen, there is no systematic relationship between ability 
correlation and factor correlation. To be consistent with the previous 
results, the factor correlation should have declined slightly with the 
decrease in ability correlation. The results for datasets 5 and 8 are 
consistent with this, but the results for datasets 6 and 7 are quite 
Inconsistent with this. As yet, no satisfactory explanation for this 
phenomenon has been determined. 

Table 45 shows the trend in eigenvalues as ability correlation decreased 
for the test 2 data. As can be seen, these results are much more consistent 
with the item analysis results. There was a slight decrease in the size of 
the first eigenvalue with the decrease in ability correlation. The decrease 
is more marked from dataset 7 to 8 than between the other tests, but so was 
the decline in KR-20. There was a negligible increase in the size of the 
second eigenvalue as the ability correlation decreased. 


Table 45 

Relationship Between Ability Correlation 
and Eigenvalues for Test 2 


Dataset 


Ability 


Correlation 


Eigenvalues 



.70 

12.32 

1.23 

1.21 

.50 

11.18 

1.28 

1.25 

.35 

10.33 

1.31 

1.26 

.00 

8.19 

1.39 

1.30 


A pattern found for the test 2 analyses which is in marked contrast to 
the results for test 1 Involves the correlations between the true a-values and 
the factor loadings. For test 1 there was a tendency for an Increase in 
correlations between a-values and factor loadings as the ability correlation 
decreased. All of the a-value-factor loading correlations were relatively 








































high, though. For the test 2 data, the true a-value-factor loading 
correlations were all around 0.0, regardless of the ability correlation. 


MIRT Analysis Results . As the correlation between ability dimensions 
decreased, the correlation between the true and estimated d-parameters 
decreased slightly, except for dataset 8, for which the correlation was the 
same as for dataset 6. The correlation between the true and estimated 
a-parameters was essentially 0.0 for all four test 2 datasets. 


As the correlation between ability dimensions decreased, the correlations 
between the two a-parameter estimates for the two dimensions diverged from the 
correlation between the true a-parameters. This trend is shown In Table 46. 
The true a-parameter correlation for test 2 was -0.985. 


Table 46 


Relationship Between Ability Correlation 
and Inter-Dimension a-Value Correlation 


Dataset 


Ability Correlation 


a-Value Correlation 


-0.837 

-0.840 

-0.836 

-0.784 


Table 47 shows the relationship between true ability correlation and the 
correlation between the true and estimated ability parameters. As can be 
seen, as the true ability correlation decreased, so did the correlations 
between the true and estimated ability parameters. 


Table 47 


Relationship Between Ability Correlation 
and the Correlations Between the True and Estimated 
Ability Parameters for Test 2 



-■••• 

5 

0 

.70 

0.592 

.> 6 

0 

.50 

0.424 


0 

.35 

0.381 

8 

r-TT— 

0 

.00 

0.288 



Table 48 shows a very interesting pattern involving the inter-dimension 
ability estimate correlations for test 2. As can be seen, the sample true 
ability dimension correlations are quite close to the target population 




















































valnes. For the ability estimates, however, the correlation differs 
substantially from the true ability correlation. In every case the 
correlation is nej^ative, and it becomes more negative as the true value 
approached 0.0. 

Table 48 


Relationship Between Ability Correlation 
and Inter-Dimension Ability Estimate 
Correlation for Test 2 


Dataset 

Ability Correlation 

(Population) 

Ability Correlation 

(Sample) 

Ability Estimate 

Correlation 

5 

0.70 

0.69 

-0.20 

6 

0.50 

0.49 

-0.38 

7 

0.35 

0.33 

-0.38 

8 

0.00 

-0.04 

-0.47 


Overall Results 

It is quite clear that it is not enough to talk about the 'dimensionality 
of a test', or about whether the 'dimensions of a test' are correlated. There 
are two distinct concepts Involved, and they play quite different roles in 
determining the latent structure of response data. The first concept is 
latent item structure, and the second is latent ability structure. 

The latent structure of a test item refers to the number and 
interrelationships of the dimensions required for performance on the item. In 
this research, two types of latent item structure we^'e employed. For test 1, 
each item required basically only one dimension. The first half of the items 
required the first dimension, while the second half required the second 
dimension. Thus, there were two dimensions underlying the test, each of which 
operated relatively independently of the other. 

For test 2, each item required two dimensions. For some items the first 
dimension was more dominant than the second, while for the remaining items the 
second dimension was dominant. Since all items required both dimensions, the 
two dimensions did not operate as Independently as for test 1. 

The latent ability structure of an examinee refers to the number and 
interrelationships of the dimensions underlying the examinee's responses. In 
this study the latent ability space was always two-dimensional, but the 
correlation of the two examinee latent ability dimensions varied. 

It is the interaction of these two concepts which determines the latent 
structure of response data. When the item dimensions operate relatively 
Independently, the dimensionality of the response data depends to a great 
extent on the latent ability structure of the examinees. Correlated abilities 
























46 


tend to yield response data with a single dominant component or, viewed 
differently, response data with correlated latent dimensions. Uncorrelated 
abilities tend to yield response data with relatively uncorrelated latent 
dimensions. 


When the latent item dimensions do not operate independently, the effect 
of the latent ability structure is less pronounced. The effect of the 
correlation between latent ability dimensions is the same, but less extreme. 

The effect of the interaction of latent ability and item structures has 
serious implications for the analysis of test data and, perhaps more 
Importantly, for test development. Clearly it is not sufficient to consider 
only item characteristics when constructing or analyzing a test. It is not 
appropriate to assume that the latent ability structure is determined by item 
characteristics. It is necessary to consider how the two interact to produce 
a latent structure for test data. 

The results of this stuiy also have important implications for the 
application of MIRT methodology. The presence of correlated abilities 
certainly had pronounced effects on the results obtained from the application 
of the MIRT model selected for this study. The most important finding of this 
study regarding the use of MIRT methodology involves the inability of the MIRT 
model estimation program to recover the true dimensions when the dimensions 
were correlated. 

When latent item dimensions are independent, the procedure works fairly 
well, even when the latent ability dimensions are correlated. However, an 
Increased correlation between latent ability dimensions does lower the 
correlations between the true and estimated abilities. This does not 
necessarily mean that the estimation process breaks down. It simply means 
that the nature of the ability dimensions recovered by estimation is somewhat 
different than for the true dimensions. It is entirely possible that the 
estimated dimensions are in some sense a rotation of the true dimensions due 
to the fact that the estimation procedure and/or model does not explicitly 
account for inter-dimensional correlations. 

When latent item dimensions are not independent, the recovered dimensions 
are different from the true dimensions regardless of whether or not the latent 
ability dimensions are correlated. Again, some type of rotation of the latent 
ability structure might be involved. Under the circumstances, if MIRT 
methodology is to be viable, research on this question must be conducted. If 
a rotation is involved, it is imperative that its nature be discovered. 
Moreover, if MIRT parameters are to be invariant and interpretable, it seems 
likely that it will be necessary to develop something analagous to factor 
rotations in factor analysis. 

Summary and Conclusions 

A study was conducted to assess the effects of correlated abilities on 
test characteristics, and to explore the effects of correlated abilities on 
the use of a multidimensional item response theory model which does not 
explicitly account for such a correlation. Two tests were constructed. One 
test had two relatively unidimensional subsets of items, while the other had 
items that were all two-dimensional. For each test response data were 


'.■vV-'.l 




•V. 


V •j‘. • 




generated according to a multidimensional two-parameter logistic model using 
four groups of examinees. The groups of examinees differed in the degree of 
Inter-dlmension ability correlation. 

To evaluate the effects of correlated abilities on test characteristics, 
the simulated response data were analyzed using Item analysis and principal 
component analysis techniques. To assess the effects of correlated abilities 
on the use of the multidimensional model, the parameters of the model were 
estimated, and the estimates were compared to the true parameters. 

The results of this study Indicated that the presence of correlated 
abilities has Important Implications for the characteristics of test data, and 
for the application of multidimensional Item response theory models. It was 
concluded that It Is necessary to consider latent Item structure as well as 
latent ability structure In test construction and analysis. It was also 
concluded that use of multidimensional Item response theory models that do not 
explicitly account for correlated abilities may not yield accurate information 
about the nature of underlying dimensions. It was suggested that research 
should be conducted to determine the relationship between the observed and 
true correlation between abilities, and to perhaps develop an Item response 
theory analogue to factor rotation. 


























References 



Blrnbaum, A. (1968). Some latent trait models and chelr use In inferring an 
examinees ability. In F.M. Lord and M.R. No/lck, Statistical theories of 
mental test scores. Reading, MA: Addlson-Wesley. 


McKinley, R.L. and Reckase, M.D. (August 1983). An extension of the two- 
parameter logistic model to the multidimensional latent space (Research 


Report ONR 83-2). Iowa City, lA; The American College Testing Program 
(a) 



McKinley, R.L. and Reckase, M.D. (August 1983). An application of a 

multidimensional extension of the two-parameter logistic latent trait model 


Research Report ONR83-3). Iowa City, lA: The American College Testing 
Program, (b) 


McKinley, R.L. and Reckase, M.D. (1983). MAXLOG: A computer program for 
the estimation of the parameters of a multidimensional logistic model. 
Behavior Research Methods and Instrumentation, 389-390. (c) 


Reckase, M.D. and McKinley, R.L. (1982). Some latent trait theory in a 
multidimensional latent space. Paper presented at the 1982 Invitational 
IRT/CAT Conference, Wyzata, Minnesota. 








* • * - * 4 . * ^ 



































V •. 


•; 1 Ir. lick lond 

04<tci o4 Itvil ItWirch 
Liiiion Of Met, Far East 
IPO San Franclico, Cl fiS03 

1 Ir. lobart Irtaux 
-V MVTRAEOUIPCEI 
Coda I-095R 

V Orlando, FL 32813 

• • 

1 Dr. Stanlay Collytr 
, Off;c» of Naval Technology 

I 60C N. Ouincy Street 

ftflmqton, VP 2221'’ 

1 CDF Hike Cc'rar. 

Office of Na«al Research 
BOO H. Quincy St. 

Code 27!: 

Arl;-.gten, VP 21217 

1 Dr. Joh- E'.lis 
Na<> Personnel RVD Cente' 

Sar. Diego, CP 92252 

1 Or. Richard Elster 
Dtpa rtaent of Adainistrative Sciences 
■aval Postgraduate School 
Monterey, Cl 9'.940 

1 N. PAT FEDERICO 
Code P13 
VROC 

San Diego, CA 92132 

I Hr. Dick Hoshae 
MVOP-135 
Arlington Annex 
Rooa 2834 

Mashington , DC 20350 

1 Dr. Norean J. Kerr 
Chief of Naval Technical Training 
Naval Pir Station Neaphis (75) 
Millington, TN 38034 

1 Dr. Leonard hroeker 
lav> Personnel RkD Center 
San Diego, CP 92152 

1 Dar.i; Lang 
Na<y Personnel RID Center 
San Diegc, CA 92152 


1 Dr. Nilliaa L. Maloy (02i 
Chief of Naval Education and Training 
Naval Air Station 
Pensacola, FL 32308 

1 Dr. Jues Hciride 
Navy Personnel RED Center 
San Diego, CP 92152 

I Dr Nilliaa Montague 
NPRDC Code 13 
San Diego, CP 92152 

1 Library, Code P201L 
Navy Personnel RED Center 
San Diego, CA 92132 

1 Technical Director 
Navy Personnel RED Center 
San Diego, CA 92152 

6 Personnel E Training Resea'ch Sroup 
Code 442PT 

Office of Naval Research 
Arlington, VP 22217 

1 DR. MARTIN F. HISKOFF 
NAVY PERSONNEL RE D CENTER 
SAN DIE80, CA 92152 

1 Dr. Carl Ross 
CNET-PDCD 
luilding 90 

Brcat Lakes NTC, IL 40088 

1 Nr. Drew Sands 
NPRDC Code 62 
San Diego, CA 92152 

1 Nary Schrat: 

Navy Personnel RED Center 
San Diego, CA 92152 

1 Dr. Robert 6. Seith 
Office of Chief of Naval Operations 
0F-9B7ri 

Hashingtcn, DC 20350 

I 2r. Alfred SecSe, Director 
Departiient N-" 

Naval Yraining Equipeent Center 
Orlando, FL 328:7 
















1 Ir. Richard Smw 
LMton Scitntift 
04{ice o4 Raval Rttcarch 
Iranch Office, London 
Box 39 

FPO Hc« York, RY 09S10 

1 Dr. Richard Sorenoon 
Na>y Personnel RID Cente' 

Sar Diego, CA 92152 

1 Dr. Frede'^icl Steinheise' 

CNC - OPli' 

Naw Anne> 

Ar;ir.gton, VA 20370 

1 hr. Brad Syopson 
Nav, Personnel RID Center 
Sa* Diego, CA 92152 

I Dr. Jaoes Toeeddale 
Tec^nkal Director 
Navy Personnel RLC Ceme' 

Sa'. Diego, CA 92152 

1 Dr. Frank Vicino 
Navy Personnel RkD Center 
San Diego, CA 92152 

1 Dr. Eduard Negoan 

Office of Naval Rotearch (Code 411StP) 
800 North Duincy Street 
Arlington, VA 22217 

1 Dr. Ronald Hcitiaan 
Naval Postgraduate School 
Departoent of Adoinistrativi 
Sciences 

Honterey, CA 93.940 

1 Dr. Douglas Hetzel 
Code 12 

Navy Personnel RtO Center 
San Diego, CA 92152 

1 DR. MARTIN F. NISKOFF 
NAVv PERSONNEL Rt D CENTER 
SAN D1E60. CA 92152 

1 Hr John H. No’.fe 
Nav) Personnel Rtl' Center 
San Diego, CA ^2152 


1 Dr. Hallace Nulfeck, III 
Navy Personnel RtD Center 
San Diego, CA 92152 


Narine Corps 

1 Jerry Lehnus 
CAT Project Office 
H6 Narine Corps 
Nashington , DC 20360 

1 Headquarters, U. S. Narine Corps 
Code NPI-20 
Nashington, DC 20360 

1 Spec lal Assistant for Narine 
Corps Natters 
Code 1001 

Of'Ke of Na.al Research 
BOC N. Cuincy St. 

Arlington, VA 22217 

1 Najor Frank Yohannan, USNC 
Headquarters, Narine Corps 
(Code NPl-20) 

Hasringtc.*, DC 203BC 


Aray 

1 Dr. Kent Eaton 
Aroy Research Institute 
5001 Eisenhouer Blvd. 

Alexandria , VA 22333 

1 Dr. Nyron Fischl 

U.S. Aroy Research Institute for the 
Social and Behavioral Sciences 
5001 Eisenhouer Avenue 
Alevandna, VA 22333 

I Dr. Clesser Nartm 
Aroy Research Institute 
5001 Eisenhouer Blvd. 

Alexandria, VA 22333 

1 Dr. Nilliai E. Nordtrock 
FHC-ADCO Box 25 
APO. NY 0971C 

1 Dr, Ha-oli F. C’Neil. Jr. 

Director, Training Research Lab 
Ariy Research Institute 
5001 Eisenhouer Avenue 
Alexandria, VA 22333 





















































r 

S’ 

% 



] CMundtr, O.S. Ariy RttHrtk Inftitute 
for tht Miavioril t Social Scitncis 
ATTN: PERI-IR (Dr. Judith Orasanu) 

9001 EistnhoMT Dvtnut 
Oliiandria, VA 22333 

I Nr. Robtrt Ross 

U.S. Aroy Rosearch Institute for the 
Social and letiavioral Sciences 
300! EistnhoMr Avenue 
Alciandria, VA 22333 

1 Dr. Robert Sasoor 
U. S. Aroy Research Institute for the 
Dehavioral and Social Sciences 
3001 Eisenbouer Avenue 
Alexandria. VA 22333 

1 Dr. Joyce Shields 
Arty Research Institute for the 
Bshav'oral and Social Sciences 
5001 Eisenhover Avenue 
Alexandria. VA 2233: 

1 Dr. Hilda Ring 
Arty Research Institute 
SOCl Eisenhouer Ave. 

Alexandria, VA 22333 


Air Force 

1 Dr. Earl A. Alluisi 
HR, AFHRL (AFSC) 
Drooks AFi, TI 78235 



1 Nr. Rayoond E. Christal 
RFHRL/NOE 

Drooks AFD, Tl 78235 



I Dr. Alfred R. Fregly 
AFOSR/NL 

Dolling AFB, DC 20332 

1 Dr. Senevieve Haddad 
^rcgrat Harager 
;.;(e Sciences Drectorate 
AFCSR 

Selling AFB, DC 20332 

1 Dr. Patrick Ryllonen 
AFHRL/HOE 


1 Nr. Randolph Park 
RFNRL/NOAM 

Drooks AFB, TI 78235 

1 Dr. Roger Pennell 
Air Force Huaan Resources Laboratory 
LOM-y AFB, CO 80230 

1 Dr. Nalcola Ree 
AFHRL/HP 

Brooks AFB, TI 78235 

1 Hajor John Nelsh 
AFHRL/NOAN 

Drooks AFD , TI 78223 


Departrent of Defense 

12 Defense Technical Inforoation Center 
Cateron Station, Bldg 5 
Alexandria, VA 22314 
Attn: TC 

1 Nilitary Assistant for Training and 
frsonnel Technology 
Office of the Under Secretary of Oefens 
for Research I Engineering 
Root 33129, The Pentagon 
Mashington, DC 2C30I 

i Dr. H. Steve Selloan 
Office of the Assistant Secretary 
of Defense (HPA I L) 

2B269 The Pentagon 
Mashington, DC 2030! 

1 Dr. Robert A. Nisher 
OISDRE (ELS 

The Pentagon, Rooa 3D129 
Mashington, DC 20301 




Drooks AFB, Tl 78235 






^ - kv 
. • - ■ « 









,1 J . I,-H 

>i-:- 



CivlllM AgtOClca 

1 Or. Virn H. Urry 
Pirsoflntl OtO Ctnttr 
IHfict o4 Ptrsonntl Nanagmnt 
IfOO E Strtft m 
tlat^in|ton, K 20413 

1 Hr. TkOMi A. Mart 
U. S. Coait Suard Inatituta 
P. 0. Substation IB 
Oklahoaa City, OK 73169 

1 Dr. Jostph L. Young, Director 
Heaory I Cognitive Processes 
National Science Foundation 
Hashington, DC 20550 


f?' 





Private Sector 


1 Br. Jaaes Algina 


university of Florida 


Bainesville, FL 32605 


1 Br. Erling B. Andersen 

• 

Beparteent of Statistics 


Itudiestraede 6 


1433 Cepenhagen 

J 

BENHARK 

. 1 -J 

1 Or. .iaac leja'- 


Educational Testing Service 


Princeton, NJ 0fl43C 

_.v3 

1 Dr. Henucha Birenbaua 

■ vj 
*• 

School of Education 

■ 

Tel Aviv University 

• 

Tel Aviv, Raeat Aviv 69976 


Israel 


1 Or. Nerner Sirke 

•W 

• . ’ 

Personalstaeaaat der Bundesuer' 


D-300C Koelr- 90 


NEST 6ERNANY 

iii 

1 Dr. R. Darrell Ixk 

. »• • 

Beparteent of Education 


University of Chicago 


Chicago, IL 60637 

•J 


I 


te 

]■■- '*■ 


V-iN* 


^4 




I Nr. Arnold lobrer 
SKtion oF Psychological Research 
Caserne Petits Chateau 
CRE 

1000 Brussels 
Bilgiua 

1 Dr. Robert Brennan 
Aeerican College Testing Prograes 
P. 0. lo« 168 
loea City. lA 32243 

1 Dr. Blenn Bryan 
6206 Pee Road 
Bethesda, HD 20B17 

1 Dr. Ernest R. Cadotte 
307 stokely 

University of Tennessee 
Knoxville. ’N 3’?16 

1 Dr. John B. Ca"’ol! 

409 Elliott 'd. 

Chapel Hill, RC 






* • " j. » 



















1 Ir. Notmi) Cliff 
••ft. of Ptycholofy 
Itoiv. of So. California 
llRivirfity Pa. k 
Ui UnftlM, Cil V0007 

> Ir. Mint Croib*; 
Elucation Rtatarcb Ctnttr 
Onivarsity of Ltyftn 
loorbaavdaan 2 
2334 EH Layden 
Tha NETHEPLAHDS 

1 laa Cronbact 
IS Laburnua Road 
Atharton, CA 9^205 

I CTI/Hc6rat-Hill Library 
2300 Gardan Road 
Honta'’ay, CA ?j940 

1 Dr. Haltar Cunningbai 
Uni varsity of Hiaoi 
••partoant of Psychology 
Sainesvilla, FL 32611 

1 Dr. Dattpradad Divgi 
Syracusa Uni varsity 
••partoant of PsycAoiogr 
Syracusa, ME 33210 

I Ir. Eooanual lonchin 
•apartaont of Psychology 
(Riivorflty of Illinois 
OMapaign, IL *1820 

1 Ir. Hai-Xi long 
•all Foundation 
Rooa 314, luilding I 
•00 Roosevalt Road 
Gian Ellyn, IL *0137 

1 Or. Frit: Drasgoa 
Oapa'tKnt of Psychology 
Univarsity of Illinois 
6{)3 E. Danial St. 
Chaapaign, IL 61S20 

1 Dr. SuSi' Eafcarlscn 
fsrcHGLOS'- rEP4'‘:PE^' 


1 ERIC Facility-Acquisitions 
4833 Rugby Avanua 
•atbasda, HD 20014 

1 Or. 8anjaaia A. FairAank, ir. 
McFann-lray t Asaociatas, Inc. 

923 Callaghan 
Suita 225 

San Antonio, Tl 78228 

1 Dr. Laorard Faldt 
Lindquist Carta' ir Baasuraant 
Univarsity o* Ion 
loaa City. lA 32241 

1 Ur;v. Prof. Or. Sarhard Fischar 
Liabiggassa 3/3 
A 1010 Vianna 
AUSTRIA 

1 Professor Honald Fitigarald 
Univarsity of Ha* England 
Araidala, Rau SOkth Halas 2351 
AUSTRALIA 

1 Or. Daitar Flatcbar 
University of Oregon 
Oapartaant of Co^utar Science 
Eugene, DR P7403 

1 Or. John R. Fradariksan 
•olt laranak * Rauaan 
SO Houlton Street 
Caabridga, HA 02138 

1 Dr. Janice Gifford 
Univarsity of Hassachusatts 
School of Education 
Aaharst, HA 01002 

1 Dr. Robert Slasar 

Learning Research * Oevalopaent Center 
University of Pittsburgt 
3'»39 O’Hara Street 
PITTSIURSN, PA 15260 

1 Dr. Harvin D. Slock 
217 Stone Hall 
Cornell University 
Ithaca, MY 14853 




IL! J.' L' »m I'J WA' '-F/V WM 'W*»■* *.» •}■ *.i'' 



1 tr. Itrt Srtcn 
Johns Hopkins University 
Departoont of Psychology 
CbarlH I 34th Street 
laitioore, HS 21218 

1 M. JMES 6. 6REEM 
LHK 

UHIVERSITY OF P1TTSIUR6H 
3439 O’HARA STREET 
PIT^SBURSH, PA 15213 

1 Dr. Ron Haobleton 
School of Education 
University of Hassachusetts 
Aeherst, HA 01002 

1 Or. Paul Horst 
i77 6 Street, 1184 
Chwli Vista. CA 9001C 

1 Dr. Llovd Huipbrevs 
Depa''taent cf hsychslog, 

University of Illinois 
807 East Daniel Street 
Chaepaign, IL 81020 

1 Dr. Steven Hunka 
Departoent of Education 
University of Alberta 
Edoonton, Alberta 

CANADA 

1 Ir. Earl Hunt 
Dept, of Psychology 
University of Nashington 
Seattle, M 9S10S 

1 Dr. Jack Hunter 
2122 Coohdge St. 

Lansing, HI 48908 

1 Dr, Huynh Huynh 
College of Education 
University of South Carolina 
Coluebia, SC 29208 

1 Dr. Douglas H. Jones 
Advanced Statistical Technologies 
Corporation 
10 Trafalgar Cou''t 
Lau'enceville, HI 08:46 


1 Dr. Harcel Just 
Departoent of Psychology 
Carnegie-Hellon University 
Pittsburgh, PA 1S213 

1 Dr. Doeetrios Karis 
Departoent of Psycho!ogy 
University of Illinois 
803 E. Daniel Street 
Chaepaign, IL 81820 

1 Professor John A. Keats 
Departaent of Psychology 
The University of Neucastle 
N.S.M. 2308 
AUSTRALIA 

1 Or. Milliao Koch 
University of Tevas-Austin 
Neasureoent and Evaluation Center 
Austin, Tt 78703 

1 Dr. Alan Lesgold 
Learning R80 Center 
University of Pittsburgh 
3939 O’Hara Street 
Pittsburgh, PA 15280 

1 Or. Hichael Levine 
Departoent of Educational Psychology 
210 Education Dldg. 

University of Illinois 
Chaepaign, IL 81801 

1 Dr. Charles Leeis 
FKolteit Sociale Hetenschappen 
Rijksuniversiteit Broningen 
Oude Boteringestraat 23 
97126C Broningen 
Netherlands 

1 Dr. RcRiert Linn 
College of Education 
University of Illinois 
Urbana, IL 81801 

1 Hr. Phillip Livingston 
Systees and Applied Sciences Corpcratic 
861: Keniluorth Avenue 
Riverdale. HD 2084C 













1 Dr. Robtrt Lockun 
Center tor Naval Analysis 
200 North Beauregard St. 

Alexandria, VA 22311 

1 Dr. Frederic R. Lord 
Educational Testing Service 
Princeton, NJ 08341 

1 Dr. Jaees Luasder 
Defarteent of Psychology 
University of Nesterr Aust'il.a 
Nedlands H.A. iOOP 
AUSTRALIA 

1 Or. Don Lyor, 

P. 0. Bo> 4< 

Higley , AZ 8523f> 

1 Dr. Gary Harco 
Stop 31-E 

Educational ^estin; Ser.ice 
Princeton, NJ 0843! 

1 Dr, Scott Kaxtell 
Departeent of Psychology 
University of Notre Oaee 
Notre Oaae, IN WZb 

1 Dr. Saauel T. Nayc 
Loyola University of Chicago 
120 North Hichigan Avenue 
Chicago, IL Will 

1 Nr. Robert HcKlnley 
Noerican College Testing Prograos 
P.O. Dox la8 
Iowa City, lA 32243 

1 Dr. Barbara Heans 
Huoan Resources Research Organization 
300 North Nashington 
Alexandria, VA 22314 

1 

Processor Jason Nil lean 
Departeent c' Education 
Stone Hall 
Corbel; lir.iversit* 

If ica. Nv 1403: 


1 Dr. N. Alan Niceeander 
University of Otlahoea 
Departeent of Psychology 
Oklahoea City, OF 730i9 

1 Dr. Donald A Noroan 
Cognitive Science, C-013 
Univ. of California, San Diego 
La Jolla, CA 92093 

1 Dr. Ne'.vin R. Novic> 

336 Lindquist Cente' for Heasureent 
Uni.ersity of loea 
loea City, lA 32242 

1 Dr. Janes Olson 
NICAT, Inc. 

1873 Scuth State Street 
Orea, UT 84037 

1 Hayne N. Patience 
Anerican Cccnci'. on Education . 
BED Testing Service, Suite 20 
One Dupont Cirle, NN 
Nashington, DC 20036 

1 Dr. Janes Paulson 
Dept, of Psychology 
Portland State UniveSrsity 
P.O. Box 731 
Portland, OR 97207 

1 Or. Janes H. Pellogrino 
University of California, 

Santa Barbara 
Dept. of Psychology 
Santa Darabara , CA 93106 

1 Dr. Douglas H. Jones 
Advanced Statistical Technologies 
Corporation 
10 Trafalgar Court 
Laerenceville, NJ 08148 

1 Dr. Steven E. FoltrocL 
Bell Laboratories 2D-444 
60C Nountain Ave. 

Hur'-av Hii;, NJ 07974 

1 Dr. Na-'k D. Reciase 
ACT 

P. 0. Bor 168 
love City, lA 32243 


1 D'. Robert Hislew 
711 lilincis Street 
Geneva. IL 6C!!4 



















































I Ir. Thoui Rtynolds 
Univtriity of Tnat-Oillti 
lUrketing ItpirtMnt 
P. 0. loi 686 
Rickariton, TX 79080 

1 8r. LawMCt Radntr 
403 Elt Ovinut 
T*koM Park, m 20012 

1 Dr. J, Ryan 
Dtpartaent of Education 
University of South Carolina 
Coluabia, SC 29208 

1 PROF. FUHIKO SANEJIHA 
DEPT. OF PSYCHOLOBY 
UNIVERSITY OF TENNESSEE 
KNOIVILLE, TN 3^916 

1 Frank L. Schaidt 
Departient of Psychology 
Rldg. 66 

George Mashington University 
Nashington, DC 200C2 

1 Or. Halter Schneider 
Psychology Departaent 
603 E. Daniel 
Chaapaign, IL 61820 

1 Leaell Schoer 
Psychological 6 Quantitative 
Foundations 
College of Education 
University of loaa 
loaa City, lA S2242 

I Dr. Eaaanuel Donchin 
Departaent of Psychology 
University of Illincis 
Chaapaign, IL 61820 

I Dr. Kaiuo Shigenasu 
7*9'24 Kugenuaa-F.aigan 
Fujusana 251 
JAPAN 

1 Cr. Hilliaa Sias 
Cente*" for Na«a! final vs; s 
200 North Beaurec.rd Street 
filekandna, VA 21!!! 


1 Dr. H. Nallace Sinaiko 
Prograa Director 

Nanpoaer Research and Advisory Services 
Saithsonian Institution 
801 North Pitt Street 
Alexandria, VA 22314 

1 Martha Stocking 
Educational Testing Service 
Princeton, NJ 08541 

1 Dr. Peter Stolof* 

Center for Naval Analysis 
200 North Beauregard Street 
Alexandria, VA 22311 

1 Dr. Hilliaa Stout 
University of Illinois 
Departaent of Natheaatics 
Urbana, IL 61801 

1 DR. PATRICK SUPPES 
INSTITUTE FOR HATHEHATICA. STUOIES IN 
THE SOCIAL SCIENCES 
STANFORD UNIVERSITY 
STANFORD, CA 94305 

1 Dr. Hariharan Saaainathan 
Laboratory of Psychoaetric and 
Evaluation RHearch 
School of Education 
UnivKsity of Massachusetts 
Aaherst, NA 01003 

1 Dr. Kikuai Tatsuoka 
Coaputar Based Education Research Lab 
252 Engineering Research Laboratory 
Urbana, IL 61801 

1 Or. Maurice Tatsuoka 
220 Education Bldg 
1310 S. Sixth St. 

Chaapaign, IL 61820 

I Dr. David Thissen 
Departient of Psychology 
University of Kansas 
Laurence, KS 66044 

1 Dr. Douglas Toune 
Univ. of Sc. Cali^cnia 
Behavioral TethnoJc^, ljijs 
1845 S. Elena Avt. 

Redondc Bea:“. Cfi 91277 





















































c 


.’v/^ 1 #r. Robtrt T«ttt*ki>«i 
v oi Stitiitic* 

0»iv#r»ity of Hitiouri 
• ColuabK, HO 45201 

^ 1 If. LHyifd Tocktr 
UBivoriity of Illinoi* 
Otpirtoont of Piycholooy 
603 E. 0«i*l Strut 
Chaopiign, IL 61B2C 


1 Or. V. R. R. Uppuluri 
Union Carbide Corporation 
Hue 1 ear Diviiion 
P. 0. lo> Y 
Oak Ridge, TN 37830 

1 Or. David Vale 
Assesownt Systees Corporation 
2233 University Avenue 
Suite 310 
St. Paul, HH SSIM 

1 Dr. Houard Maine' 

Division of Psychological Studies 
Educational Testing Service 
Princeton, M 08540 

1 Or. HicAael T. Haller 
Departoent of Educational Psychology 
University of Hisconsin—Hiluaukee 
UilMukK, Ml 53201 

1 Dr. Irian Haters 
HuaRM 

300 Morth Hashington 
Aletandria, VA ^314 

1 Or. David J. Heiss 
H660 Elliott Mall 
University of Minnesota 
75 E. River Road 
Minneapolis, HN 5S4SS 


1 Or. Rand R. ll;lco« 

University of Southern Califc'nia 
Decartnent of Psychology 
Lcs Angeles, CA P0CC7 


1 lerean Military Representative 
RTTH: Hoifgang Hildegrube 
Strcitkraefteaet 
I-S300 knn 2 

4400 Irandyuine Itreet, MH 
Uasbington . K 20016 

1 Dr. Iruce HilliaH 
Departeent of Educational Psychology 
University of Illinois 
Urbana, IL 61801 

1 Ms. Marilyn Hingersky 
Educational Testing Service 
Princeton, NJ 0854; 

1 Or. Hendy Yen 
CTI/Mc6ra« Mill 
Del Monte Research Park 
Monterey, CA 93940 





'2V Cw0*ifIVlV lV^.A