Applied Psychometrics using SAS

279 Pages·2014·2.204 MB·English

by  Holmes Finch, Brian F. French, Jason C. Immekus

Applied
Psychometrics
Using SAS

Applied
Psychometrics
Using SAS
W. Holmes Finch
Ball State University
Brian F. French
Washington State University
James C. Immekus
University of Louisville
INFORMATION AGE PUBLISHING, INC.
Charlotte, NC • www.infoagepub.com
Library of Congress Cataloging-in-Publication Data
A CIP record for this book is available from the Library of Congress
http://www.loc.gov
ISBN: 978-1-62396-694-2 (Paperback)
978-1-62396-695-9 (Hardcover)
978-1-62396-696-6 (ebook)
Copyright © 2014 Information Age Publishing Inc.
All rights reserved. No part of this publication may be reproduced, stored in a
retrieval system, or transmitted, in any form or by any means, electronic, mechanical,
photocopying, microfilming, recording or otherwise, without written permission
from the publisher.
Printed in the United States of America
Contents
Preface ...................................................................................................ix
1 Introduction to Psychometric Concepts .........................................1
Measurement Basics ..........................................................................1
Classical Test Theory ........................................................................2
Item Response Theory ......................................................................5
Dichotomous Items ............................................................................5
Polytomous Items ............................................................................10
2 Item Analysis .................................................................................15
Introduction ....................................................................................15
Classical Test Theory Item Difficulty ...........................................15
CTT Item Difficulty for Dichotomous Items ....................................15
CTT Item Difficulty for Polytomous Items ........................................20
Classical Test Theory Item Discrimination ...................................22
Extreme Groups Calculation Method ...............................................23
Biserial/Point Biserial Correlations .................................................25
Proc Corr Nocorr Alpha .................................................................28
Developing an Automatic Scoring Routine for a Test/Survey ....32
3 Reliability .....................................................................................35
Introduction ....................................................................................35
Measures of Internal Consistency ..................................................36
KR-20 ............................................................................................37
Cronbach’s α for Ordinal Data ........................................................47
v
vi  Contents
Split-Half Reliability .......................................................................50
Test–Retest Reliability .....................................................................52
Chapter Summary ...........................................................................54
4 Generalizability Theory ................................................................55
Introduction ....................................................................................55
G-studies/D-studies ........................................................................56
Variance Components ....................................................................57
Generalizability Coefficient/Phi Coefficient ................................60
Example 1: One Facet Crossed Design ..............................................61
Example 2: Two Facet Crossed Design ..............................................68
5 Validity ...........................................................................................75
Introduction ....................................................................................75
Types or Sources of Validity Evidence ...........................................77
Associations with Other Variables: Concurrent Validity .............80
Considerations in Concurrent Validity Assessment ...........................85
Associations with Other Variables: Predictive Validity ................87
Discriminant Groups Validity ........................................................89
Construct Validity ...........................................................................95
Exploratory Factor Analysis as a Tool for Investigating
Construct Validity ....................................................................96
Confirmatory Factor Analysis as a Tool for Investigating
Construct Validity ...................................................................117
Fitting a CFA Model Using SAS ....................................................120
Chapter Summary .........................................................................132
6 Issues in Scoring ..........................................................................135
Introduction ..................................................................................135
Types of Scores ..............................................................................136
Raw Scores ...................................................................................136
Weighted Scores .............................................................................137
Percentile Scores ............................................................................137
Standard Scores ............................................................................139
Calculation of Raw Scores Using SAS .........................................141
Calculation of Weighted Scores Using SAS ................................145
Calculation of Percentiles Using SAS ..........................................148
Calculation of Standardized Scores Using SAS ..........................149
Chapter Summary .........................................................................150
Contents  vii
7 Item Response Theory .................................................................153
Introduction ..................................................................................153
IRT versus CTT .............................................................................154
Dichotomous IRT Models ............................................................157
Rasch Model .................................................................................157
One-Parameter Logistic Model ......................................................169
2PL Model ...................................................................................172
Fitting the 3PL Model in SAS ........................................................177
Polytomous IRT Models ...............................................................184
Partial Credit Model .....................................................................185
Generalized Partial Credit Model ..................................................191
Graded Response Model ................................................................195
Comparing Fit for IRT Models ......................................................199
Chapter Summary .........................................................................202
8 Differential Item Functioning .....................................................205
Introduction ..................................................................................205
DIF versus Impact .........................................................................208
Mantel-Haenszel Test ....................................................................209
Logistic Regression .......................................................................213
Examples ......................................................................................216
Logistic Regression Example ..........................................................224
Chapter Summary .........................................................................235
9 Equating ......................................................................................237
Introduction ..................................................................................237
Equating Sampling Designs .........................................................238
Mean Equating ..............................................................................241
Linear Equating ............................................................................244
Equipercentile Equating ..............................................................250
Chapter Summary .........................................................................259
References ............................................................................................261

Preface
This book was developed to provide a “one-stop” resource for many
common analyses that an applied researcher might complete when
working with various instruments to measure educational and psychologi-
cal traits. We have developed examples, collected our favorite examples and
resources, and provided explanations of analyses in one easily digestible
text. Many of the analyses presented assist in providing the recommended
evidence to support the inferences drawn from scores of such instruments.
That is, the results from applying these techniques assist in providing score
reliability and validity evidence.
Through our years as graduate students and the first segment of our
academic lives, we have explored the use of various programs for scale de-
velopment and the study of the psychometric properties of the scores to
provide such evidence. We have had the experience, as I am sure many of
you have had, of turning to multiple books for instructions and examples
to complete analyses in the scale development and validation process. For
those readers just beginning on the psychometric ride, you too will soon ex-
perience this. By no means will this book stop the need for multiple sourc-
es; in fact, that is always encouraged. However, this book should allow the
reader to use this as a main guide and supplement to experience analyses
described in major text books. Our examples are intended to be clear and
Applied Psychometrics Using SAS, pages ix–xii
Copyright © 2014 by Information Age Publishing
All rights of reproduction in any form reserved. ix