Comparability Study of Armed Services Vocational Aptitude Battery Scores from Answer Sheet and ...

17 Pages·2011·0.47 MB·English

DOCUMENT RESUME
TM 003 978
ED 097 350
Valentine, Lonnie D., Jr.; Cowan, Douglas K.
AUTHOR
Comparability Study of Armed Services Vocational
TITLE
Aptitude Battery Scores from Answer Sheet and Answer
Card Administration. Final Report.
Air Force Human Resources Lab., Lackland AFB, Tex.
INSTITUTION
Personnel Research Div.
AFHRL-TR-73-55
REPORT NO
Jan 74
PUB DATE
16p.
NOTE
MF-$0.75 HC-$1.50 PLUS POSTAGE
EDRS PRICE
*Answer Sheets; *Comparative Analysis; Mil.Aary
DESCRIPTORS
Personnel; Occupational Tests; *Response Mode;
*scores; Testing; Test Results; Vocational
Aptitude
*Armed Services Vocational Aptitude Battery
IDENTIFIERS
ABSTRACT
Armed Services Vocational Aptitude Battery (ASVAB)
Form 2 was standardized for Digitek answer sheets. In September 1973,
the Digitek scoring facility at Randolph AFB will be inadequate to
handle anticipated scoring load. Consequently, ASVAB answer forms
have been redesigned for processing via a Hewlett-Packard mark-sense
reader from IBM card size forms which are more compressed than was
the Digitek form. Score comparability from administrations on the two
form
answer forms was investigated. It was found that the answer
change had negligible effect on average examinee performance overall
the battery's tests.
score distributions, or correlations among
Significant Interaction between initial ability (as measured by the
Armed Forces Qualification Test) and answer form used was found for
two tests (Word Knowledge and Coding Speed); however the interaction
did not follow a clearcut pattern. It was recommended that existing
ASVAB conversion tables continue in use until completion of a full
restandardization study.
(Author)
AFHRL-TR-73.65
R FORCE El
COMPARABILITY STUDY OF ARMED SERVICES
VOCATIONAL APTITUDE BATTERY SCORES
FROM ANSWER SHEET AND ANSWER CARD
H
ADMINISTRATION
U
By
M
Lonnie D. Valentine, Jr.
Douglas K. Cowan
A
PERSONNEL RESEARCH DIVISION
Lackland Air Force Base, Texas 78236
January 1974
Approved for public release; distribution unlimited.
u s DEPARTMENT or HEALTH
EDUCATION & WELFARE
NATIONAL INSTITUTE
OP
,
EDUCATION
uf
tat f 4
,
f1Ar. .
,' I
t)
t)
t)
I LP)...
v
F
.F
CA'',14P4
P't .4)",
t
Y r
,)c) N
v..
,:r;
t;
Put
Nr.,
:.)(
4
r
N4 9',.
4N°
F );
I
r
r,
.. r)k, p,),
u pc,
t-P
4,
LABORATORY
S
AIR FORCE SYSTEMS COMMAND
BROOKS AIR FORCE BASE,TEXAS 78235
BEST
COPY
4161081E
NOTICE
When US Government drawings, specifications, or other data are used
other than
definitely
purpose Government
for
related
any
a
operation,
the
thereby
Government
procurement
incurs no
responsibility nor any obligation whatsoever, and the fact that the
Government may have formulated, furnished, or in any way supplied
the said drawings, specifications, or other data is not to be regarded by
implication or otherwise, as in any manner licensing the holder or any
other person or corporation, or conveying any rights or permission to
manufacture, use, or sell any patented invention that may in any way
be related thereto.
This final report was submitted by Personnel Research Division, Air
Force Human Resources Laboratory, Lack land Air Force Base, Texas
78236, under project 7719, with the Hq Air Force Human Resources
Laboratory, Brooks Air Force Base, Texas 78235.
This report has been reviewed and cleared for open publication and/or
public release by the appropriate Office of Information (01) in
accordance with AFR 190.17 and DoDD 5230.9. There is no objection
to unlimited distribution of this report to the public at large, or by
DDC to the National Technical Information Service (NTIS).
This technical report has been reviewed and is approved.
LELAND D. BROKAW, Chief
Personnel Research Division
HAROLD E. FISCHER, Colonel, USAF
Commander
Unclassified
SECURITY :LASSIFICATION OF THIS PAGE (When Rine Frnered)
READ INSTRUCTIONS
REPORT DOCUMENTATION PAGE
BEFORE OWL FTING FORM
RECIPIENT''., cAT aLor, NUMBER
2. GOVT ACCESSION NO.
REHOR ' NUMBER
3
1
AFFIRIAR-73.55
TYPE OF REPORT 6 PERIOD COVERED
5
TITLE !old Sublett,'
4
COMPARABILITY STUDY OF ARMED SERVICES VOCATIONAL
Final
AVM DE BATTERY SCORES FROM ANSWER SHEET AND
ANSWI.R CARD ADMINISTRATION
6. PERFORMING ORG. REPORT NUMBER
CONTRACT OR GRANT NUMBER(5)
AUTHOfq /
6
7.
Lonnie D.'Valentine, Jr.
Douglas K. Co,,'in
10. PROGRAM ELEMENT. PROJECT. TASK
RGANIZATION NAME AND ADDRESS
PES FnrIMIN
9
AREA 6 WORK UNIT NUMBERS
Personnel Research Division
77191002
Air Force Human Resources Laboratory
Lackland Air Force Base, Texas 78236
12. REPORT DATE
11. CONTROLLING OFFICE NAME AND ADDRESS
January 1974
Fig Air Force Human Resources Laboratory
Brooks Air Force Base, Texas 78235
NUMBER OF PAGES
13
14
SECURITY CLASS. (of this report)
14. MONITORING AGENCY NAME Pt ADORESS(if different from Controlling Office)
15
Unclassified
DECLASSIFICATION DOWNGRADING
15a
SCHEDULE
STATEMENT /of this Report)
16
DISTRIBUTION
Approved for public release; distribution unlimited.
DISTRIBUTION STATEMENT (of the almtract entered in Block 20, if different from Report)
17.
Ie. SUPPLEMENTARY NOTES
0
19. KEY 140 IDS (Continue on reverse side if nece.,saty and Identify by block number)
Armed E entices Vocational Aptitude Battery
answer t mnat
perform tnce comparability
high school testing
.
A
(Continue on teversu side If ne essary and identify by block number)
ABSTRACT
20.
Armed Services Vocational Aptitude Battery (ASVAB) Form 2 was standardized for Digitek answer sheets. In
Septetnuer 1973, the Digitek scoring facility at Randolph APB will be inadequate to handle anticipated scoring load.
Consequently, ASVAB answer forms have been redesigned for processing via a HewlettPackard marksatise reader
from IBM card size forms which are more compressed than was the Digitek form. Score comparability from
administrations on the two answer forms was investigated.
It was found that the answer form change had negligble effect on average examinee performance overall score
distributions, or correlations among the battery's tests. Significant interaction between initial ability (as measured by
the AFQT) and answer form used was found for two tests Word Knowled e and Codin 1 Steed . hoWever the
"
DO "
EDITION OF 1 Nov 66 IS OBSOLETE
Unclassified
1473
1 JAN
7 1
beta entered)
SECURITY CLASSIFICATION OF THIS PAGE (When
Unclassified
S5GURITY CLASSIFICATION OF THIS PAGE(When Data Enter itd)
Item 20 Continued.
interaction did not follow a clearcut pattern. It was recommended that existing ASVAI3 conversion tables continue
in use until completion of a full restandardization study.
Unclassified
SECURITY CLASSIFICATION OF THIS PAGE(Wheft Data Entered)
PREFACE
This report describes effect of answer form change (from a sheet to IBM size cards)
on performance on the Armed Services Vocational Aptitude Battery. Work was
accomplished under Project 7719, Ai; Force Personnel System Development on
Retention, Promotion, and
Control,
Quality
Evaluation,
Selection,
Assignment,
Utilization; Task 771910, Armed Forces Operational Selection Tests.
TABLE OF CONTENTS
Page
5
Introduction
I.
5
Procedure
H.
5
Results
III.
6
Conclusions and Recommendations
IV.
14
Refemces
'444.04,
LIST OF TABLES
Page
Table
7
Sample Distribution by AFQT Decile
1
7
Distribution of Educational Level
2
Number of Cases Reporting Completion of High School Mathematics
3
7
and Science Courses
7
AFQT Mental Category Distribution of Samples
4
8
Distribution of AQE Scores
5
9
Intercorrelations Among Twenty Variables
6
10
ASVAB Means and Standard Deviations for Samples 2XC and 2X
7
Analysis of Variance Summary
11
8
12
ASVAB Means and Standard Deviations with AFQT Deci les
9
Raw Score Distributions on Coding Speed
13
10
14
Raw Score Distributions for ASVAB Tests 2 Through 9
11
COMPARABILITY STUDY OF ARMED SERVICES VOCATIONAL.
APTITUDE BATTERY SCORES FROM ANSWER SHEET AND
ANSWER CARL) ADMINISTRATION
Test (AFQT)
on Armed Forces Qualification
I. INTRODUCTION
scores contained in their records. Matching was
as many percentile
completed
for
The Armed Services
Aptitude
scores
Vocational
as
available from within the total number of airmen
Battery (SVAB), described by Vito la and Alley
Distribution of AFQT deciles for the
tested.
(1968) an ..1 Bayroff and Fuchs (1970), is currently
for testing of high school seniors in
samples is given in Table 1.
operation; 1
voluntarily participating high schools throughout
Testing was accomplished under normal testing
the United States. The current form of ASVAB
room conditions. ASVAB-2XC and 2X were
use of a DIGITEK answer sheet,
requires the
administered alternately during morning and after-
scored on a DIGITEK DM100 optical scanner
noon testing sessions; each form was administered
located at Randolph Air Force Base, Texas.
half-day to
an equal number of times during each
preclude time-of-day effects. Scoring was accom-
With only one form of the ASVAB available,
Randolph AFB on the DIGITEK
plished
at
test security requirements dictated that alternate
DM-100 for Form 2X and on the Hewlitt-Pachrd
forms of the test be produced. Ceinsequently,
mark sense reader for Form 2XC.
ASVAB-2 and -3 were developed and were normed
using a geographically representative sample tested
Two samples were processed separately. Means,
specified Armed Forces Entrance and
at
standard deviations, and Pearson Product Moment
Examining Stations (AFEES). Beginning
in
for
matrices were computed
int ercorrelation
September 1973, ASVAB-2 will be used for high
twenty variables (these included years of educa-
school testing, with ASVAB-3 reserved as the joint
tion, AFQT, four composites from the Airman
services qualifying entrance test.
Qualifying Examination, data on completion of
specific high school courses, and all nine tests of
Form-2 of the ASVAB was standardized on the
the ASVAB). In addition, each of the two samples
DIGITEK answer sheets, but with the beginning of
was divided into eight subsamples based on AFQT
high school testing during the fall of 1973, the
decile, and means and standard deviations of the
DIGITEK DM-100 scoring capability at Randolph
ASVAB tests were computed for each resulting
AFB will he inadequate to handle the anticipated
subsample. Raw score distributions on the nine
load.
ASVAB tests were prepared for each of the two
A revised answer form, printed on a three part
samples.
IBM card. was designed to replace the answer
For each ASVAB test, a two-way classification
sheet, will, a resultant reduction in the amount of
analysis of variance was computed to assess effects
space allowed examinees for answer marking. The
of format and of the format x AFQT decile inter-
revised form was designed for scoring via the
action on scores. In this regard, it is noted that a
Hewlett-Packard model WAOO /D mark sense
for AFQT decile would be
effect
significant
reader also at Randolph AFB.
expected in all cases as a funrtion of correlation
Test norms, as
between AFQT and the
test.
A prat iinary study, to compare the two test
if
in-
appropriate
previously
established,
are
response 1:cording methods, was conducted at
consequential effects are found both for format
Lackland ifB. Texas, to determine whether there
interaction of AFQT decile with
for the
and
was test ne rin slippage due to answer form change.
format; a significant F ratio on either of these
effects, however, would suggest normative adjust-
ments as a consequence of format change.
II. PROCIDURI.
Two matched samples. each consisting of 248
III. RESULTS
male USAF basic airmen assigned to Lackland
AFB for processing and training, were admin-
Although the answer
sample (2X)
sheet
istered Fo..in 2 of the ASVAB. One sample was
contains fewer cases with education beyond high
administered Form 2XC (answer card format)
school than does the answer card sample (2XC),
while the ether was tested on ASVAB Form 2X
the average educational
levels are comparable.
(answer slit et format). The samples were matched
5
BEST COPY AVAILABLE
BEST COPY AVAILABLE
Answer sheet
sample airmen (2X) indicated
significant at the .01 level for word knowledge and
completion for more high school math and science
at the .05 level for coding speed. Interestingly; the
courses than did answer card sample cases (2XC)
interaction
largest
effect
would have been
even though sample
2X(' had more airmen
expected on coding speed since
is a closely
it
completing 12 or more years of education (Tables
timed test of simple clerical speed and
accuracy:
2 and 31.
consequently, a change in size of the
response
form had greater potential
for
affecting
The two samples were matched on AFQT score;
performance of "high
intelligence" and "low
each case in sample 2X was matched with
a sample
intelligence" subjects differentially. An inspection
2XC case with the same AFQT score. AFQT scores
of the decileby-decile
group means for the two
appear to be fairly evenly distributed across the
format groups (Table 9) reveals that in AFQT
deciles 20 through 90; there were no scores below
decile 20.29, mean coding speed performance of
20. Actual score range was 21 through 98. Enlist-
the answer sheet group is 6.16 raw score points
is not authorized for mental category V
ment
higher than that of the answer card group, and
( AFQT score 0 - 9) applicants, and recent policy
in decile 50.59, a difference of about the
that
has also excluded those scoring below 21 on
.same magnitude occurs, but in favor of the answer
AFQT. The percentage of airmen in the samples'
card group. Other decile group means for coding
AFQT categories I through IV appear to be quite
speed are approximately equal. In the
case of the
similar to the distribution for recent first term
word knowledge test, format group
means differ in
USAF airman accessions. (Unpublished data. See
favor of the card format group by
a little more
Table 4 for AFQT category distributions.)
than three raw score points in the AFQT decile
Further evidence of comparability of the two
50.59 and by a little over two raw score'points in
samples was found in the distribution of AQE
favor of the answer sheet format in the AFQT
percentile
While
statistically
scores.
significant
decile 60-69. Other decile group means are approx
differences were not
found between the two
imately equal.
groups on any of the four AQE aptitude
These two significant interaction effects would
composites, there were small mean differences in
suggest possible differential effect of test response
favor of the answer sheet sample; the largest of
format change on test performance as a function
these were on the
Administrative (2.27) and
of level of AFQT performance. In both format
Electronics (1.77) Ms. Distributions, means, and
samples, correlation between AFQT and coding
standard deviations of the AQE composites are
speed is quite modest (low .30s), and correlation
shown in Table 5.
between AFQT and word knowledge
in both
Minor differences.of no significance were noted
samples is in the low .50s. AFQT has served as the
among the intercorrelatiuns for the two samples.
standard against which ASVAB norms are
This serves as further evidence that format change
calibrated, but
with
modest
relatively
these
did not materially affect the battery (Table 6).
is possible that part of the within
correlations, it
AFQT decile difference washes out in conversions
Table 7 shows, for both samples, means and
utilizing the full samples (as a function of AFQT
for each ASVAB test.
deviations
standard
In
distri!itional
decile
addition, no
overlap).
this
However,
significant ASVAB test
mean
cannot be
eltarly determined front these data
differences were found; the largest difference (.62)
because of the truncation of the samples on AFQT
is on space perception. Thus, it can be seen that
the bottom 20 percent of
(approximately
the
the format change makes no difference in average
normative reference population is nut represented
performance.
here).
Outcomes of the nine two-way classification
Raw score distribution for both groups on all
analysis of variance problems are summarized in
nine ASVAB tests are presented in Tables 10 and
Table 8. As expected, highly significant F ratios
It
the ASVAB test
11.
noteworthy
that
is
were obtained
cases for AFQT deck,
all
in
distributions of the two format samples differ very
reflecting correlation between the test and AFQT.
little from each other.
In no instance was there a statistically significant
answer format main
effect;
this
finding
is
the quite small overall mean
with
consistent
IV. CONCLUSIONS AND RI.COMMI.NDATIONS
differences between the two sanyies (largest
difference was .62 on space percept ion). Signifi-
The results of this.study suggest compaable
cant format x ANT decile effects were found for
ASVAB score results using either the answer card
two of the nine
interaction was
This
tests.
6
or the
answer sheet. Presently estab-
Because of AFQT truncation in the data for
I )1G1.1
lished A:NABForm 2 norms may he satisfactory
this study,
determination of whether
final
a
tier answ, card administration. Although a signifi
conversions for the ASVAB tests are affected by
cant format by Al:QT decile interaction was found
answer form format change must await completion
on coding speed and word knowledge. overall
of a full standardization study. Data for such
a
means. s...indard deviations, and intercorrelations
study have been collectedand are being analyzed.
for the two format conditions are nearly the same.
is reconitnended that present conversions for
It
This suggests that, despite the significant inter-
ASVAB Form 2 continue in use until completion
action or; these two tests. conversion tables may
of this standardization study.
be affected very little if at all.
Table 1. Sample Distribution
Table 3. Number of Cases Reporting
by AFQT Decile
Completion of High School Mathematics
and Science Courses
Sample*
Total
AFQT Cecile
2X
Percent
2XC Sample*
2X
2 XC
Course
70
90.99
35
35
14.11
28
11.29
28
80,89.
56
154
Algebra
162
60
30
30
70-79
12.10
Geometry
95
106
60.69
27
54
27
10.89
Trigonometry
40
29
66
33
33
50-59
13.31
63
Chemistry
76
4049
66
33
33
13.31
Physics
34
42
30.39
74
37
37
14.91
25
25
50
20.29
10.08
'chi square tests revealed no significant differences
496
248
248
100.00
Total
between the two groups.
The two samples are evenly distributed across the
range of AFQT scores. No score lower than 21 nor higher
The two
than 98 was reported.
samples arc
pre.
Table 4. AFQT Mental Category
cisley matched on AFQT scores.
Distribution of Samples
Distribution of Educational
Table 2.
Sample*
Level
Mental
2X
Category
Total
2XC
Percent*
Sample*
Ed ucat Ion
Total
(years)
2 KC
2 X
(93.100)
46
23
9.27
23
II
8
3
5
9
(65.92)
80
9
4
80
32.26
160
13
42
10 23
III
19
28 37
65
(31.64)
11
46.77
116
232
116
147
292
12
145
IV
20
41
13
21
(10.30)
29
29
58
11.69
14
10
25
15
Total
496
248
248
99.99
6
15
3
3
6
16
7
1
*Percentage of Basic Airmen in each mental category
248
496
Tot al
248
is similar to the percentage in each category
into
enlisted
USAF during recent years (July 69 through September
11.96
11.76
11.86
Mean
71).
S.D.
1.25
1.16
1.21
'A t t . of difference between the sample means
revealed no agnificant difference (t = 1.83).
7