1. Predicting Technical Aptitude:
Relations between Predictor Variables,
Technical Aptitude and Technical
Training Performance
(O.N.R. Contracts Nr. N00014-10-M-0087 & N00014-10-C-0505)
Ryan Glaze & Martin J. Ippel
CogniMetrics, Inc., San Antonio, TX
1
Paper presented at the 53rd Annual Conference of the
International Military Testing Association, Bali (Indonesia).
October 31 – November 4, 2011
2. Preview
Analysis I
Overview of ASVAB and Technical Knowledge Tests
Introduction to Technical Aptitude
Proposed Model
Methods, Results, and Discussion
Analysis II
Incremental Validity Analysis Predicting Technical Aptitude
with ASVAB Selection Composites and the ITAB
Methods, Results, and Discussion
2
3. ASVAB
US Armed Forces must select, classify, and train
personnel to work in highly technical work
environment
ASVAB and Technical Knowledge Subtests used for
technical Navy Ratings
General Science (GS)
Mechanical Comprehension (MC)
Auto Shop (AS)
Electrical Information (EI)
3
4. Technical Knowledge Subtests
Technical Knowledge Subtests have several
limitations:
Represent arbitrary and limited sample of domain
Measures Technical Knowledge but not Technical
Skills
Provide modest predictive validity
4
5. Technical Aptitude
Technical Aptitude (Ippel & Glaze, 2011)
Technical Knowledge Aptitude (TKA)
Aptitude to learn technical knowledge (concepts)
Derived from performance on eight common knowledge tests
Technical Skill Aptitude (TSA)
Aptitude to learn technical skills
Derived from performance on seven common skill tests
5
6. Technical Aptitude
Technical Aptitude (TA)
Represents a construct with a short logical
distance to criterion performance in Apprentice
Technical Training (ATT) performance
Will be used to assess construct validity of TK
subtests
6
16. Method
410 Navy Recruits participating in A.T.T. program
ASVAB
TK Subtests: GS, MC, AS, EI
AFQT
Considered a measure of crystallized intelligence
A.T.T. post-training test scores
Eight common knowledge tests
Seven common skill tests
Dichotomously scored (Pass/Fail with 70 point cut score)
16
17. Method
Technical Knowledge Aptitude
IRT-based ability estimate derived from common
eight knowledge tests
Technical Skill Aptitude
IRT-based ability estimate derived from common
seven skill tests
17
18. Results: First Knowledge Test
18
RMSEA = 0.000
PClose = 0.886
WRMR = 0.040
Technical
Aptitude
AFQT
GS MC AS EI
Post-Test
20. Results: First Knowledge Test
20
Technical
Aptitude
AFQT
GS MC AS EI
Post-Test
.137.130
.175
.530*
21. Results: First Knowledge Test
21
Technical
Aptitude
AFQT
GS MC AS EI
Post-Test
.217* .017 -.119 -.021
22. Results: All Knowledge Tests
22
MeanRMSEA = 0.000
MeanPClose = 0.868
MeanWRMR = 0.039
Technical
Aptitude
AFQT
GS MC AS EI
Post-Test
23. Results: All Knowledge Tests
The paths between Technical Knowledge
Aptitude, Technical Knowledge Subtests,
AFQT were nearly identical for all knowledge
tests
Focus will be on:
AFQT → Post-test
TA → Post-test
23
24. Results: All Knowledge Tests
24
Technical
Aptitude
AFQT
GS MC AS EI
Post-Test
.530* .084
25. Results: All Knowledge Tests
AFQT was related to all Technical Knowledge
Tests, but not post-training test scores
Technical Knowledge Aptitude was related to
post-training test scores, but not Technical
Knowledge test scores
Indirect effects of Technical Knowledge
Aptitude on post-training test scores via TK
subtests were not significant
25
26. Results: First Skill Test
26
RMSEA = 0.024
PClose = 0.477
WRMR = 0.187
Technical
Aptitude
AFQT
GS MC AS EI
Post-Test
27. Results: First Skill Test
27
Technical
Aptitude
AFQT
GS MC AS EI
Post-Test
.066
.373*.536* .179* .267*
28. Results: First Skill Test
28
Technical
Aptitude
AFQT
GS MC AS EI
Post-Test
.218*
.174* .032 .064
29. Results: First Skill Test
29
Technical
Aptitude
AFQT
GS MC AS EI
Post-Test
.076 .062 -.029 .062
30. Results: All Skill Tests
30
MeanRMSEA = 0.024
MeanPClose = 0.477
MeanWRMR = 0.187
Technical
Aptitude
AFQT
GS MC AS EI
Post-Test
31. Results: All Skill Tests
The paths between Technical Skill Aptitude,
Technical Knowledge Subtests, AFQT were
nearly identical for all skill tests
Focus will be on:
AFQT → Post-test
TA → Post-test
31
32. Results: All Skill Tests
32
Technical
Aptitude
AFQT
GS MC AS EI
Post-Test
.218* .066
33. Results: All Skill Tests
AFQT was related to all Technical Knowledge
Tests, but not post-training test scores
Technical Skill Aptitude was related to post-
training test scores, but not Technical
Knowledge test scores
Indirect effects of Technical Knowledge
Aptitude on post-training test scores via TK
subtests were not significant
33
34. Results: Technical Aptitude
Technical Knowledge Aptitude was only
slightly related to Technical Skill Aptitude
(r = .136)
Technical Aptitude was related to post-
training test scores, but not Technical
Knowledge test
34
35. Analysis II
Results of Analysis I suggest Technical
Aptitude was related to post-training test
scores, but Technical Knowledge was not
Analysis II seeks to identify predictors of
Technical Aptitude
Current predictors of training performance consist
of various ASVAB Selection Composites (ASC)
I.T. Aptitude Battery (ITAB) was designed to
measure technical aptitude
35
36. Method
Two selection composites that the Navy
currently uses to assign recruits to ratings
ASC01 consists of WK, PC, AR, and MC
ASC02 consists of MK, AR, GS, and EI
ITAB consists of two fully interactive tests
Hidden Target Test
Battery Test
36
