S313352 optimizing java device testing with automatic feature discovering
Open Source IAT - SPSP 2013
1. An Open Source IAT for
Online Data Collection
Winter Mason
Stevens Institute of Technology
2. Implicit Association Test (IAT)
The Implicit Association Test (IAT; Greenwald, McGhee, &
Schwartz, 1998) was devised as a way to measure implicit
attitudes towards categories of objects or people.
Since then it has been used extensively in the field
(McConnell & Leibold, 2001; Conrey, et al., 2003; Nosek, 2005,
2007)
Including online applications (c.f., Project Implicit:
http://projectimplicit.com).
However, until now, the means to run the IAT has required
commercial software such as e-Prime for laboratory studies
and has not been widely and freely available for online
research.
3. Open Source software
Open-source software is software that has been developed by
and/or shared with a community of developers
Firefox, Linux, R, etc., etc.
Based on the idea that sharing resources reduces unnecessary
duplication of effort and increases social efficiency
This idea should be applied to all aspects of science
Sharing methods and materials reduces duplication of effort and
facilitates replication
Sharing data allows transparency and reproducibility
Also see openscienceframework.org
4. Requirements
Experimenter: Has been successfully tested
A web server that runs PHP with
v4.0 or greater Mac OS 10.5, 10.6,
Windows XP, Vista
Ubuntu 11.
Participant:
A web browser that allows It has also been successfully
Javascript tested with
Mozilla’s Firefox
Google Chrome
Internet Explorer 8 & 9.
27. Enter a unique identifier for the
participant (a random string is pre-
filled). The output data file will be
associated with this ID. You can
also use this to match with a
Turker‟s submitted HIT.
28. The name of the
active template is
shown here.
30. The templates are stored in a folder with the same
name as the template, and image files are stored
in the “img” folder inside the corresponding
template folder.
32. If the participant gets it wrong,
the X appears until the correct
key is pressed.
33. Output
Trial Round Category Item Errors RT
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …
The output is stored in the “output” folder in the template‟s folder as
a comma-delimited file. The name of the file follows the pattern
[Template Name]-[Subject ID]-YYYY-MM-DD-HH-mm. For example,
Race-JohnDoe-2013-01-22-18-52.csv.
34. Output
Trial Round Category Item Errors RT
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …
There are 7 Trials in the standard IAT
35. Output
Trial Round Category Item Errors RT
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …
There are 20 or 40 round per trial,
depending on whether it is a
practice trial or not
36. Output
Trial Round Category Item Errors RT
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …
The character(s) used here to
indicate the category are defined in
the template.
37. Output
Trial Round Category Item Errors RT
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …
The item number refers to which
particular item was presented. The
order of item presentation is
randomized. The number indicates
its order in the template (starting
from zero).
38. Output
Trial Round Category Item Errors RT
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …
This is the number of errors the
participant made on that round.
39. Output
Trial Round Category Item Errors RT
0 1 F 4 0 560
0 2 M 3 0 432
0 3 F 0 1 913
… … … … … …
This is the participant‟s response
time in milliseconds for that item on
that round. The final IAT score is
determined by these reaction times.
40. Validation
The standard deviation of
system error was 29.4ms, with
fewer than 4% larger than 50ms
and the largest error recorded
at 167ms.
The typical standard deviation
of a user's reaction time is
approximately 300ms, as
reported in Greenwald, et. al
(1998, 2003) and observed in
the replications reported here.
The error introduced by the
system is an order of
magnitude less than user
error.
41. Replicating prior work
63 participants recruited from Amazon‟s Mechanical Turk
46.7% female
median age of 29 (mean age 31.8)
median household annual income less than $30k
median education of a Bachelor’s Degree
55.5% from India, 38% from the U.S., and the remainder were
from Pakistan, Lithuania, Algeria, and Sweden.
42. Flowers-Insects IAT
Congruent first Incongruent first
(Flowers-Good / Insects-Bad) (Flowers-Bad / Insects-Good)
44. Future extensions
Allow easy duplication of templates in experimenter
interface
Data storage in a database instead of text files
Automatic aggregation and display of results in
experimenter interface
Suggestions? Requests?