Powerpoint slides from Jill Castek's presentation at COABE 2015 Conference on "Using PIAAC's Education and Skills Online to Examine Adults' Skills Locally".

1. Using PIAAC's Education
and Skills Online to
Examine Adults' Skills
Locally
Jill Castek, Amy Honisett, & Andrew Pizzolato
Portland State University & Multnomah County Library
Oregon, USA

2. Using PIAAC's Education
and Skills Online to
Examine Adults' Skills
Locally
With contributions from Cindy Gibbon, Vailey Oehlke, and
Matthew Timberlake
Multnomah County Library, Portland, OR USA
And Dr. Stephen Reder, Portland State University
Portland, OR USA

3. This work is supported by
 Advancing Digital Equity in Public Libraries:
Assessing Library Patrons’ Problem Solving in
Technology Rich Environments (LG-06-14-0076)
 Tutor-Facilitated Digital Literacy Acquisition
in Hard-to-Serve Populations: A Research
Project (LG-06-11-0340-11)
COABE, 2015 Denver, CO

4. What is the Program for the International
Assessment of Adult Competencies (PIAAC)?
COABE, 2015 Denver, CO
Tool designed to…
measure the skills of the adult working-age population
both nationally and internationally
help researchers examine and analyze what conditions
and factors impact skills growth, maintenance, or loss
over a working-age life cycle
provide international comparison to better understand
U.S. global competitiveness
benchmark how well education and training systems are
meeting emerging skill demands

5. This session is different
Not analysis of PIAAC data
Innovative partnership program
Use of PIAAC’s Education and Skills Online
 A valid and reliable assessment tool
 In an assessment of library patrons
Think about Problem Solving in Technology Rich
Environments (PSTRE) in relationship to libraries
Advancing Digital Equity in Public Libraries: Assessing
Library Patrons’ Problem Solving in Technology Rich
Environments
COABE, 2015 Denver, CO

6. Welcome & Introduction
Literacy, Language, and Technology Research Group
•within an Applied Linguistics department
•commitment to building literacy and language pathways to
social and economic justice
Multnomah County Library
enriches lives by fostering diverse opportunities for all
people to read, learn and connect
upholds the principles of intellectual freedom and the
public's right to know
provides people of all ages with access and guidance to
information and collections that reflect all points of view

7. Welcome & Introduction
What are your roles in adult
education?
Teacher?
Program administrator?
Policy maker?
Researcher or evaluator?
What settings/systems do you
work in?
• K-12?
• Community college?
• CBO?
• Government?
• University?
COABE, 2015 Denver, CO

8. How did we come to collaborate?
Shared Commitment to Promoting Digital Equity
Connecting National Efforts with Local Efforts
Literacy, Language, and Technology Research
Group
Broadband Technology Opportunities Program
(BTOP)
Digital Literacy Acquisition Research Project
Multnomah County Library
history of supporting life long learning
Major hub for technology access and training
over one million Wifi sessions last year
COABE, 2015 Denver, CO

9. What do the PIAAC data reveal? (NCES, 2013)
COABE, 2015 Denver, CO
U.S. adults
scored below the PIAAC international average in literacy,
near the bottom of the rankings in numeracy,
shared the lowest ranking in problem solving in
technology-rich environments.

10. Did you know?

11. piaacgateway.com

12. Why are PIAAC data relevant for Libraries?
Libraries are institutions that
Support interest-driven lifelong learning
Are hubs for Internet access and digital
literacy training
Are a consistent presence across the
country
Deliver services using technology

13. Purpose of this Project
Extend national
work on digital
literacy acquisition
to inform local
efforts
Bring libraries into
the PIAAC
conversation
Maximize resources
and meet
community needs
around lifelong
learning and access
Education and Skills Online: Problem Solving in Technology-rich environments

14. Problem Solving
in
Technology-rich
Environments
(PSTRE)
a fully
computerized
measure of
cognitive and
non-cognitive
skills
incorporates
flexibility and
adaptability to
specific learner’s
knowledge and
skill level provides reliable
and valid
measures of skills
associated with
work, home and
the community
Tool Available August 2015

15. What is Problem Solving in Technology-Rich
Environments PSTRE?
COABE, 2015 Denver, CO
What does it measure?
Cognitive skills: planning, reading, thinking, searching, skimming,
scanning, information literacy, critical evaluation of information, and
communication of ideas
What doesn’t it measure?
Affective domain: personal relevance, resilience, stick-with-it-ness,
focus in terms of carrying the task through multiple materials/
multiple screen navigation
Basic digital literacy skills
What do results tell us?
Three levels of proficiency

16. Three Levels of PSTRE proficiency
Sort emails into
pre-existing
folder using given
criterion
Respond to a
request by
locating
information in a
spreadsheet and
e-mailing the
requestor
Manage requests
to reserve
meeting room
using a
reservation
system. Discover
schedule conflict,
e-mail to decline
the request
Level 1
COABE, 2015 Denver, CO
Level 2 Level 3

17. Job Search
Developing a
contact list
Keeping Track of
Applications
Evaluate a job
posting.
Is this
appropriate for
my interest and
experience?
Search online
databases
Produce a resume
Respond to job
ads by navigating
through online
systems
Level 2Level 1 Level 3
Denver, COCOABE, 2015

18. Problem Solving Item: Level 2
From Education and Skills Online Sample Items

19. Digital Equity in Libraries Project
Mapping common library tasks
on to the PSTRE framework
Problem Solving
in
Technology-rich
Environments
Assessing
Library Patrons
Assessing
Library Staff

20. Level 1:
• complete tasks in which the goal is explicitly stated
and for which the necessary operations are
performed in a single and familiar environment.
• solve problems whose solutions involve a relatively
small number of steps, the use of a restricted range
of operators, and a limited amount of monitoring
across a large number of actions.
Problem Solving: Level 1 Description
From p. 90 of OECD Skills Outlook 2013: First Results from the Survey of Adult Skills

21. At Level 2, adults can:
• complete problems that have explicit
criteria for success, a small number of
applications, and several steps and
operators.
• monitor progress towards a solution and
handle unexpected outcomes or impasses.
Problem Solving: Level 2-Description
From p. 90 of OECD Skills Outlook 2013: First Results from the Survey of Adult Skills

22. Adults at Level 3 can:
• complete tasks involving multiple
applications, a large number of steps,
impasses, and the discovery and use of ad
hoc commands in a novel environment.
• establish a plan to arrive at a solution and
monitor its implementation as they deal
with unexpected outcomes and impasses.
Problem Solving: Level 3 Description
From p. 89 of OECD Skills Outlook 2013: First Results from the Survey of Adult Skills

23. Attempts to Group Individuals
and Items on the Same Scale
Below Level 1
Level 1
Is this
feasible?
Level 2
COABE, 2015 Denver, CO
Level 3

24. Cognitive Skills Tech Proficiency
sorting e-mails into pre-
existing folders
Remain focused through
several steps and operators
monitor progress towards a
solution and handle
unexpected outcomes or
impasses
Extract information using a
sort function
Making use of the available
tools
Planning & Self-
organizing
Setting Goals and
Monitoring Progress
Using Information
Acquiring and Evaluating
Information
COABE, 2015 Denver, CO

25. Tasks Involved Job Search
• Planning & Self-organizing
• Develop a keyword list
• Keeping track of applications
• Develop A Contact List
• Setting Goals and Monitoring Progress
• Searching Online
• Signing Up for a Class (increase skills)
• Using Information
• Produce a resume
• Apply for a job
• Acquiring and Evaluating Information
• Evaluate job posting
• Is the post legitimate?
• Does it match my skill set?
Proficiency Level PSTRE
• Below Level 1
• Level 1
• Level 2
• Level 3

26. Consideration of Different Perspectives
Library
Patron
Library
Staff
PST-RE
Framework
Developers
Do skills assessed fit with what
we know library
patrons need to be able to do?
Do skills assessed fit with
library
patron’s experiences?

27. Common library tasks
Job search
Homework help
Obtaining material to watch/read
Family history
Volunteering in the library or in the
community
Find a Class/program and participate
in it
Finding medical information
Finding social services
Recreational Internet use
Taxes
Implications for Programming
COABE, 2015 Denver, CO
How do different groups conceptualize the
skills needed to accomplish these tasks?
Library
Experts:
Staff and
Educators
Library
Patrons
PSTRE
Framers

28. Next Steps
Collect Data • Explore the assessment
Analyze
Results
• What does it tell us?
Interview
Patrons • Triangulate
Plan for
programming

29. Learn More About our Project
www.pdx.edu/linguistics/pstre

30. Questions & Connections
Jill Castek – jcastek@pdx.edu
Amy Honisett - amyh@multco.us
Andrew Pizzolato - apizzola@pdx.edu
COABE, 2015 Denver, CO