This document discusses using open badges as a potential solution for evaluator certification by the American Evaluation Association (AEA). It outlines challenges with certification, describes the Canadian Evaluation Society's credentialed evaluator designation as a working example, and proposes how an open badge system could address AEA's needs. Key benefits of open badges mentioned include transparency, flexibility, rigor, and sustainable management. The document provides examples of open badge structures and metadata to illustrate how badges could recognize different levels of evaluation knowledge, skills, and experience.

1. Using Open Badges as a Certification Solution for Evaluators
Randall S. Davies
Daniel L. Randall
Richard E. West
BYU
Image by photosteve101. Used under CC BY License.

2. The Issue of Evaluator Certification
In 1999 the topic of certification was fervently debated
Not everyone feels the AEA needs an evaluator certification
Others feel a certification process would be valuable, and that it may
become required for practitioners working with government
organizations (Smith et al., 2011)
AEA is no closer to having an evaluator certification than we were a
decade ago. Many seem to have concluded that an evaluator
certification system may be overly complicated and controversial
(Morris, 2011)
AEA

3. Challenges to Certification
Credentials, Certifications, and Licenses
An Evaluator Certification would evaluate an individual’s
Knowledge, Skills, and Experience, verifying that he or
she has attained a certain level of expertise.
Any solution AEA might adopt must be
– Transparent (Clear Expectations)
– Rigorous BUT Flexible and Efficient
– Sustainable
AEA

4. Working Examples of an Evaluator Credentialing Designation
Canadian Evaluation Society’s (CES)
Credentialed Evaluator (CE) designation
This is a designation is not a certification of evaluation
proficiency or a license to practice evaluation.
It simply verifies “the holder has provided evidence of
the education and experience required by the CES to be
a competent evaluator” (http://www.evaluationcanada.ca, np).
AEA

5. CES Credentialed Evaluator Designation
The CE designation is voluntary.
Candidates must have at least two years of evaluation-related
work experience within the last ten years, with letters of
reference from clients.
They must pass a test requiring a series of narratives (each
150 words or less) adequately addressing at least 70% of
the specified evaluator competencies in each of five
domains.
AEA

6. CES Credentialed Evaluator Designation, Cont.
Members of a credentialing board consider the
application and award the designation.
In addition, credentialed evaluators are required to
complete ongoing training (40 hours every three
years) in order to maintain their status.
This application and renewal process is implemented
though an online system designed to manage
applications and candidate accounts, with a fee to
cover administration costs.
AEA

7. Issues & Concerns AEA Must Consider
Training Concerns:
– Degrees vs training, Experience without a degree
(capacity building). What combination of training,
Skill, and Experience is needed.
Assessment Concerns:
– Certification Exams vs Credentials, Validity of
assessment.
Administration Issues:
– Costs involved in establishing and maintaining a
system
AEA

8. Potential for an Open Badges Solution
Transparency:
– Badges communicate specifics, Meta-data,
Flexibility:
– Modification is relatively easy, Skill subsets can be defined,
badges can be added or revised individually.
Rigor:
– Individuals Authorized to Award Badges, Allows for levels of
proficiency to be identified (basic – advanced)
Management Sustainability:
– Top down design, Automation of much of the process
possible, Division of labor model
AEA

9. What are “Open Badges”?
Images from OpenBadges.org

10. Merit Badges and Digital Badges
Boy Scout Merit Badges
Digital Badge
- Acknowledge accomplishment
- Display skills gained
- Motivation
Same Benefits as physical badges -
Typically not sharable -
Gamification -

11. Khan Academy Badges (not Open!)
Retrieved from https://www.khanacademy.org/badgeson 11/30/13.

12. Duolingo (not Open Badges)
Screenshot from the Duolingo app on an Android Tablet. 1/24/14.
Encourages daily
practice
Completed
badge
Progress
indicators
Competition
with others
is optional
Previously earned
badge not at full
strength

13. Open Badges
Open Badges
Same Affordances as Digital Badges, Plus:
- Uses Open Badge Infrastructure (OBI)
- Display badges via web
- Metadata (Criteria and Evidence links)

14. Open Badge Metadata

15. Mozilla Backpack

16. Backpack Collection
Multiple collections can
be created.
Collections can remain
private or can be made
public and shared.

17. Mozilla’s Vision of Credentialing
Learn and gain skills in formal
and informal settings
Collect and Display Badges
New Opportunities

18. Applications
Gamification: breaking education into achievable levels
where progress is recognized and rewarded
Duolingo, Khan Academy, many others
Recognizing expertise within a community
Coding communities, translation communities
Marrying informal and formal learning
Concordia, WGU
Providing more authentic credentials, tying evidence to
criteria to credential
IPT Ed Tec

19. Strong criteria
Key Principles of Badging
• A badge’s value is based on the rigor of the criteria for
earning it
Easy submission
• Make it simple for people who deserve a badge to get it
Web-based evidence
• Portfolios, links, Google documents, etc. Allows for evidence
to be viewed with a badge
Skilled assessments
• It is essential to trust the person making the assessment
Open and shareable badge collections
• What’s the point if you can’t share your badges in multiple
ways?

20. iMovie
Personal
Tech
Technology
Integration
One Possibility for an AEA Badge System
AEA

21. Early Badge System by Mozilla
Mozilla’s Webmaker badges plan used the
“constellation” concept in which smaller
badges are earned and added together to
receive a higher level badge.
Badges image adapted from the Mozilla Open Badges Project. Retrieved from http://erinknight.com/post/29830945702/webmaker-badges on 11/9/12.

22. Google
Sites
IPT EdTec Badge System
Personal
Tech
Choice
1
Choice
2
Choice
3
Student selected Internet
Communication Technology
Student selected
Multimedia Technology
Student selected
Personal Technologies
Additional
Concepts
Mobile
Learn
Internet
Safety
Copy
right
iMovie
Lower level badges are not
issued for these projects
Project level badge not
issued for these
additional concepts
Educational
Technology
Course
Level Badge
Project
Level Badge
Lower
Level Badges

23. iMovie
First Iteration
Personal
Tech
Theory
Skill
Experience
Technology
Integration
Certified Evaluator
Level Badge
2nd
Level Badge
3rd
Level Badges
Possible AEA Badge System

24. Anatomy of a CE Badge
(theory, experience, skill, or
green for the CE badge). Center of badge
Pips act as stars:
1 = Basic
2 = Intermediate
3 = Advanced
Each badge
indicates what category
the badge is from
Outer ribbon
signifies this badge
is the “Certified
Evaluator” (CE)
badge
is reserved for
images to
individualize
badges

25. Pips and the CE Badge
At least 1 pip from each category
7 - 9 pips earned from 2nd level
Basic
Intermediate
Advanced
4 - 6 pips earned from 2nd level

26. To Learn More…
Randall, D. L., Harrison, J. B., West, R.E. (2013). Giving credit where credit is
due: Designing Open Badges for a technology integration course.
TechTrends, 57(6), 88-95. doi:10.1007/s11528-013-0706-5
View our presentation at the Open Education Conference by clicking here.

27. Thank You
Randall S. Davies – randy.davies@byu.edu
Daniel L. Randall – dan.randall26@gmail.com
Richard E. West – rickwest@byu.edu
Contact us with Questions