In this white paper, I lay out a framework for designing and developing large-scale curricula for workplace learning. I've successfully helped organizations tame their monster curriculum needs with this approach.
1. Monster Curriculum!
Part 1: A Curriculum Development Framework
White Paper
Author: Gus Prestera, PhD
Date: January 15, 2012
Citation: Prestera, G.E. (2012). Monster Curriculum! Part 1: A
Curriculum Development Framework (white paper).
Prestera FX, Inc.: Glen Mills, PA (retrieved from
www.presterafx.com).
PRESTERA FX, INC. | P.O. Box 5, Glen Mills, PA 19342 | 484.343.6474 | WWW.PRESTERAFX.COM
2. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
TABLE OF CONTENTS
Abstract ......................................................................................................................... 2
What is a Monster Curriculum?.................................................................................... 3
Types of Curricula ....................................................................................................................................... 3
A Curriculum Map....................................................................................................................................... 4
A Good Curriculum is More Than a Map...it’s about Proficiency ............................................................... 5
So What Then is a Good Curriculum? ......................................................................................................... 6
What’s so Different? ..................................................................................................... 7
Size Matters ................................................................................................................................................ 8
Complexity Demands Attention ................................................................................................................. 8
Importance is a Double-Edged Sword ........................................................................................................ 8
Shelf Life Requires Long-Term Commitment .............................................................................................. 9
A Curriculum Development Framework .................................................................... 10
Phases ....................................................................................................................................................... 11
Analysis Phase ...................................................................................................................................... 11
Design Phase......................................................................................................................................... 11
Buildout Phase ...................................................................................................................................... 11
Rollout Phase ........................................................................................................................................ 11
Maintenance Phase .............................................................................................................................. 12
Functions .................................................................................................................................................. 12
Program Management ......................................................................................................................... 13
Curriculum Design ................................................................................................................................ 13
Curriculum Development ..................................................................................................................... 16
Change Management ........................................................................................................................... 18
Marketing Communications ................................................................................................................. 19
Program Operations ............................................................................................................................. 20
Program Evaluation .............................................................................................................................. 21
What’s Next? ............................................................................................................... 22
Updated: 1/15/2012 PRESTERA FX Page 1
3. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
ABSTRACT
It’s easy to assume that building a curriculum, or any other large-scale corporate training program consisting of
multiple courses, is just like building a stand-alone course, only with multiple repetitions of the development cycle.
Those of us who have seen the beast up close know that there are many differences between building a course, or
even series of courses, and building a large, integrated, cohesive curriculum. Some of these differences relate to the
size, complexity, importance, and shelf life of a monster training curriculum. In this paper, we will explore those
differences and their implications for training professionals. We will use that understanding to build a curriculum
development framework that addresses the unique challenges a monster curriculum represents, while still being able
to scale down to smaller curriculum initiatives. This framework will address the entire process: analysis, design,
buildout, rollout, and maintenance. It will also consider the various ongoing functions that are needed to support the
curriculum, including program management, curriculum design, and change management. Future papers will dive
deeper into each element of this framework, so we ask the reader to consider this paper a foundation upon which we
can build an understanding of best practices.
Updated: 1/15/2012 PRESTERA FX Page 2
4. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
WHAT IS A MONSTER CURRICULUM?
A course is typically associated with one set of instructional goals and one subject matter. Often, we
associate a course with the knowledge set of a single expert—typically, that expert is the instructor. A
course could last 30 minutes, a day, or maybe even a few months, but usually not much more than that.
You might build a series of courses and group them into a program of related courses, such as a selling
skills program. A curriculum, on the other hand, is designed around the needs of a particular job rather
than a particular skill. It involves many instructional goals and different subject matters pressed into
service to suit the needs of a given audience. Depending on the depth and breadth of a curriculum, it
could take months and sometimes years to complete.
TYPES OF CURRICULA
There are three common breeds of curricula found in workplace learning:
Job-Specific Onboarding: A curriculum that teaches people new to a position everything they
need to know in order to reach proficiency in that position and be allowed to operate
independently in that role.
Leadership Development: A curriculum that prepares future managers to succeed in their
leadership responsibilities or one that trains current managers on their leadership
responsibilities.
Certifications: A curriculum that officially certifies a learner’s proficiency in a particular discipline,
such as a Six Sigma Black Belt certification or a Certified Professional in Learning and
Performance (CPLP) designation.
Other types may be more specific to certain industries, such as a Corporate Integrity Agreement (CIA)
Curriculum, which is commonly seen in the health sciences field. A new process, system, or method may
also call for a large training program that borders on being considered a curriculum. Typically, though, a
program teaches workers a specific set of things that they need to know; whereas a true curriculum
attempts to teach them all the things they need to know in order to be proficient in a particular job. A
new machinist working at an assembly plant may be required to learn all of the company’s policies and
procedures. In addition, he/she may need to complete the OSHA-mandated safety program, which could
consist of a series of courses. The safety program is one element, though, within a broader New
Machinist Curriculum that could consist of stand-alone courses on operating a particular piece of
equipment or entire programs that cover every aspect of the plant’s operations. In terms of a hierarchy,
the course is a basic building block; multiple courses can make up a program; and multiple programs and
courses can contribute to a broader curriculum.
Key Point: A curriculum is made up of building blocks,
consisting of individual courses and
groups of courses, which are often called programs.
Updated: 1/15/2012 PRESTERA FX Page 3
5. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
A monster curriculum often pulls from many different sources within a training department’s body of
resources, and often from outside the training department. For example, a new First Time Supervisor
leadership development curriculum could draw courses from an already-existing communications skills
program that includes courses on active listening, conducting presentations, and running effective
meetings. It could also draw from a coaching program that includes courses on goal setting, providing
feedback, and conducting difficult performance conversations. Those programs and their courses could
be utilized across a variety of different audience-specific curricula. The coaching program could, for
example, be used for the First Time Supervisor curriculum, and it could also be used for the Developing
Future Leaders curriculum. A curriculum incorporates whatever serves the needs of a given audience.
Because of this propensity to draw from existing resources, instructional designers must often conduct a
curriculum analysis to see what’s available and what can be utilized.
A CURRICULUM MAP
One thing that all curricula worthy of the name have in common is that they are audience-specific. When
we speak of a curriculum, we refer to it in terms of the audience: the Store Manager Curriculum, the
Franchisee Curriculum, the Shop Floor Supervisor Curriculum, or the Nanofabrication Lab Assistant
Curriculum. A curriculum is not designed around a body of content but rather around the needs of a
particular audience.
Key Point: All curricula worthy of the name
are audience-specific.
As we think of a curriculum framework, it’s helpful to consider courses as building blocks and programs as
groups of building blocks that have the same color or shape. Those building blocks can be organized,
sequenced, and tailored to different audience-specific curricula. Visually, that can start to look like Figure
1, where courses from the different programs are listed along the right side of the table; each column
represents the needs of a different audience; and courses/programs are selected based on their
alignment with those audience-specific needs. Taken altogether, this begins to form a curriculum map.
Those different audiences could be different levels of managers, ranging from future leaders to first-time
supervisors, to middle managers, to department heads, to business unit heads, to C-suite executives.
Some readers may object that even those audience segments may be too broad: maybe your organization
has four variations of first-time supervisors depending on which part of the business they operate. In any
case, you are probably starting to observe one of the critical aspects of curriculum design, differentiating
audience needs, or what instructional designers typically refer to as a job analysis.
Updated: 1/15/2012 PRESTERA FX Page 4
6. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
Figure 1. Framework for a curriculum map that organizes building blocks by audience needs
Audience W Audience X Audience Y Audience Z
Program 1
Course A
Course B
Program 2
Course C
Course D
A GOOD CURRICULUM IS MORE THAN A MAP...IT’S ABOUT PROFICIENCY
Based on what we’ve discussed so far, we might define a curriculum as a collection of courses and
programs organized in a sequence around the needs of a particular audience. That may be an adequate
working definition for what a curriculum IS, but it does not begin to capture what a good curriculum
should be. For that, we need to look at the intent of a curriculum.
When an organization goes about creating a curriculum for a particular job, it is making a sizable
investment of time, money, focus, and other resources. Why? The organization makes this investment
with the expectation that training the individual worker in some systematic way is more effective than
simply allowing that worker to learn on the job through trial & error, osmosis, mimicry, or whatever other
techniques they pick up in the wild. We know that without formal training, workers can become
proficient, assuming they have good role models, adequate resources, and good leaders: it just takes time
and patience. However, time and patience usually translate into mistakes that cost the organization in
terms of low productivity, lost revenue, increased waste, reduced quality, and unhappy customers. The
more time and patience are given, the greater the cost to the organization. Therefore, the intent of the
curriculum—from a business standpoint—is to minimize the time and costs associated with getting a
newbie from the point where their performance on-the-job is a detriment to the organization to a point
where that individual is contributing to the organization positively with minimal errors and acceptable
productivity. Instructional designers refer to that end point as proficiency. In simplistic terms, the point
of a workplace curriculum is to minimize time-to-proficiency and cost-to-proficiency. What can we do to
get an individual proficient on-the-job in the fastest, most efficient way possible? That’s the question that
should drive all decisions that an instructional designer makes when building a curriculum.
Updated: 1/15/2012 PRESTERA FX Page 5
7. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
Key Point: The point of a workplace curriculum is to minimize
time-to-proficiency and cost-to-proficiency,
so instructional designers need to consider: what is the
fastest, most efficient way to get a worker proficient on-the-job?
Note here that proficient on the job is not the same as proficient in the classroom, or what instructional
designers often refer to as mastery. Passing a paper-and-pencil test is, for many job skills, what passes for
mastery these days, but that is not sufficient evidence that the worker is proficient in doing the work on
the job. Even so, many curricula are built to stop at mastery and abandon learners to their own devices,
once they have passed a paper-and-pencil test or once they have sat through a certain amount of seat
time. We criticize schools for this, yet in the corporate sector we have copied the same practice.
If the point of a corporate training curriculum—the reason why management invests in training—is on-
the-job proficiency, then on-the-job proficiency is what we need to measure and use as our end point. As
an example, one of my financial services clients developed an authentic assessment of a financial
advisor’s critical skills, including the ability to analyze a client’s portfolio and offer sound
recommendations. New financial advisors—after completing their boot camp training—must
demonstrate that they can perform their real-world job tasks under realistic conditions before they can
start talking with real clients. Even then, those rookie financial advisors are still operating within the
curriculum, under close scrutiny, and are on probation until they can demonstrate proficiency on-the-job.
When your monster curriculum has teeth like that, it can truly have an impact on real-world performance.
SO WHAT THEN IS A GOOD CURRICULUM?
With time- and cost-to-proficiency in mind, below are some characteristics that distinguish effective
curricula from ineffective ones:
1. Lean: Contains no fluff…all content and learning activities are designed to prepare the learner for
on-the-job performance, and anything not relevant or essential for that purpose has been
stripped out of the curriculum.
2. Practice: Decades of research confirm that practice does indeed make perfect, and in order to
reach proficiency, learners must have ample opportunity to practice applying what they’ve
learned to real-world tasks.
3. Context: Both the presentation of content and the practice exercises need to contain real-world
examples, scenarios, challenges, and solutions, so that learners are prepared to go out and use
what they’ve learned on-the-job more quickly.
4. Relevance: It’s not enough to have real-world context embedded into the learning: that context
needs to be relevant for that particular audience and their work. Greater relevance leads to
faster knowledge and skill acquisition; greater knowledge retention; faster recall; and greater on-
the-job transfer.
Updated: 1/15/2012 PRESTERA FX Page 6
8. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
5. Integrated: Contains overlaps and points of intersection between courses, so that individual
courses feel part of a larger whole. Though too much can create fluff, some redundancy among
courses is important as it can reinforce critical information and skills and create synergies within
the curriculum that would otherwise not be possible.
6. Progression: One course or program builds on the knowledge of previous courses and programs,
creating a feeling for the learner that there is a logical progression to the curriculum.
Progressions help to accelerate learning, because the learner is building new knowledge onto a
strong foundation.
7. Cohesive: Content consistently echoes the same general school of thought and seems to be
written in one voice, so the learner is able to organize, store, and later retrieve the information
more rapidly and more accurately. Cohesion also helps to speed up learning.
8. Technology-Smart: Though instructional technologies can be cool and fun, the real value that
they bring to workplace learning is the potential to speed up learning and reduce the cost of
learning. Some can speed up learning by enabling learners to get more practice repetitions in a
safe environment and more individualized feedback. They can reduce cost by reducing travel,
and by enabling the expert/instructor to reach a larger audience at once (scalability). The smart
use of technologies is another hallmark of well-designed curricula.
9. Synthesis: A curriculum involves a wide variety of content domains focused through the lens of
one job. Therefore, it’s critical that the curriculum offer ample opportunities for learners to
synthesize all they have learned—across all of the different courses and programs—into one
integrated body of knowledge and skills.
10. Authentic Assessments: The criteria for proficiency need to be embodied in the form of an
authentic assessment that enables learners to demonstrate that they can do the job under real-
world conditions. This enables us to be more confident that the learner is truly proficient and it
gives the curriculum a razor-sharp clarity of purpose.
WHAT’S SO DIFFERENT?
Throughout this paper, we have asserted that a curriculum is different than a program or course. A
course is a building block that can be used for multiple audiences. A program is a collection of courses
centered on a particular subject (e.g., a product training program or a compliance training program). The
curriculum is a collection of courses and programs filtered through the needs of a particular audience.
There are some other characteristics that distinguish a curriculum, and they are significant because they
dramatically influence how someone needs to go about the work of building a curriculum. Specifically,
we’ll discuss four important characteristics: size, complexity, importance, and shelf life.
Key Point: Size, complexity, importance, and shelf life differentiate
curricula from training programs and courses.
Updated: 1/15/2012 PRESTERA FX Page 7
9. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
SIZE MATTERS
When you are trying to teach workers everything they need to know in order to be proficient in their jobs,
each job-specific curriculum can get rather large. The number of individual courses involved could
number in the dozens, sometimes hundreds. To the extent that many of those courses need to be built
from scratch (some will be re-purposed from other curricula and some may be sourced off-the-shelf), the
buildout of the curriculum could require a large number of designers and developers; multiple, concurrent
workstreams; and a great deal of project management effort. Just as important will be the curriculum
designer’s ability to see the entire forest through the many individual trees. Someone needs to have a
clear vision of what the curriculum is intended to accomplish; how all of the curriculum elements will fit
together when finished; and what the learner experience should look like when navigating through the
curriculum. The ability to envision such a large, and often complex, design; the ability to communicate
that vision to the many contributors who will be responsible for building and/or implementing parts of
the curriculum; and the ability to drive the team’s efforts toward the agreed-upon vision are all critical
skill sets for a curriculum designer.
COMPLEXITY DEMANDS ATTENTION
Curricula can range in size, but even when they are small, they are often complex. By their nature,
curricula include content from a variety of knowledge and skill domains. For example, training sales reps
often involves teaching them a great deal of product knowledge, but that curriculum likely also contains
inter-personal communication, negotiation, proposal-writing, and selling skills training. It probably also
contains compliance training, to make sure the reps know what they can and cannot do, by policy and by
law. It may involve technology training to help them learn how to use their standard-issue tablets and
their Customer Relationship Management (CRM) platform. The point is, a curriculum can involve a wide
variety of content. In addition, those different courses may need to be built and delivered using different
technologies. In a curriculum, it is not unusual to see a combination of formats used, including classroom
training, self-paced e-learning tutorials, online simulations, mobile learning modules, synchronous e-
learning, paper-based workbooks, and online tests. It is also not unusual to see a combination of formal
training and informal training, such as coaching, mentoring, secondments, and special assignments.
When you combine the complexity of multiple content domains with the complexity of multiple delivery
formats, you can get a curriculum that is extremely complex to build, implement, and maintain. With that
in mind, it is easy to see why curriculum designers require unusually strong organizational skills as well as
experience with project management and/or program management.
IMPORTANCE IS A DOUBLE-EDGED SWORD
For most organizations, building a curriculum is considered a capital expenditure and is seen as a strategic
company initiative. Of course, not all jobs are equally important. At a brokerage firm, for example, the
position of Financial Advisor is considered mission-critical, because FAs are responsible for building client
relationships; analyzing client needs; recommending solutions; and generating brokerage fees for the
firm. When the FAs perform well, the brokerage firm performs well. Other job families are important—
maybe even critical to the success of the firm in their own way—but none are as critical as the FA job
family. Therefore, it’s no surprise that when a brokerage firm designs a curriculum for its FAs, that effort
Updated: 1/15/2012 PRESTERA FX Page 8
10. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
is going to generate a great deal of interest from all sorts of senior stakeholders. At other businesses, it
will be other job families that garner the lion’s share of the attention and resources.
Stakeholder attention can be a good thing. It makes it easier to get funding when senior leadership
considers the curriculum mission-critical. It can also be easier to break through organizational barriers
and gain cooperation. Obstacles that seem insurmountable on a smaller project are brushed aside by the
momentum of a large curriculum initiative. However, with all that attention and support from senior
leadership comes greater scrutiny and higher expectations for achieving significant business results
quickly. When a curriculum gets attention, every manager in the organization wants to be involved with
the initiative and wants to have input. No one wants to be left out in the cold. The fact that most large
curricula pull content and resources from across the organization only magnifies the effect. What does
this mean for the curriculum designer? Active and proactive stakeholder management becomes a
necessity. The curriculum buildout and implementation need to be managed like large organizational
change efforts. The curriculum designer needs to be comfortable working with large numbers of senior
stakeholders, communicating effectively with them, and keeping the large egos from getting in the way of
progress.
SHELF LIFE REQUIRES LONG-TERM COMMITMENT
A typical course has a shelf life of 1-3 years. It’s much like a hamster: every few years, you can go to the
store and get a new one. If so, then a curriculum is more like a dragon that starts small but gets very big,
that lives for decades. Because of that, the curriculum’s owner needs to make arrangements to ensure
that the curriculum is cared for, maintained, and updated on a regular basis, over time. Maintenance
planning should follow immediately on the heels of the curriculum’s rollout.
A second implication of a curriculum’s long shelf life is that it will evolve and morph over time, and
workers trained today potentially will experience a significantly different curriculum than those trained a
year from now. This can create gaps in the workforce’s knowledge and skills. Indeed, most monster
curricula are constantly undergoing updates and improvements. What does this mean for the curriculum
owner? As part of the curriculum design, we need to consider how we will update workers who have
gone through prior versions of the curriculum and ensure that their legacy needs are tracked and
addressed, even as we are focusing our efforts on new workers starting the curriculum today.
Updated: 1/15/2012 PRESTERA FX Page 9
11. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
A CURRICULUM DEVELOPMENT FRAMEWORK
To summarize, a curriculum development initiative is typically viewed by business managers as a strategic,
capital investment, so there is a great deal of organizational pressure to succeed and lots of stakeholder
involvement that needs to be managed. In addition, the size, complexity, and long shelf life of a
curriculum demand greater attention to managing its buildout, rollout, and ongoing maintenance. The
curriculum designer needs to think more strategically than a course designer and needs to be able to drive
the vision across a large number of concurrent workstreams. To accomplish this, the curriculum designer
needs to leverage a combination of consulting, design, and project management skills throughout the life
cycle of a curriculum initiative. If we put all of these ingredients into a blender, and pepper in training
industry best practices and my own experiences, we arrive at the Prestera Curriculum Development
Framework shown in Figure 2.
Figure 2. Prestera Curriculum Development Framework
PHASES
Analysis Design Buildout Rollout Maintenance
Program
Management
Curriculum
Design
Curriculum
Development
FUNCTIONS
Change
Management
Marketing
Communication
Program
Operations
Program
Evaluation
Updated: 1/15/2012 PRESTERA FX Page 10
12. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
PHASES
The Prestera Curriculum Development Framework consists of phases and functions. The phases represent
distinct segments of the overall initiative. We could think of each phase as its own project, if that helps
visualize the work effort. There are five phases in the Prestera model: Analysis, Design, Buildout, Rollout,
and Maintenance. Let’s take a walk through each of them.
ANALYSIS PHASE
During the Analysis Phase, we analyze the needs of the organization and the target audience(s) as well as
consider the resources initially available to satisfy those needs. From that analysis, we identify what
needs to get done and how.
DESIGN PHASE
The Analysis Phase quickly gives way to the Design Phase, where we architect the curriculum, mapping
out all of the curriculum elements in sequence for each audience and each sub-audience. We also
describe the vision of what we would like the curriculum to look and feel like from the learner’s
perspective. Based on this curriculum design strategy, we start to create a buildout plan that describes
how the curriculum components will be sourced and staged for implementation. During this Design
Phase, we will also consider how the curriculum will need to be positioned with stakeholders and
audience groups; what change management efforts will be required; what communications will be
needed; and how all of that should unfold such that the curriculum launch is set up for success; and more
importantly, such that our learners are set up for success in their jobs.
BUILDOUT PHASE
As the name suggests, the Buildout Phase is all about building the curriculum we’ve blueprinted in the
Design Phase. The curriculum buildout effort is likely to include some development of custom
components, but it is just as likely to involve sourcing off-the-shelf components as well as re-purposing
and customizing components that already exist within the organization. Therefore, buildout does not
necessarily mean development or production. It means sourcing, purchasing, developing, customizing,
and assembling all of the curriculum components needed—and doing so in a way that ensures
cohesiveness, consistency, and alignment with the original design vision. During this time, we are also
putting together the plan for implementing the curriculum—the rollout plan. In the rollout plan, we will
define a schedule for what rolls out, when, where, how, and to whom. The rollout plan includes the
distribution of marketing communications, course registration information, and the curriculum
components themselves.
ROLLOUT PHASE
The Rollout Phase involves implementing the change management, communications, logistics, and
curriculum components for the first time. It can range from establishing an online Learning Management
System (LMS) portal and populating it with courses and resources to conducting classroom training at
multiple locations with live instructors. You might ask: why not call it “implementation” instead of
Updated: 1/15/2012 PRESTERA FX Page 11
13. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
rollout? Implementation refers broadly to the ongoing delivery of the curriculum, but the rollout is a one-
time special event for each curriculum component, the time when it is first implemented. Careful
attention needs to be paid to the rollout of new curriculum components, because first impressions can
create positive momentum for a curriculum or they can derail a curriculum’s progress (remember, senior
leadership is watching the monster closely). Typically, curriculum components will be piloted, revised,
and then fully launched and marketed as part of the rollout. Also, note that not all curriculum
components will necessarily roll out at the same time. Version 1.0 of the curriculum could contain many
of the components, but then additional components are scheduled to come online in subsequent months.
It is not unusual for it to take 3 months to a year to roll out a new curriculum in its entirety. In fact, I
recommend staging your development efforts with the idea that there will be multiple versions of the
curriculum. This takes some pressure off of the team; enables you to roll something out more quickly,
then refine as you gain experience and feedback from the field. After the rollout is completed, the
curriculum continues to get implemented, but it then moves into the Maintenance Phase.
MAINTENANCE PHASE
During the Maintenance Phase, the curriculum continues being implemented. Sometimes, the curriculum
needs to be implemented with all legacy personnel within the target audience, which can involve a
massive initial implementation to the hundreds or thousands of people in that audience. Once that group
has been trained up, the implementation work settles down, focusing primarily on new hires. Even so,
there will always be legacy needs to address, especially as the curriculum is continually updated and
upgraded. For that reason, owners of monster curricula often segment their audiences into cohorts, so
they can track what each cohort has received and has not. This way, they can go back and provide
“What’s New” training to cohorts that missed the opportunity to get that training while going through the
curriculum. When new content is added or content is changed, we need to consider those legacy needs.
In addition to maintaining the implementation efforts, we need to develop a maintenance plan for the
curriculum, which describes what components need to be updated, when, how often, and by whom.
Though unexpected needs for updates can occur, there should be some default schedule for reviewing
components to ensure that they are up-to-date. One other thing to consider is that a monster curriculum
is like a long train with many railcars. Some of those railcars may have reached the Maintenance Phase,
while others are just entering the Rollout Phase, and while still others are still in the Buildout Phase.
Key Point: It’s rare for all components to be rolled out at once,
and even more rare for them to be updated all at once.
FUNCTIONS
The functions within the Prestera Curriculum Development Framework represent the various types of
work that need to take place throughout the initiative. Again, if it helps to visualize the work effort, we
could think of each function as a person or team; or better yet, as a hat that someone needs to wear. On
a smaller curriculum, one person could wear many, perhaps even all of these hats. On a very large one,
many people could share responsibility for just one of these hats.
Updated: 1/15/2012 PRESTERA FX Page 12
14. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
PROGRAM MANAGEMENT
The Program Management function relates to all of the activities involved in managing contributors,
timelines, budgets, reviewers, stakeholders, and risks. The function could be fulfilled by a single person,
or it could be fulfilled by a team of managers responsible for the curriculum. For our purposes here, I will
use the Program Manager label to represent the person or the group of people ultimately responsible for
managing the work of building and implementing the curriculum. The Program Manager will have many
day-to-day oversight, coordination, risk management, and project communications responsibilitties
throughout the initiative, but there are four critical planning pieces that a Program Manager needs to get
right in order for the curriculum to succeed: the charter, buildout plan, rollout plan, and maintenance
plan.
Charter. The charter is an agreement between the sponsor and the team responsible for building
the curriculum, laying out what the requirements, resources, stakeholders, and constraints of the
initiative are. What is the end point we’re trying to reach? Whether that’s captured in a
document or not, the team needs to be clear on the desired outcomes.
Buildout Plan. The buildout plan is a project plan—or more likely a collection of project plans—
that describes how all of the curriculum components will be made ready for rollout. Some
components will be purchased off the shelf; some built from scratch; some re-purposed; and
some customized. Some components will be ready by January: others may come online in
March. Some will involve outside vendors: some will require contributions from internal staff.
All will require some planning.
Rollout Plan. The rollout plan likewise is a project plan—or a collection of project plans—that
describes how all of the components will be tested, reviewed, approved, piloted, finalized,
staged, marketed, and implemented—as appropriate. The logistics involved in rolling out one
component may be more complex than that of another component, so each requires due
consideration.
Maintenance Plan. The fourth critical program management deliverable is the maintenance plan.
Once a curriculum component has been rolled out and has reached the status of “routine”
implementation, it’s time to consider if/when that component needs to be reviewed for updates;
how often; and by whom. If updates are made, how will legacy learners be informed of those
changes. In other words, the maintenance plan describes how each curriculum component will
be kept relevant and accurate over time.
CURRICULUM DESIGN
As important as all the other functions are, if the curriculum design does not hit the mark, not much is
going to go right. Instructional designers who focus on curriculum-level design work are often referred to
as a Curriculum Architects, Program Architects, Instructional Design Architects, and Learning Strategists.
As with the program management function, the curriculum design function can be performed by one
person or a combination of people, so for our purposes, I’ll refer to this person or group as the Architect.
Updated: 1/15/2012 PRESTERA FX Page 13
15. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
The Architect’s job is to craft a vision of what the curriculum will look like and how learners will
experience it. This requires a great deal of creativity, conceptual thinking, and mental stamina, because
there are so many moving parts that the Architect needs to be able to visualize over a long development
cycle. Moreover, the Architect will need to communicate that vision in a compelling way to the sponsor
as well as a wide variety of contributor and stakeholder groups. An architect of a building communicates
his/her vision for the construction through blueprints, models, materials lists, and other specification
documents. The architect walks around the construction site and makes sure that the carpenters, pipe-
fitters, electricians, and other trades people are constructing the building according to the architect’s
specifications and in alignment with the architect’s vision. Similarly, our Architect must create a vivid set
of curriculum design blueprints and specifications, and our Architect must inspect all of the curriculum
components at all stages of development to make sure that each one is consistent with the specifications;
is integrated into a cohesive set of materials; and is in alignment with the broader vision of the
curriculum. What could be more important than that? To accomplish all that, here are the most critical
deliverables that the Architect will be responsible for during the initiative, many of which are found in the
Curriculum Design Strategy Document.
Job, Curriculum, and Gap Analyses. There are three types of analysis normally associated with a
curriculum development initiative. (1) The Architect will need to analyze the job itself in order to
define what “proficient” looks like for that job. This primarily involves observing and
interviewing people who know how to perform the job effectively. (2) The second type of
analysis needed is called a curriculum analysis, and it involves reviewing all current curriculum
components—or, in the absence of existing components—reviewing any components that could
potentially be made available for this curriculum. The idea here is to take stock of what’s
available; to determine its usefulness; and to figure out how it could or could not be leveraged in
building the new curriculum. (3) The third type is a gap analysis, and it is normally only
conducted when the curriculum is being built for the purpose of closing a perceived performance
gap. You might ask: aren’t all curricula built for that purpose? No, not really. Most curricula are
built for new hires, who have no performance history with the organization and therefore would
not normally have known performance gaps. Sometimes, a curriculum is built for current
employees, not for new hires. In that case, if the curriculum is being created in response to a
perceived performance gap within that target audience, it makes sense to figure out what the
difference is between what management considers optimal performance and what actual
performance looks like in the field. Often, when we conduct a gap analysis, we find that if there
is truly a significant performance gap, it is more likely to be caused by a lack of information,
resources, role clarity, or incentives than by a lack of knowledge and skills (Rothwell, 1996). Even
so, the organization may still choose to move forward with the curriculum, but hopefully armed
with some non-training related performance improvement interventions.
Curriculum Design Strategy. The strategy is a succinct explanation of how the curriculum is
going to address the needs of the organization and the target audience. For one of my clients—a
large financial services firm—the strategy boiled down to reducing early training costs by
providing most of the bootcamp training online and on-the-job; culling the flock of new hires
based on performance, keeping only those who show promise; and investing heavily into the
development of those higher performers. The strategy sets the tone for all other design
decisions, and helps to anchor all contributors and stakeholders to the initiative, even as it
Updated: 1/15/2012 PRESTERA FX Page 14
16. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
experiences many twists and turns. The Architect must develop the strategy, communicate it,
and sell it.
Curriculum Architecture: Most curricula these days involve a mixture of different types of
content, approaches, and delivery formats. One curriculum may start with webinars and self-
paced online tutorials, then proceed to online assessments, then classroom training, and then be
capped off with coaching in the field. Another may involve a series of virtual classroom events,
interspersed with team-based projects and readings assignments. Yet another may consist of
quarterly classroom events, each followed by a series of weekly webinars, interspersed with e-
learning tutorials, online simulations, e-books, vodcasts, and telephone check-ins with a regional
training manager. Whatever that combination is, that’s at the heart of your curriculum
architecture, and it will give your sponsor, contributors, and stakeholders a good sense of what
the curriculum involves.
Curriculum Map: The next level of granularity happens at the course level. What are all the
courses and programs that a given audience needs to complete in a given period of time? For
any curriculum, there could be several target audiences and sub-audiences, so expect that there
will be multiple curriculum maps needed. Curriculum maps are sometimes called roadmaps,
learning paths, and learning pathways. Regardless, they spell out what the desired route is for
the learner from beginning to end.
The Learner Experience. Though not always done, I recommend that the Architect go one more
step and define the learner experience. What does the Architect expect the learner to see and
feel as he/she is going through the curriculum? An architect of a building might describe a visitor
entering the atrium of his/her building and experiencing a sense of awe at the openness of the
space, the serenity of the waterfall in the lobby, the warm feelings elicited by the soft hues of the
paint and plush carpeting. These types of descriptions are not reserved for artsy folks.
Architects frequently speak in these terms when describing their buildings. Likewise, our
Architect needs to be able to describe how, if applicable, the learners will be immersed in the
learning because of the use of real-world scenarios and video-based problem-solving activities
that kick off each instructional unit, and how they will refer to their workbook whenever they’re
moving from component to component, so as not to get disoriented within the curriculum.
Giving the team—especially the contributors—a sense of what the Architect’s vision of the
learner experience will be like helps to get the team in alignment with each other and with the
vision, so that the buildout can go more smoothly and the finished components can be more
consistent in form.
Recurring Themes and Tactics. In addition to describing how the learner will experience these
instructional tactics that will be used throughout the curriculum, it is also valuable for the
Architect to describe recurring content themes and tactics to be used by contributors in building
out the components. For example, in a sales training curriculum I worked on, we identified in the
upfront analysis that the most successful salespeople are not shy about asking for the business or
for the referral, and so the idea of advancing the sale became a recurring theme, whether we
were covering closing techniques or cold calling techniques. Giving the team recurring themes
anchors all of the contributors and creates threads that can be weaved throughout the
Updated: 1/15/2012 PRESTERA FX Page 15
17. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
curriculum. Likewise, instructional tactics that will be get re-used throughout, such as case
studies that will be leveraged multiple times during a curriculum, a common cast of characters
that will be referenced, a process animation that will be used to introduce different parts of the
process within multiple components: these are all examples of recurring instructional tactics.
The more the Architect can anticipate the re-use of certain instructional elements and
communicate that to contributors, the more consistency and cohesiveness is baked into the
curriculum.
Key Point: The Architect is primarily responsible for driving the design vision
to all corners of the curriculum, making sure that all components are
consistent, cohesive, and in alignment with that original vision.
CURRICULUM DEVELOPMENT
As with the previously-described functions, Curriculum Development is something that can be done by
one person, but it may involve a number of different contributors. In fact, for monster curricula, it can
involve a large number of internal and external subject matter experts, instructional designers and
developers, and vendors of all shapes and sizes. Here I will use the term Developer to encompass all of
these different players. Often, the Architect is one of the Developers, but he/she is not the only
Developer. As soon as someone other than the Architect is building out curriculum components—
someone who was not involved in crafting the original vision, strategy, and tactics—there is a risk that the
curriculum components will be disjointed. In fact, this is probably the biggest risk that needs to be
managed throughout the life of a curriculum: with so many hands involved in the making of a curriculum,
it’s very easy for it to become disjointed, looking more like a patchwork than a quilt. The Architect will of
course need to communicate frequently with the Developers before and during the buildout, and the
Architect will need to review all components to ensure alignment, consistency, and cohesiveness. In
addition, there are several things that an Architect and the Developers can do to drive the development
of a seamless curriculum:
Standards. The Architect and Developers can collaborate to establish standards that can be
applied to all of the components. These can include branding standards, related to the use of the
company’s name and logo as well as the curriculum’s branding elements. Writing standards can
also help to establish consistency around tone, voice, syntax, referencing, and formatting.
1
Agreeing on external standards, such as Strunk and White’s (2000) style book or Microsoft’s
2
(2004) technical writing standards, can save time while giving everyone a common reference
point. Establishing and maintaining a lexicon, or glossary, of curriculum-specific terminology can
also be helpful, so that everyone on the team is defining terms in the same way. That lexicon can
later be published as its own curriculum component to help learners reference definitions as
well. Overall, the more that the Architect can establish, document, and communicate quality
1
Strunk, W. & White, E.B. (2000). The Elements of Style (fourth edition). Allyn and Bacon: Needham Heights, MA.
2
Microsoft Corporation Editorial Style Board (2004). Manual of Style for Technical Publications (third edition). Microsoft: Redmond, WA
Updated: 1/15/2012 PRESTERA FX Page 16
18. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
standards to the various Developers, the easier it will be to maintain consistency, cohesiveness,
and alignment. To the extent that the Developers are not aware of the standards, they will
regress towards their own means, creating deviations from the Architect’s vision, which will then
require the Architect to spend more time and energy correcting them.
Templates. It is always a good investment of time and energy to create templates that can be re-
used by all of the Developers. In the past, I have designed templates not only for the finished
materials—for example, slide decks, facilitator guides, e-learning skins—I have also created
templates for work products that get used during the buildout phase. For example, I ask the
designer of each curriculum component to prepare a treatment plan (aka, design brief, design
document, or design specification) that describes the component-level learning objectives,
instructional tactics, and content structure/flow. The curriculum-level design document
describes all of the high-level goals and objectives, global strategies and tactics, etc., so I just
need the component designer to specify what’s unique about the design of that particular
component. If we have 20 different components and each has a different designer, we could get
back 20 very different treatment plans. To prevent that, I create a treatment plan template that
tells the designers what information I need from them, in what format, and to what level of
detail. This helps ensure some level of consistency and enables me as the Architect to compare
oranges with oranges and do a better job of policing design consistency, cohesiveness, and
alignment. As a general rule, then, whenever I suspect something is going to get used more than
once, I create a template for it, so that I can impose consistency and control on the process.
Libraries. When we have a slew of components getting built by different people at the same
time, there are few efficiencies to be had unless we equip those various Developers with a
centralized library of shared resources that they can all leverage. These can include image
libraries, where we can re-use and re-purpose recurring graphic models, character photos,
avatars, etc. Programming, or code, libraries enable Developers to borrow each other’s code
more easily, so that new functionalities can be built on top of previously built ones and no one is
re-inventing a functionality that already exists in the library. Content libraries for source
documents and other subject matter inputs can help the various instructional designers educate
each other, especially as it relates to recurring content themes. Tools, templates, examples, case
studies, music, special effects, and any other elements within a curriculum that could be re-used
across multiple Developer teams can and should be organized into shared libraries. The more
information and resources we can share across the various teams, the more efficiently those
teams can work and the better aligned their efforts will be with the Architect’s vision.
Prototypes. For some things, the best way for the Architect to communicate his/her design vision
is to create models, what e-learning developers often refer to as prototypes. A prototype can be
a small version of a curriculum component that embodies the specifications and the spirit of
what the Architect envisions. If a curriculum is heavily reliant on e-learning courses, the
Architect might create a 5- to 15-minute module. If it relies heavily on classroom training, the
Architect might create slides, guides, and handouts for a 90-minute segment. The benefit of
creating prototypes is that they more concretely communicate specifications than the design
documents alone; they help to reduce variances from the design vision; they provide the team
with templates they can use; and they enable the team to test-run the development process,
Updated: 1/15/2012 PRESTERA FX Page 17
19. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
working out the kinks before a larger batch of courses flows through it. Prototypes can also be
used to get early feedback from pilot groups, so that adjustments can be made before mass
producing content, saving time and money later. There are a lot of benefits to creating
prototypes, but there is one drawback. Building prototypes can hold up the start of mass
production. I’ve found it helpful to incorporate prototyping into the start of the buildout phase.
I give myself a little head start, then build the prototype as quickly as I can, even as the other
instructional designers are starting their courses. If I can show them the prototypes within a
week or two of their start, that’s good enough to gain a lot of the alignment and consistency
benefits. To gain speed, I can initially create the e-learning prototype in PowerPoint, building
navigation and interactivity into it as best I can replicate, so that the team can more quickly see a
working model of the course. Regardless of how you do it, consider building prototypes as a way
of modeling how you want components designed.
CHANGE MANAGEMENT
Many curricula are established as part of a broader change effort. If the curriculum is for a new job—say
a company establishes the role of Key Account Manager (KAM) within its Sales department—that’s
typically a big organizational change for everyone who will come into contact with that new job. For
customers, they will need to understand what it means to have a KAM, rather than the regular Account
Representatives they had previously. For the Account Representatives, it could mean that some of them
will get pulled off of accounts that are now considered “key accounts” and given to these new KAMs. For
the Sales Managers, how will they supervise and support KAMs differently than they do their other reps?
For the Sales and Marketing staff, how will they support the KAMs differently than they do the reps? For
the CEO of the company, will a direct phone call from a KAM mean something different? It probably
should. Even with this relatively innocuous event of introducing a new KAM role into an existing
organization, there are a lot of potential change implications that ripple through the entire structure.
Imagine now a situation where instead of adding a new job, the organization has decided to change the
performance expectations around an already-existing job. For example, let’s say that a manufacturing
division decides that it wants its supervisors to spend 80% of their time working on the shop floor, directly
supervising the production line, and only 20% of their time working in the office doing paperwork and
other administrative tasks. If the current state is that those supervisors spend 90% of their time in the
office, that’s a huge change not only for the supervisors but also for the people on the production line
who are not used to seeing their supervisors very often and for the folks back at the office who are used
to seeing lots of administrative output coming from the supervisors. A seemingly small change in job
expectations can have huge change implications that ripple out across the organization. With that, my
assumption is that when I’m working on a curriculum design initiative, there is always going to be a need
for change management in order to help the organization transition from the current state to the new
desired state. I also try to remember that the curriculum itself is not the change. It is the battering ram
for the change effort: the change is actually going to happen (or not) outside of the curriculum, on the
shop floor, in the offices, around the water cooler, in conversations, and in people’s minds.
Key Point: The curriculum is not the change…it’s the battering ram for change.
The change is actually going to happen…in people’s minds.
Updated: 1/15/2012 PRESTERA FX Page 18
20. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
As with the other functions, change management is not necessarily the responsibility of one person: it can
be shared within a team, though generally there is one person who is the inspirational leader of that
change effort. Sometimes, that person is the sponsor but often it’s a leader who works for the sponsor
and has accepted the mission of driving this change forward. For our sake here, I will refer to that person
and any contributors and consultants helping that leader as the Change Agent.
The Change Agent is annoying, because he/she represents a disruption to the safety and security of the
status quo. Let’s face it, change is a hassle. Even if we accept and embrace change, most people directly
affected by change consider it at best a necessary evil. For consultants like me, who support change
efforts, we cannot understand why anyone would be opposed to changing things for the better. When
I’m starting to think that way, I remember a quote from the movie Jurassic Park, where chaos theorist Dr.
Ian Malcom says: “What's so great about discovery? It's a violent, penetrative act that scars what it
explores. What you call discovery, I call the rape of the natural world.” Now substitute the word “change”
for “discovery.” When workers and managers have gotten used to doing things a certain way and getting
predictable outcomes—even if they’re bad outcomes—they get comfort and a sense of safety from the
natural order of things—it’s like your favorite blanket on a cold winter night. The Change Agent is your
college roommate who pours cold beer on you and your blanket at 3:00 in the morning, after crawling
home from a night at the bars. It’s not so much fun when you’re the one in the bed. You could do
without the laughing and slurred speech too, and your hope is that your roommate will pass out before
doing any more damage. Likewise, workers often wish their Change Agents would run out of steam and
go to bed. I try to remember those sentiments when dealing with change efforts, because it helps me
stay humble when I think I’m a big shot change consultant.
Change Management Plan. Whether it is a formal, written document or not, it is important for the
Change Agent to have a plan. The Change Management Plan should define what the scope of the change
effort entails; who the various stakeholders are; how they are expected to respond (e.g., apathy, relief,
anxiety, opposition); and what techniques we plan to use in order to manage that response. A common
change management technique, for example, is inclusion. If you think some people are going to oppose
an initiative, ask them to help you organize it. They then can’t oppose it because they’d be opposing their
own initiative. If they refuse to help you organize it, you can later combat their opposition by citing the
fact that you gave them the opportunity to help you organize it. It’s a wonderfully sneaky technique that
often does help reduce active opposition. There is still passive opposition, though, which is difficult to
combat when a person or group is really not in agreement with the change and does things to undermine
your efforts. I generally favor a more direct approach—tell them the truth. If a change is needed, lay out
the situation for your stakeholders, equipping them with the facts; offer your solution; and invite them to
provide input and alternative suggestions. Sometimes, you will get good ideas from them, and perhaps
even, you might uncover a completely different approach. This form of inclusion is much more sincere
and earnest. What matters, though, is that people understand why it is necessary to change; that they
appreciate that you are open and honest with them about it; and that they see their input heard,
processed, and where appropriate incorporated into the overall solution.
MARKETING COMMUNICATIONS
Whether you agree that creating a new job or changing an existing job is worthy of change management
efforts or not, it is difficult to argue that a monster curriculum rollout doesn’t require a fair amount of
Updated: 1/15/2012 PRESTERA FX Page 19
21. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
marketing communications effort. Whether you are re-designing an existing curriculum or rolling out an
entirely new one, you need to generate awareness, at a minimum, from your target audience. Preferably,
your communication efforts also generate interest in participating in the curriculum and/or supporting it.
Possibly, your efforts can even help to generate some initial enthusiasm, enough to get the curriculum
launched successfully, and then you hope that positive word-of-mouth advertising will sustain it over
time.
Communication Plan. A good Communication Plan is going to define who the various message
audience are; what their communication needs are before, during, and after the rollout; what
messaging you’re going to put in front of them; when; and what communication vehicle you’re
going to use to do that. Some of your communications may be targeted at the learners
themselves, getting them pumped up that training is coming; recruiting participants for the pilot;
inspiring them to register for courses during the rollout; and making sure that they have positive
things to say about the curriculum after completing it. You might also need to communicate with
the managers of the target learners, perhaps to ensure that those managers will allow their
reports to sign up for training and take time off the line to complete the training. Maybe you
need the managers to take an active role in the training, reinforcing new knowledge and skills.
What about senior leadership—the folks who pay the bills for the curriculum? You need them to
be aware of the curriculum and the impact it’s having on the organization, so that you can
continue to get funding and perhaps even be able to request more funding in the future.
Communication Pieces. Often, a curriculum will have its own branding, possibly a logo, a tagline,
and a set of talking points. Some communication plans involve using that branding as a starting
point to create communication pieces such as brochures, web sites, YouTube video
advertisements, and special email blast announcements containing graphics and animations.
Communication pieces don’t need to be flashy and expensive: a simple campaign of email
communications can be very effective, so long as there has been some thought put into the
needs of the audience, the messaging, and the timing. The marketing communications
surrounding a curriculum are often overlooked by the Program Managers and Architects of the
world, so it’s up to the Change Agent to realize the importance of reaching all of the stakeholder
groups with appropriate messaging, so that the curriculum is successfully launched and has the
kind of impact it is intended to have. For this reason, the Communication Plan is sometimes a
subset of the Change Management Plan, and the Change Agent is often the person spearheading
the design and rollout of communication pieces.
PROGRAM OPERATIONS
All the great Program Management, Curriculum Design, Change Management, and Marketing
Communications can be for naught, if the curriculum falls apart at delivery. If the instructor doesn’t make
it to the classroom or poorly delivers the content; if the e-learning course doesn’t launch from the LMS; if
the web conferencing technology crashes part-way through the webinar, all the great work that went into
building a great curriculum can be sabotaged. Having worked in training operations, I know the
importance of logistical planning and high attention to detail, but often Program Managers, Architects,
and Change Agents are so consumed with the other aspects of the curriculum that they neglect the lowly
technicalities of getting classroom materials printed on time, scheduling classrooms, configuring the
Updated: 1/15/2012 PRESTERA FX Page 20
22. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
technology platforms, registering and tracking learners through their individual learning paths, and the
thousands of other little things that add up to a successful curriculum. The activities of Program
Operations rarely fall to a single person: it takes a village of training coordinators, LMS and portal
administrators, facilities personnel, IT personnel, instructors, printing professionals, and others to
implement a monster curriculum. For our purposes, I will refer to this motley collection as the Operators.
I could provide a huge laundry list of the things they do in order to make a curriculum run smoothly, but I
will save that for its own white paper.
Key Point: You can have the nicest trains running on the
nicest tracks, but if the train is late or doesn’t show,
you’re going to lose your passengers.
PROGRAM EVALUATION
Last but not least, we get to the Program Evaluation function. There are some aspects of program
evaluation that are formative and ongoing—such as collecting and reviewing “smile sheet” data for each
component, test scores, and program completion metrics—but there are some aspects that are
summative, intended to judge the effectiveness and worthiness of the curriculum. As with the other
functions, evaluation work can be the responsibility of one or more people. Some of them will design the
evaluation strategy and create the evaluation instruments. Others will primarily collect the data and input
them into some sort of database. Others will pull from that data to aggregate, analyze, and report on the
curriculum. Others will consume those reports and make decisions about how to interpret the
information and about what, if any, modifications are needed. I will refer to this entire collection of data
gatherers, analyzers, and decision-makers as the Auditors. Typically, the Auditors will consist of people
internal to the organization. Sometimes, they could be people from departments other than Training.
For example, I once saw a curriculum audited by a business metrics professional who worked in the
Finance department. Occasionally, a sponsor will bring in an outside auditing body. For example, I was
once asked by the National Science Foundation to help evaluate a grant-funded nanotechnology
curriculum. Bringing in an outsider to evaluate a capital expenditure makes good business sense, and it’s
surprising that more senior leaders do not do this more often than they do, even when a monster
curriculum costs millions of dollars per year to operate.
3
You may be familiar with Donald Kirkpatrick’s four levels of training evaluation: (1) reaction, or learner
satisfaction; (2) learning, or evidence of understanding and retention; (3) behavior, or transfer of learned
4
behaviors to the workplace; (4) and results, or performance impact. More recently, Jack Phillips added a
fifth level, Return on Investment (ROI). For most small-scale training efforts, we collect level 1 and
sometimes level 2 data: smile sheets and test scores. Some of my diehard training colleagues will collect
and evaluate level 3 data, taking the time and effort to gauge the extent to which the knowledge and skills
learned in the classroom were successfully transferred on the job. I would be very happy to see this done
for all training efforts that are intended to change workplace behaviors. For a large-scale curriculum,
3
Kirkpatrick D. L. (1959). Techniques for evaluating training programs. Journal of American Society of Training Directors, 13 (3): pp. 21 - 26.
4
Phillips, J.J. (1996). Measuring the results of training. The ASTD Training & Development Handbook. Robert L. Craig (ed.). New York: McGraw-Hill.
Updated: 1/15/2012 PRESTERA FX Page 21
23. MONSTER CURRICULUM! | Part 1: A Curriculum Development Framework
White Paper
where capital investments and senior leadership are involved, level 4 and even level 5 program evaluation
are not uncommon.
Measuring impact generally requires capturing performance metric data before the curriculum is put in
place—back at the Analysis phase, to see what current performance looks like—and comparing it with
post-rollout data. It can also be done by comparing the performance of a cohort that has gone through
the curriculum with one that has not, over the same period of time. In either case, the difference
represents the impact, and when that impact is quantified in terms of dollars earned or saved and
extrapolated to represent the total value gained of running the curriculum, the resulting number can be
divided by the total cost of the curriculum to come up with ROI. For example, if a sales training
curriculum nets an extra $100,000 per trained salesperson, representing additional first year revenue, and
the company spends $1 million training 50 salespeople, the ROI is $4 million, or 400% ($100k x 50 - $1m =
$4m in profit; then $4m/$1m = 400%). That kind of information is immensely valuable in defending the
curriculum’s worth to the leaders who sponsor it and secure funding for it. More importantly, without
this kind of information, we put the curriculum at risk to be scaled back or eliminated when the time
comes to slim budgets and prioritize how resources are allocated. If we wait until then to start justifying
the expenditures, it will likely be too late.
WHAT’S NEXT?
There it is, that’s the Prestera Curriculum Development Framework, five phases and seven functions to
organize our procedures, tools, and best practices. The framework can be scaled up for monster
curriculum initiatives and scaled down for small ones. Functions can be consolidated with one person,
shared among a small team, or distributed across a large number of contributors. If you need to build or
re-build a curriculum, this big picture framework will help you organize your people and your work
activities, ensuring that no major set of activities is overlooked. In subsequent white papers, I will dive
deeper into each phase and function to describe procedures, tools, and best practices for how to apply
the framework, providing examples and artifacts from real projects. In the meanwhile, please send me
your feedback, questions, suggestions, and any examples you’d care to share from your own curriculum
development efforts. The more we share, the more our collective knowledge expands and the better we
can serve our learners.
Gus Prestera is a Learning and Performance
Strategist, working at the intersection of blended
learning, curriculum development, and
performance improvement. He holds a PhD in
Instructional Systems, an MBA, and a Certified
Performance Technologist designation and has
over 15 years of experience designing, developing,
and implementing large-scale training programs
for a variety of businesses. If you have questions,
suggestions, or feedback, please contact Gus at
gus@presterafx.com.
Updated: 1/15/2012 PRESTERA FX Page 22