Slides from the workshop presentation on Design-Based Implementation Research for the Multidisciplinary Program in Education Sciences (MPES) at Northwestern University. Presented by Bill Penuel and Barry Fishman on May 24, 2013.

1. Developing and Supporting Design-
Based Implementation Research:
What Should Early Career
Researchers Know and Do?
Bill Penuel
Barry Fishman

2. Purpose
• Help you understand strategies for
building a career focused on supporting
collaborative, systemic change efforts.
– Is there an inherent (or institutionalized)
tension between doing “rigorous research”
and having a positive impact on practice?
– If so, how do you negotiate that?
– The woods are full of bears… you can survive
them and still do good work in the woods

3. Introductions
• Name
• Program (year)
• Goals for today (or burning question)

4. Focus
• Forming and maintaining partnerships
• Defining one‟s contribution in large
research teams, and
• Answering your questions about this work.

5. Organization

6. Warning!
• The work will go (too) quickly.
• Plan on revisiting the ideas with
participants and as your own
work evolves.

7. When and how do you form
partnerships?
Challenge Cycle 1

8. The Case of CSILE

9. Case Analyses
• Read both cases
• Discuss your ideas in a group of 3-4
people
• Share back your thoughts…and be
thinking about which case is one that
you‟re more interested in thinking more
about.

10. Norms for Researchers
• To be responsive to teachers‟ concerns and
those of other members of the school
community (including parents and community
members, as well as school leaders) in an
ongoing way.
• To engage in and learn from a process of co-
design of software.
• To learn from the use of technology and
activities the role that handhelds can play in
changing the focus of students‟ attention in
the classroom toward their own thinking.

11. Norms for Researchers
We decided on specific formats and activity
structures within which we shared this expertise:
• By reflecting on contrasting “cases” of science
teaching drawn from outside the district to help
clarify that each member of the SRI design
process also brought a unique perspective to the
problems of teaching and assessment.
• By fostering and supporting teacher inquiry on
documents or frameworks that were meaningful to
us, through readings and discussion.
• By posing questions to teachers about
consequences of particular design decisions that
we might see, based on our prior encounters with
similar types of classroom situations.

12. Case Analyses
• Read both cases
• Discuss your ideas in a group of 3-4
people
• Share back your thoughts…and be
thinking about which case is one that
you‟re more interested in thinking more
about.

13. Changing the Object of Design
• From developing and testing innovative
learning environments…
• To changing practice and
infrastructures for improvement

14. Traditional Design-Based Research
Researcher

15. Researcher
How Practices Are (Really)
Organized

16. Researcher

17. Research
Team
New Divisions
of Labor

18. Research
Team
New Divisions
of Labor
Multiple
Layers of
Infrastructure

19. Research
Team
New Divisions
of Labor
Multiple
Layers of
Infrastructure
Leverage
Diverse
Experiences

20. Some Key Elements of
Infrastructure
• Schools and districts
– Standards and assessments
– Curriculum materials
– Pacing guides
– School organizational routines (teacher teams,
data teams)
– Master schedules
– Professional development opportunities
– Instructional support linkages between district
and schools

21. Some Key Elements of
Infrastructure
• Organized activities for children and youth outside of
school
– Ties between local and national youth organizations
(administrative structures, „curriculum‟)
– Staffing and professional development
– Volunteer infrastructure
– Transportation
• Families
– Daily, weekly, monthly, and annual routines
– Values and commitments that occasion informal teaching
– Social networks that foster circulation of knowledge of
learning opportunities
– Membership in different communities of memory (faith-
based, cultural)

22. Whole Group Discussion
• What infrastructures are needed to
support the interventions in the cases?
• How do you imagine working with people
to support these?
• What might be a study you as an
individual researcher might do that builds
knowledge and contributes to the overall
effort?

23. More on iHub
• Raymond, a math curriculum and
instruction student interested in curriculum
use
– How do teachers interpret and make sense of
the mathematical practices of CCSM inside a
co-design process?
• Sam, a learning sciences student
interested in DBIR
– How do district leaders and researchers
negotiate the joint focus of their work?

24. Challenge Two:
Defining Your Contribution
• Our goals:
– Help you identify your contribution within a
large collaborative project.
– Help you develop multiple framings of their
commitments and goals for their research for
different audiences, including practitioners,
education organization leaders, and
policymakers.

25. Questions to Spark Discussion
• What‟s your main commitment in your
research? What are you hoping to
accomplish with your dissertation? By the
time you earn tenure or your first big
promotion? (This could be a basic research
goal, a change the world goal.)
• What are the areas you‟d need to build
toward?
• Where does this intersect with concerns of
practitioners?

26. Tool: Translating a Pitch
• Your task:
– Write out the “need” “approach” and “benefits” for
3 of the 4 following groups (you pick): smart
acquaintance at a cocktail party; future colleague
as part of a job interview; a district leader or
museum director; teacher or informal educator
who you want to help you design.
– Use only one sentence for each.
– Write them side by side, so we can together
explore the differences
• Format: NABC

27. NABC (Elaboration)
• Need
– The other person’s need or concern that your design
will address.
• Approach
– The angle or strategy you are bringing on how to
address that person‟s need.
• Benefits
– The benefits to the other person or groups with whom
they are concerned that will result from taking your
strategy.
• Competition
– The alternatives to your strategy that may be well
known, popular, have been tried and failed, etc.

28. Questions for Discussion
• What are the points of difference and
overlap you notice in your statements?
• Does one feel more “authentic” than
another? If so, why?
• Does writing the educator/education
leader frames lead you to want to change
how you frame your work for future
colleagues at all? If so, how?

29. Matching Phase of Development
to Questions and Methods
Phase of Development Driving Questions Sources of Evidence
Problem Negotiation What problem of
practice should be the
focus of our joint work?
Available data from
multiple sectors
Research evidence
Perspectives and values
of stakeholders
(including nonschool
actors)
Co-design What should be the
focus of our work?
To what extent do teams
leverage the diverse
expertise of
stakeholders?
Design Rationales
Ethnographic accounts
of design processes

30. Matching Phase of Development
to Questions and Methods
Phase of Development Driving Questions Sources of Evidence
Early implementation How do implementers
adapt the innovation to
their local contexts?
How do implementers use
the innovation to
reconstruct their practice?
What are the appropriate
measures of impact?
Observations of
implementation
Interviews
Assessment design
Efficacy What is the potential
impact of the innovation on
teaching and learning?
What mediates impacts on
learning?
Randomized Controlled
Trials
Interrupted Time Series
Designs
Explanatory Case
Studies

31. Matching Phase of Development to
Phase of Research
Phase of Development Driving Questions Sources of Evidence
“Translation” (Type II) What supports are needed
to implement the program
effectively?
What are the conditions for
sustainability?
Experimental
comparisons of different
means of support
Explanatory comparative
case analysis