1.
Mixed Martial Arts
for Researchers
Design-based Research (DBR)
Christian Glahn & Agnieszka (Aga) Palalas
mLearn 2013

2.
Be prepared for the truth
and nothing but the truth!

3.
Chapter 1
The Nature of the Art

4.
“Design experiments” > design-based
research = design research = DBR = EDR
= researching innovative educational
designs in their naturalistic settings
interventional research >practice>research
(Brown, 1992; Collins, 1992)
4

5.
A systematic but flexible methodology
aimed to improve educational practices
through iterative analysis, design,
development, and implementation, based
on collaboration among researchers and
practitioners in real-world settings, and
leading to contextually-sensitive design
principles and theories.
(Wang & Hannafin, 1999, p. 7)
5

6.
DBR is a qualitative multimethod approach for exploring
uncharted domains while
designing effective solutions
for these domains.

7.
Qualitative means that the
outcomes of DBR will not be
able to test hypotheses
DBR is not a qualitative method!

8.
Exploring means that, both,
practice and theory do not
suggest anything meaningful.
DBR identifies working
hypotheses

9.
Uncharted means that theory
(aka "prior research") does not
predict the potential outcome of
a project.
DBR lets theory emerge by
isolating research problems.

10.
Designing means that the
project outcomes include at
least one working solution
supports your claims
DBR does not simply transfer
a solution from one domain
into another

11.
Chapter 2
When it is Time to Fight and When to Run

12.
DBR is a flexible and powerful
approach unless you are
unprepared or unaware

13.
Informed Exploration
Enactment
Evaluation:
Local Impact
Evaluation:
Broader Impact

14.
Rule No. 1
Find out what you are doing
(next)

15.
Run if you want to
• Want to build a mobile app
• Use a mobile app in your practice

16.
Fight if
• You have identified a problem
rooted in practice
• You can't solve it by combining
and deducing mobile solutions
and theories

17.
Rule No. 2
Find out what others did

18.
Run if
• There are similar (commercial)
tools for the problem
• There is a lot of literature

19.
Fight if
• Tools need more than just a
redesign
• The literature does not make
sense for your problem

20.
Rule No. 3
Make sure that DBR is indeed
appropriate

21.
Run if
• Your project can be structured
into logical steps
• There are (working) hypotheses
that can get tested

22.
Fight if
• A practical solution is required
• The literature does not provide
sufficient information

23.
Chapter 3
There is always more than One Way

24.
(Van den Akker, 1999, p. 9)
24

25.
What are the characteristics of an
<intervention X> for the purpose/
outcome Y (Y1, Y2, …, Yn) in context Z?
(Plomp, 2009, p. 19)
2

26.
Multi-method means that all
means necessary are allowed.
It does not mean that it will be
any easier

27.
Rule No. 1
Always start from a solid
research questions to guide the
design process

28.
Rule No. 2
Always answer your questions

29.
Rule No. 3
Identify methods you can use

30.
Rule No. 4
Select a method that helps
isolating a working hypothesis

31.
Rule No. 5
Document the identified working
hypotheses for further research
by others

32.
DBR
• Practice ï ðresearch
• Drawing on engineering and technological research
• Focusing on design, construction, implementation and
adoption of learning solutions
• Process focused and iterative
• Interventionist: applied solutions to real educational
problems
• Contextual: real people, context, and cultural
background, in-situ investigation/evaluation

33.
DBR
• Applied
• Participatory
• Collaborative
• Utility oriented
• Theory driven
• Unifying theory and practice
• Evolving
• Systemic

34.
DBR
• Rich feedback
• Responsive
• Flexible
• Agile
BUT
o Multileveled and multifaceted
o Complex
o Messy

35.
Participatory 1/2
• Co-developing theory and design in-situ
• Testing in real-world learning situations
involving all actors/end users
• With people for people
• End-users as partners (active co-creators)
• Combination of motivation and efforts
• Students as researchers
• Diverse evaluation perspectives
“We are moving away from a passive information age
towards an active participation age” (Farmer & Gruba, 2006, p. 149)

36.
Participatory 2/2
• Diverse expertise of practitioners & researchers
• Collaboration & support
• Network of colleagues
o Flexible dialogue from whenever
o Online platform and tools (e.g., Wiggio, Collaborate)
• Dynamic feedback from actors
o At milestones and agile

37.
Benefits of DBR (m-learning)
Context is king
• Evolution of theory
• Evolution of practice

38.
More Benefits of DBR (m-learning)
• Build foundational understanding of m-learning trends
and needs
• Gain insight into learner m-learning behaviours
• Generate inspiration and ideas for appropriate design
• Define what functionality is most critical
• Improve the usability of infrastructure
• Inform future design solutions
• Provide professional development
• Raise awareness and understanding of m-learning

39.
(Palalas & Hoven, 2013)
(Palalas & Hoven, 2013)

40.
§
The scope of DBR
•
•
•
•
•
•
§
The role of the researcher
•
•
•
§
complexity of the system – breadth
coordination of research and design activities
no objective measure of learning
amount of data
consensus-reaching and interaction among actors
intensity
multifaceted
conflicting roles
threats to validity
Transferability of findings
4

41.
DBR: Recommendations
•
•
•
•
•
•
•
•
Rigorous data collection/analysis procedures
Regular communication
Findings documentation and reporting
Sound conceptual framework
Clear deadlines and deliverables
“Flexible” (academic schedules)
Solid project management a
Collaborative DBR research team
“If a researcher is intimately involved in the conceptualization, design, development,
implementation, and researching of a pedagogical approach, then ensuring that
researchers can make credible and trustworthy assertions is a challenge”
(Barab & Squire, 2004, p.10)

42.
Plan Well
• Project context
• Objectives
• Audience
• Location
• Methods
• Schedule
• Outcomes

43.
aga@epluslearning.com
LinkedIn: http://www.linkedin.com/in/apalalas
Presentations: http://www.slideshare.net/agaiza
Publications: http://athabascau.academia.edu/apalalas
4

44.
Bannan, B. (2009). The Integrative Learning Design Framework: An illustrated example from the domain of instructional technology. In T. Plomp & N.
Nieveen (Eds.), An introduction to educational design research (pp. 53-73). SLO: Netherlands Institute for Curriculum Development.
Barab, S., & Squire, K. (2004). Design-Based Research: Putting a stake in the ground. The Journal of the Learning Sciences, 13(1), 1-14. doi:10.1207/
s15327809jls1301_1
Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions in classroom settings. Journal of
the Learning Sciences, 2(2), 141–178.
Collins, A. (1992). Towards a design science of education. In E. Scanlon and T. O’Shea (Eds.), New directions in educational technology (pp. 15–22).
Berlin: Springer.
Dede, C. (2004). If Design-Based Research is the answer, what is the question? The Journal of the Instructional Sciences, 13 (1).
Design-Based Research Collective. (2003). Design-Based Research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1),
5-8. Retrieved from http://www.designbasedresearch.org/reppubs/DBRC2003.pdf
Farmer, R., & Gruba, P. (2006). Towards model-driven end-user development in CALL. Computer Assisted Language Learning, 19(2 & 3), 149-191.
Palalas, A. (2012). Design guidelines for a Mobile-Enabled Language Learning system supporting the development of ESP listening skills (Doctoral
dissertation, Athabasca University). Retrieved from http://hdl.handle.net/10791/17
Palalas, A., & Hoven, D. (2013). Implications of using DBR to investigate the iterative design of a mobile-enabled language learning system. CALICO
Plomp, T. (2009). Educational design research: An introduction. In T. Plomp & N. Nieveen (Eds.), An introduction to educational design research (pp.
9-36). SLO: Netherlands Institute for Curriculum Development.
Reeves, T. (2006). Design research from a technology perspective. In J. van den Akker, K. Gravemeijer, S. McKenney & N. Nieveen (Eds.),
Educational design research: The design, development and evaluation of programs, processes and products (pp. 52-66). New York: Routledge.
Van den Akker, J. (1999). Principles and Methods of Development Research. In J. van den Akker, R.M. Branch, K. Gustafson, N. Nieveen, & T. Plomp
(Eds.), Design approaches and tools in education and training (pp. 1-14). Boston: Kluwer Academic.
Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research and
Development, 53(4), 5-23.