This document summarizes a seminar on research methodologies in technology enhanced learning. It discusses various topics related to doctoral research including defining research questions, literature reviews, methodologies, evaluation processes, research communities and paradigms. Examples from past student theses are provided to demonstrate mixed methods approaches, iterative design processes, and lessons learned regarding defining problems, collecting and analyzing data, and engaging with peer review.
TỔNG ÔN TẬP THI VÀO LỚP 10 MÔN TIẾNG ANH NĂM HỌC 2023 - 2024 CÓ ĐÁP ÁN (NGỮ Â...
Yannis@patras seminar a_20150512a
1. Research methodologies in
Technology Enhanced Learning
Prof. Yannis Dimitriadis
GSIC/EMIC research group
University of Valladolid
May 12, 2015
Univ. of Patras,
Patras, Greece
Doctoral student seminar
2. Seminar overview
2
Doctoral student seminar
Research issues in doctoral theses and
publications in the e-learning community
Revision of GSIC/EMIC artifacts
Theses, publications, etc.
Lifecycle (proposals, submissions, reviews, etc.).
Discussion of methodological issues:
multi-disciplinary field
evaluation/validation process
Analysis of a sample of participants’ artifacts
Lessons learnt and recommendations
3. Theory and practice in research
Research as
– Art and/or systematic approach
– Tools, instruments, theory to support research
– Good practices
– High dependence on context of research field
Research communities and Kuhn’s theory on
scientific revolutions
– Paradigm, textbooks and puzzles
Let’s try to connect theory and practice for
novel research practitioners
3
4. Research communities
Local vs. global level
– The need for a productive interaction
Individual vs. global perspective
– The researcher as an individual
The role of peer review in research communities
– Why citations are so important!
Types of research artifacts and forae
– Journals, conferences, workshops
– Proposals and final artifacts
– Science in action according to Latour
4
5. An exercise with the TEL field
A matter of terminology?
– “TEL”, “New technologies in education” , “e-
learning”
– Where is the focus?
– What are the instruments for research?
Define a concept map for TEL field and
community
– Actors, trends, methods
– Static and dynamic view
– Think of an example and associated literature
– Connect with your own (even limited) experience
5
6. Few GSIC/EMIC examples
Material available at Dropbox:
– https://www.dropbox.com/sh/rfp8z0itsdunich/AABezAXWd-
ipGc2yCCvjjW_-a?dl=0
The life of a paper: From ICALT 2004 to ETS 2005
– Seminar
Davinia Hernández (2007)
– Thesis (proposed and submitted)
Luis Pablo Prieto (2012)
– Thesis (submitted)
Juan Alberto Muñoz (2015)
– Thesis (submitted)
6
7. The life of a paper:
From ICALT 2004 to ETS 2005
Bad reviews are still bad, even if they are
positive and lead to a best paper award:
– Reviewer2: Fine
– Reviewer3: Very interesting, thank you!
Some conference papers may lead to journal
special issues
– No guarantee for an automatic paper “upgrade”
A good review process is probably the best
research dialogue on a topic
7
8. The life of a paper:
From ICALT 2004 to ETS 2005
Some interesting reviewer notes:
– “but this proposal has not been implemented yet
and, consequently, has not been empirically
evaluated”
– “The theoretical foundations and related theories
need to be weaved into their discussion…”
– “They make a number of claims but …. they do
not demonstrate the feasibility and utility of their
approach from the point of view of educators”
8
36. J.A. Muñoz:
Some notes on methodology (I)
Pay special attention to Section 1.2 of thesis:
– It contains a very comprehensive account of the main
research issues
– “Those were my assumptions about the world and the
knowledge. In other words, those were my ontological and
epistemological believes.”
– “I considered problematic to assume the objectivity and
the independence of the researcher and the phenomenon
explored”
– Finally, the research approach followed … is interpretive,
since I consider it matches well with the phenomenon
investigated (the orchestration problem for teachers … )
and its context (persons and organizations of persons)36
37. J.A. Muñoz:
Some notes on methodology (II)
EREM - anticipatory data reduction schemes
– “We considered EREM very appropriate due to the
guidance and help that the framework provides to novel
evaluators … Examples are the different conceptual tools,
such as an evaluation design diagram, or a multimedia
collaborative report”
– “anticipatory data reduction process to explore the
research question, creating a schema of “research
question – issue – topics – informative questions”. Thus,
we defined issues as the main conceptual organizers of
the evaluation process. Such issues were divided into
more concrete topics to help us understand the different
dimensions within the issues. ... each topic was explored
through various informative questions” 37
38. J.A. Muñoz:
Some notes on methodology (III)
Anticipatory data reduction and coding issues
– anticipatory data reduction schema was used as an
initial category tree for the process of coding, therefore
using a dominantly deductive a-priory approach for the
creation of an initial set of codes
– we consider a single-coder sufficient, given the
interpretive nature of the study, the dominantly deductive
approach (with a predetermined and agreed initial
category tree), and the fact that we do not aim to measure,
quantify, obtain statistical significant results or find
relations between variables, but rather make sense of the
participants perceptions and experiences
38
39. J.A. Muñoz:
Some notes on methodology (IV)
Interpretive research and its strategies
– in interpretive research, the notions of reliability,
internal/external validity and objectivity used in positivist
approaches to assure the quality of a research are
replaced with the notions of dependability, credibility,
transferability and confirmability
– The strategies to comply with these criteria from an
interpretive perspective are very different from the ones
used to ensure quality in a positivist research, and include
(among others) long permanence in the field, use of
multiple data gathering techniques, use of deep
descriptions, member checking, etc.
39
40. J.A. Muñoz:
Some notes on methodology (V)
Some strategies:
– prolonged engagement during months of work with the different
teachers and persistent observation in the field;
– member checking, obtaining feedback from the informants about the
data and the interpretations;
– acknowledgement of participant opinions, by interviewing the
teachers and by analyzing teachers’ and students’ reflections;
– integration of the thorough collaborative observation reports in a
single portfolio, thus enabling a thick description of the phenomenon
under scrutiny, reported in detail to the whole evaluation team;
– peer review within the evaluation team to avoid bias;
– exploration of the systems in different educational contexts;
– triangulation of data source, methods and researchers to cross-
check data and interpretations.
40
41. Some lessons learnt (I)
Think well of the publication forum
Consider the community paradigm
Define well and clearly the research question
Think of originality, feasibility, relevance
Do not “forget” the literature
Provide sound methodological “position”
Synthesize results (tables, diagrams)
Try out peer review
Check for plagiarism and acknowledgements41
42. Some lessons learnt (II)
Use visual research instruments (thesis
diagram, phases of DBR, etc.)
Define approach and techniques
Use them well (e.g. triangulate)
A system by itself is not enough
Collect empirical data and synthesize well
Interact with the community, use social
events
The “hill metaphor” and “Ithaki”
42