1. ICT research in Africa: Towards
appropriate research
methodologies
Duan van der Westhuizen
Faculty of Education
University of Johannesburg
2. “There is a growing concern internationally that the
investment in ICT in schools is not impacting on literacy
development. This concern arises from a belief held by
many - including governments as well as schools - that ICT
is beneficial to learning, and specifically literacy
learning.”.
Torgerson C, Zhu D (2003)
3. "It is generally believed that ICTs can empower teachers
and learners, promote change and foster the development
of '21st century skills', but data to support these beliefs
are still limited".
infoDev.org
4. The positive impact of ICT use in education has not been
proven In general, and despite thousands of impact
studies, the impact of ICT use on student achievement
remains difficult to measure and open to much reasonable
debate.
5. “There is neither a strong and well-developed theoretical
case nor much empirical evidence supporting the
expected benefits of ICT …”
Contrasting evidence: BECTA (2002) and Machin et
al(2006) found positive effects, while Fuchs & Woessman
(2004), Leuven et al (2004) and others found no real
positive effect
6. Threat 1: Publish or perish
• The research endeavour is a primary imperative for
universities
• Universities under state pressure to up research output
• Universities rewarded for publication by funding model
• Accredited journal articles preferred and rewarded
• Cascades down
– University leadership
– Faculty leadership
– Departmental leadership
– Individual faculty members
• What drives the push for increased output?
7. Solani Ngobeni:
Fostering academic mediocrity
• About 96% of research output in South Africa is journal articles
• Tied to the funding model for universities
• Financially lucrative for universities to publish in accredited
journals
• This turns universities into managerialised corporate entities
• Lecturers become revenue-generating employees
• Publishing becomes a Key Performance Area at all the levels
• Therefore, publishing is not about scholarship anymore, but
about generating revenue
• It is not about quality, but about quantity
• The system of journal publication is not without problems
• And nor is the system of ‘rated’ journals (ISI, IBBS, DoHE)
8. Threat 2: Quality of Educational Research
• Hargreaves (1996): Educational research is poor value for
money, remote from practice, and of poor quality
• Considerable amount of second rate research that is irrelevant
to practice and clutters up journals that nobody reads
• A study by OFSTED explored several (41) educational
publications.
– Partisanship in the conduct of research, the presentation of
research and the argument of research
– There are methodological issues
– Non-empirical research did not consult primary resources
– Focus was often obfuscated
• Researchers “employ weak research methods, write turgid
prose, and issue contradictory findings” (Reeves, 2002)
• Dillon and Gabbard (1998): “statistical analyses & research
methods are frequently flawed, limiting our understanding of
these important issues” and limiting generalisation of findings.
9. Pasteur’s quadrant (Stokes)
Research inspired by use
No Yes
Yes Research by Research by
Bohr for an Pasteur for
atomic model pasteurisation
Research inspired by
quest for fundamental
understanding (theory)
Discoveries and
research done
No by Edison
10. Basic Research and Applied Research
• In Science—The Endless Frontier (Bush, 1990) it is argued that
basic research should be performed “without the thought of
practical ends”
• There is a linear progression from basic research to
application, to social innovation
• Stokes (1997) claims that this thinking is fundamentally
flawed
• The motive for fundamental understanding and the drive for
application are not separate or in opposition to each other
• Research can be undertaken both as a quest for basic
understanding (rigor) and with considerations of use
(relevance).
11. Researching Education:
The hardest science of all?
• Educational Science is incredibly hard to do
• Hard and soft science: a flawed dichotomy?
• ‘Hard’ science is based on randomization, generalization, based
on ‘empiric’ data, often in laboratory conditions
• ‘Soft’ science is squishy, unreliable and imprecise
• Berliner (2002) claims ‘hard’ science is easy science, and
educational science is the hardest science of all
• Deal with contexts that limit generalization and theory
building
• Challenge to replicate findings
• Challenge to find the numbers for statistical analysis
• Does not seek to find the regularity that nature provides
12. The hardest Science (cont)
• Humans learners in schools are embedded in complex and
changing networks of social interaction
• The context can never be controlled
• The variety of ‘variables’ (e.g. ability, status, motivation,
teacher competence, conceptions of learning, beliefs about
assessment) are continuously interacting, and the direction or
reciprocity of interactions are difficult to determine
• Educational science also suffers from the ‘decade by findings
interactions’ phenomenon
– Change in social, cultural and intellectual environments
negate previous findings
13. Threat 3: Quality of Educational ICT Research
• ICT research lacks a sense of history
• Longitudinal studies lag behind ICT developments
• The focus on ICT (as an object) research, not on learning
impact
• Over-reliance on Qualitative methods
• Quantitative work is generally on ‘attitude’ and ‘usage’
• Simple descriptive analyses
• Few comparative analyses (e.g. that of Lim, 2003, and
Pelgrum, 2001)
• Small-scale studies that fail to produce findings that are
reliable and generalisable
• Much of the research is dispersed at sub-discipline level
14. Threat 4: Lack of large-scale meta-
analyses
• Most prior meta-analyses were conducted on
Distance Education
• Kulik & Kulik pre-1980
• Most recently by US DoE: Evaluation of
Evidence-Based Practices in Online Learning:
A Meta-Analysis and Review of Online Learning Studies
• Project RED: THE TECHNOLOGY FACTOR
15. Threat 5: Insufficient local research
• Van der Westhuizen (2002), there is little research
addressing and locating specifically local concerns
• Annotated Bibliography on e-Learning and Application of
Educational Technology in African Countries, or in
Contexts Relevant to Africa (Carnegie)
• Deficits in Academic Staff Capacity at African Universities
(Partnership for Higher Education in Africa, 2010)
• Growth in student numbers not met by growth in staff
• Lack of female staff
• Low numbers of PhD and M students
• Staff qualifications
• SA: CHE Report on Higher Education Landscape
16. Case Study M & D ICT Research in SA
• Finding congruence between the purpose of
the inquiry, the theoretical framework in
which the study was conducted, the research
design and the interpretation of findings.
• Not simply mapping the nature of theoretical
frameworks in the inventory, but to interpret
and critique the way in which the authors
used theoretical framing as a scholarly device.
17. Method
• Seven leading universities ito publication output
• Sample studies were identified through library
keyword searches
• Samples were either taken off the shelves or
obtained digitally
• Research instrument was applied to 103 studies
• Instrument captured summaries of
– Research question
– Supporting theories
– Research paradigms
– Findings (conclusions)
18. Discussion: towards theoretical
emancipation (Best cases)
• a small insignificant portion of the selected
studies had theoretical frameworks as
epistemological guides
• these studies were conceptualised coherently
and the various aspects of the inquiries were
congruent
• studies that were theoretically developed
yielded data that could be interpreted in more
depth
19. Discussion: towards theoretical
emancipation (2)
• A substantial majority of authors employed
theoretical frameworks in a very limited way
• theoretical frameworks in literature reviews
without using them as epistemological guides
• findings that were descriptive in nature,
contrary to a claimed interpretivist stance.
20. So ……
• Educational ICT is a developing research field and the
disciplinary training is not yet cohesive
• There is evidence that the researchers remained
practitioners of Educational ICT and did not manage
to become scholars
• It raises the concern that supervisors in M & D
programmes are not equipping their students with
the necessary skills to “find an epistemological
home” for their studies and to analyse and interpret
results.
21. Possible causes
• The demands of transformation and standardisation
have been many
• Academic and social “fall out” of mergers
• Certainly, the state of supervision in the country is
generally not sufficient, as is evident in the drive of
an organisation such as SANTRUST (ex SANPAD)
• supervisors are increasingly expected to have a high
and accelerated output and ‘through put’
• The demand to increase a university’s income is
seductive for an academic and linked to promotion
22. How?
• Appropriatestudies
Abandon case Designs?
• Abandon “user experience” ‘research’
• •Course evaluation is not research
Experimental/quasi-experimental research
•
(considering the ‘random assignment’ and have
Research projects must be theoretically founded,
theoretical outcomes
• requirement)
Research-competency training
• •Select appropriate research designs
Large-scale studies
• •Encourage research teams
Longitudinal studies
– Supervisor 1 (Senior) – PhD student
•Repeated-measures studies 1 - PhD Student 2 -
Supervisor 2 (Junior) Master Student 1, 2, 3
•Design-based Research
– Work on (dimensions of) focussed, authentic problems
– (which addresses all of the above)
International collaborative research
