Exploring the Map of Distance and e-Learning Research - a Bibliographic Analysis
1.
Exploring the map of distance and e-learning research –
a bibliographic analysis
EDEN Research Workshop, Budapest,
25 October 2010
Olaf Zawacki-Richter
University of Oldenburg, Germany
Center for Lifelong Learning
Faculty of Educational and Social Sciences
2. Prof. Dr. Olaf Zawacki-Richter 2Slide / 28
Overview
Distance Education Journals
Validated structure of distance education research areas
Review of research between 2000 and 2008: gaps and priority areas
Perspectives for future research
Zawacki-Richter, O. (2009). Researchareasindistanceeducation– aDelphi study.
International Review of Research in Open and Distance
Learning, 10(3), 1-17.
Zawacki-Richter, O., Bäcker, E. M., &Vogt, S. (2009). Reviewofdistanceeducationresearch
(2000to2008)– analysisofissues, methodsandauthorshippatterns.
International Review of Research in Open and Distance
Learning, 10(6), 21-50.
Zawacki-Richter, O., &vonPrümmer, C. (2010). Genderandcollaborationpatternsin
distanceeducationresearch. Open Learning, 25(2), 95-114.
Zawacki-Richter, O., Anderson, T., &Tuncay, N. (2010). Thegrowingimpactofopenaccess
distanceeducationjournals-abibliometricanalysis. Journal of Distance
Education, 24(3).
3. Prof. Dr. Olaf Zawacki-Richter 3Slide / 28
Part I: Distance Education Journals
What are the major journals in the field and how can they be grouped
in terms of impact and perceived quality?
How do the journals relate to each other in a network of giving and
receiving citations?
What are the most central and prestigious journals in the network and
which journals are peripheral or isolated?
4. Prof. Dr. Olaf Zawacki-Richter 4Slide / 28
Structure of Distance Education Journal Network
Journals as important carriers of knowledge in a discipline
Research builds upon and refers to previous research
Previous research is acknowledged by citations
Social Network Analysis (SNA): The nodes are journals and the ties
in the network are citations.
The journal network reveals relationships and patterns of scientific
information exchange, the "intellectual structure" of a discipline
(Liu, 2007).
Liu, Z. (2007). Scholarly communication in educational psychology: a journal citation analysis.
Collection Building. doi:10.1108/01604950710831915
5. Prof. Dr. Olaf Zawacki-Richter 5Slide / 28
Selected Journals
6. Prof. Dr. Olaf Zawacki-Richter 6Slide / 28
Journal Citation Network
Mean citation rate across all journals: 44 ("tie density")
Standardized mean citation rate: 0.46 cites/paper
8. Prof. Dr. Olaf Zawacki-Richter 8Slide / 28
Journal Citation Network – Image Matrix
Doreian, P., & Fararo, T. J. (1985). Structural
equivalence in a journal network. Journal of
the American Society for Information Science,
36(1), 28-37.
9. Prof. Dr. Olaf Zawacki-Richter 9Slide / 28
Perceived quality
Journal Prestige Survey: Editorial Board Members (83 of 277).
Rating scale according to Nelson, Buss & Katzko (1983):
Please assign numbers to the journals listed below according to how
you would rate them, where 5 = outstanding, 4 = excellent, 3 = good, 2
= adequate, 1 = poor. You may leave blanks if you do not recognize a
journal.
Nelson, T. M., Buss, A. R., &Katzko, M. (1983). Ratingofscholarlyjournalsby
chairpersonsinthesocialsciences. Research in Higher Education,
19(4), 469-497.
10. Prof. Dr. Olaf Zawacki-Richter 10Slide / 28
Perceived quality and prestige
11. Prof. Dr. Olaf Zawacki-Richter 11Slide / 28
Perceived quality and prestige
12. Prof. Dr. Olaf Zawacki-Richter 12Slide / 28
Part II: Research Areas in Distance Education
What are the most important research areas in DE and how can they
be categorized?
What are the most neglected research areas?
13. Prof. Dr. Olaf Zawacki-Richter 13Slide / 28
Research Areas: Delphi Study
Expert panel: 25 individuals from 11 countries (average of 27 years of
professional experience in distance education)
1. round: Experts were asked to list 10 important research areas
2. round: Experts were asked to rate research areas on a scale of
importance;
What are the most neglected research areas?
14. Prof. Dr. Olaf Zawacki-Richter 14Slide / 28
Classification of Research Areas
Three broad research perspectives or levels with 15 research areas:
1.Macro level: Distance education systems and theories
→ global system level: 5 research areas
2.Meso level: Management, organisation and technology
→ institutional level: 7 research areas
3.Micro level: Teaching and learning in distance education
→ individual level: 3 research areas
15. Prof. Dr. Olaf Zawacki-Richter 15Slide / 28
Classification of Research Areas
Macro level: Distance education systems
and theories
1. Access, equity and ethics
2. Globalization of education and cross-
cultural aspects
3. Distance teaching systems and
institutions
4. Theories and models
5. Research methods in distance education
and knowledge transfer
Micro level: Teaching and learning in DE
13. Instructional design
14. Interaction and communication in learning
communities
15. Learner characteristics
Meso level: Management, organisation
and technology
6. Management and organisation
7. Costs and benefits
8. Educational technology
9. Innovation and change
10. Professional development and faculty
support
11. Learner support services
12. Quality assurance
16. Prof. Dr. Olaf Zawacki-Richter 16Slide / 28
Neglected Research Areas
Macro level: cross-cultural aspects of transnational education
Meso level: innovation and change management, educational
management and organisation, costs and funding of DE
Micro level: added value of Web 2.0 applications (instructional
design)
17. Prof. Dr. Olaf Zawacki-Richter 17Slide / 28
Part III: Publication and Authorship Patterns
What are the topics with the highest and lowest number of papers?
What kind of research methods are applied and do DE journals prefer
to publish qualitative or quantitative studies?
Do female and male researchers tend to choose different research
topics? If yes, what are their priority areas?
Who are the leading authors and where do they come from?
18. Prof. Dr. Olaf Zawacki-Richter 18Slide / 28
Literature Review
Review of articles that were published in 5 prominent DE journals
9 years: 2000 to 2008
N=695 (675 English, 20 French)
Journals:
Open Learning (UK)
Distance Education (Australia)
American Journal of Distance Education (USA)
Journal of Distance Education (Canada)
IRRODL (Canada)
19. Prof. Dr. Olaf Zawacki-Richter 19Slide / 28
Research areas and levels
13
13
24
31
62
12
13
18
23
41
41
48
113
121
122
0 20 40 60 80 100 120 140
7 Costs and benefits
2 Globalisation of education and
cross-cultural aspects
5 Research methods in distance
education and knowledge transfer
9 Innovation and change
6 Management and organisation
11 Learner support services
4 Theories and models
1 Access, equity and ethics
10 Professional development and
faculty support
12 Quality assurance
8 Educational technology
3 Distance teaching systems and
institutions
15 Learner characteristics
13 Instructional design
14 Interaction and communication in
learning communities
1 Macro level
2 Meso level
3 Micro level
N=695
21. Prof. Dr. Olaf Zawacki-Richter 21Slide / 28
Gender differences in research areas (single authors)
0.4%
-1.8%
-10.4%
-5.5%
-1.9%
-1.7%
-5.6%
-3.9%
-0.5%
2.5%
3.8%
-0.3%
6.0%
11.6%
7.4%
15 Learner characteristics
14 Interaction and communication in learning communities
13 Instructional design
12 Quality assurance
11 Learner support services
10 Professional development and faculty support
9 Innovation and change
8 Educational technology
7 Costs and benefits
6 Management and organisation
5 Research methods in DE and knowledge transfer
4 Theories and models
3 Distance teaching systems and institutions
2 Globalisation of education and cross-cultural aspects
1 Access, equity and ethics
preferred
by men
preferred
by women
χ2
= 38.07, df=14, p < .001
Cramer's V = .36, p < .001
N=302
159 (M)
143 (F)
22. Prof. Dr. Olaf Zawacki-Richter 22Slide / 28
Research Methods
Trend towards more empirical research?
Berge & Mrozowski (2000): 76 % (N=727 articles; AJDE, DE, JDE
and OL, between 1990 and 1999).
2000-2008: 38 %
23. Prof. Dr. Olaf Zawacki-Richter 23Slide / 28
Authorship Patterns: Leading Contributors
1,148 different authors contributed to the 695 articles in this study.
48 authors who published at least 3 articles:
Canada (17)
USA (9)
UK (8)
Australia (5), China (5)
Israel (2)
Japan/South Korea (1)
New Zealand (1)
25. Prof. Dr. Olaf Zawacki-Richter 25Slide / 28
Authorship Patterns: Countries
The origin of the first author was taken into consideration (54 countries)
The vast majority (>80%) comes from only 5 countries
AJDE: strong North-American focus
26. Prof. Dr. Olaf Zawacki-Richter 26Slide / 28
So quo vadis? Perspectives for future research
Current research is clearly dominated by studies on the micro level.
→ Major gaps were found on the macro and meso level.
DEHub Project in Australia: http://www.dehub.edu.au/
Research consortium between 5 Australian universities established
2009
27. Prof. Dr. Olaf Zawacki-Richter 27Slide / 28
So quo vadis? Perspectives for future research
28. Prof. Dr. Olaf Zawacki-Richter 28Slide / 28
So quo vadis? Perspectives for future research
European research community is weaker compared to North
American institutions (research outputs in leading DE journals).
European research agenda for the next decade.
Clear research profile for the professional field of DE.
EDEN: Focus Group or launch a project?!
EURODL: better integration into the international journal network
needed
29. Prof. Dr. Olaf Zawacki-Richter 29Slide / 28
Thanks for your attention!
Olaf Zawacki-Richter, Ph. D.
Professor of Educational Technology
University of Oldenburg
Faculty of Educational and Social Sciences
Center for Lifelong Learning (C3L)
Germany
olaf.zawacki.richter@uni-oldenburg.de
Editor's Notes
Since this is the EDEN Research Workshop, I would like to explore with you and describe the field of DE research – the map or geography of distance and e-learning research – by means of a bibliographic analysis.
My presentation has three parts:
In the first part we will explore the international network of DE journals and the relations between them.
Then we will look INTO the journals: I will present a classification systems of research areas in DE.
And finally we will look at publication and authorship patters in a literature review to identify gaps and priority areas.
The presentation is based on a series of four studies that are already published. I did these studies together with colleages from the FernUniversity in Hagen, which is Germany&apos;s Open University, among them Christine von Prümmer. And with Terry Anderson from the Athabasca University, which is Canada&apos;s Open University.
OK, now I start to analyse the international network of DE journals by covering the following questions….
Journals are important carriers of knowledge within a research discipline.
Research builds upon previous research and previous research is acknowledged through citations. So the journals and the relations between them in terms of citations can be seen as a network of information exchange that reveals the &quot;intellectual structure&quot; of a discipline.
To investigate the network, we can use Social Network Analysis (SNA): The nodes are the journals and the ties in the network are citations.
We selected 12 journals for our study….
Between 2003 and 2008 a total number of 1,416 full papers were published in these journals.
The highest number of papers were published in TOJDE (255), the lowest number in AJDE (74) and DE (68) – they seem to be much more selective.
This is the journal citation network in a 12x12 matrix. In the rows you see the citing journals and in the columns are the cited journals.
They generated a total number of 6,292 citations which yields a mean citation rate accross all journals for the 144 entries in this table of 44. This we term the overall tie density.
The journal with the highest number of citations is AJDE (898), followed by DE (437), JDE (385), and OL (335).
OJDLA seems to constitute a separate cluster with a distinct community in distance education administration and management. This community refers to itself, 62 % (240 of 386) are self-citations.
For the further analysis and a comparison between journals we have to standardize the citation values. The absolute level of citation depends on the size of the journal (larger journals with more issues and papers per volume probably receive more citations than smaller journals). So the structure of the journal network must be viewed without any confounding effects due to size.
To control for the different journal sizes, the giving and receiving citations are standardized by dividing them by the number of papers in each journal. This yields an overall tie density of 0.46 cites per paper.
With SNA tools you can produce these buitiful network graphs…
The nodes in this network diagram are the journals and the ties between them are incoming and outgoing citations. The biggest node here is AJDE. This is the most central journal in terms of incoming citations.
A prominent, central journal in the network is a journal that is extensively involved in relationships with other journals in terms of citations. The prestige or status of a journal in the network increases as the journal becomes the object of more ties. So the simple idea is that prestigious journals receive more citations.
This is a simplified image of the journal network showing only those ties that are above the mean citation rate or tie density. Dashed lines indicate ties close the tie density of 0.46 cites per paper.
The stars in the network are the most established journals: AJDE, JDE, DE and OL. They receive the highest number of citations above the average.
Then we have journals in the periphery: These journals are much younger (mean publication of 12 years) and interestingly 4 of the 5 are open access journals. They are active members of the network in terms of giving citations. So they refer to the core, top journals, however their citations are not reciprocated. This finding is supported by Doreian and Fararo who found that leading authors who publish in top journals pay little or no attention to work contained in less prestigious journals and they do not tend to publish in such journals.
The Asian JDE, EURODL and IJDTE are isolates, at least on the international level. IJDET has a strong focus on computer science and the papers are very technically oriented, suggesting a focus on a different network outside of the other distance education journals.
Note that there is also NO link between the two European journals EURODL and OL.
What about the perceived quality of the journals by experts in the field.
We sent a journal prestige survey with a rating scale according to Nelson, Buss & Katzko to all 277 Editorial Board Members of the 12 journals. 83 of them (= 30%) responded.
Based on the Editor&apos;s ratings re: how they evaluate the quality and prestige of the journals we see that we have a top group of 5 journals with…
Then a group in the middle with…
And a group at the bottom
The highest rated journal is IRRODLE, however I don&apos;t believe that IRRODL is that much different from the other 4 top journals, because editors are biased towards their &apos;own&apos; journal as you can see on the next slide…
We tested with a series of t-tests if the ratings of editors and non-editors are significantly different from each other.
All editors rate the quality of their own journal higher than non editors. And for example for IRRODL this difference is statically significant: 4.52 compared to 4.02.
We now described the overall structure of the distance education journal network – but now let&apos;s look INTO the top journals in the field research trends and to identify gaps and priority areas…
The first thing we need to do that, is a validated classification system of research areas in distance education.
So I carried out an international Delphi study.
25 individual were on the expert panel. They came from 11 different countries, reporting an average of 27 years of professional experience in distance education.
In the first round the experts were asked to list 10 important research areas.
Three major lines of research or broad meta-levels of distance education research were derived from the expert&apos;s responses in the Delphi study…
Which is the level of the individual learner
Within these three levels, the research issues that are considered important by the experts were categorised into the following 15 research areas…
Based on this classification system we can now look at publication and authorship patters in a literature review.
Based on this classification system we can now look at publication and authorship patters in a literature review.
We reviewed all full papers that were published in the 5 top DE journals…
We selected the stars of the journal network plus the highest ranked journal in the journal prestige survey (IRRODL)
In the time period between 2000 and 2008, 695 articles were published in the five journals.
Looking at the frequencies of research articles on each topic, reveals a strong imbalance of research areas covered in the publications:
The micro-perspective (Teaching and learning in distance education) is highly over-represented. As you can see here in this nicely edited chart…
This is not very surprising since I would guess that the field of DE is of course dominated by educators who are concernced with research into individual learning processes.
This table shows a ranking of research areas by number of articles.
Here in the column with the cummulative percentages you see that over 50 % of articles deal with topics on the micro-level, whereas issues that were considered as neglected research areas are at the bottom of the list.
This supports the expert‘s opinion in the Delphi study.
This diagram shows the percentages within gender in relationship to research areas. For this diagram only 302 single author papers were taken into consideration….
There is a highly significant association between research areas and gender. I am sorry that I have to confirm common stereotypes now…
Male researchers often deal with topics stereotypically associated with them: management and technology.
Whereas female researchers are more concerned with the social aspects of learning and teaching, interaction and communication in learning communities, learner support services, and learner characteristics. The only field of educational management where women contributed more articles than men is learner support services and professional development and faculty support, often perceived as a &apos;softer&apos; discipline of management.
This table shows the frequency of applied research methods by journal.
Maybe there is a trend towards more empirical research.
In 2001 Berge & Mrozowski classified 76 % of 727 articles published in AJDE, DE, JDE and OL between 1990 and 1999 as purely descriptive.
Whereas in our analysis for the time period between 2000 and 2008, only 38 % of papers were descriptive in nature.
The table furthermore reveals that AJDE seems to prefer quantitative studies. 63,4% followed a quantitative design.
And the journal which accepted by far the highest percentage of qualitative studies is DE (29,5%).
OK, it is also very interesting to look at who are the leading contributors in DE research and where do they come from?
1,148 different authors contributed to the 695 articles in this study, and 48 authors publish three or more papers.
The vast majority of them comes from Canada. Canada seems to be the center of international distance education research. Congratulation! Where are the Canadians?
Canada is followed by…
The top author is Allan Jeong, a professor at the Florida State University. His research is focused on diversity in online interaction patterns using quantitative sequential analysis to explore different message-response sequences and group interaction patterns in the online learning environment.
Now check out the list. Here are the leading authors in our field. Show up, how many of them are here today? I know Dianne Conrad, number 2 on the list, is here. Where are you Dianne? Congratulations!
For the analysis of country-wise distribution of articles the origin of the first author was taken into consideration. They came from 54 different countries.
The research literature is dominated by only 5 countries. The vast majority (over 80 %) comes from USA, Canada, UK, Australia, and China. The UK is the only European country among the top 10 countries!
The crosstabulation of countries and journals furthermore reveals that journals tend to publish more from their country of origin…
Especially AJDE has a strong North-American focus – or maybe we can say a North-American bias – over 80 % of articles published in AJDE are contributed by US-American or Canadian authors.
However, another Canadian Journal, IRRODL, is the most international Journal with authors coming from 34 different countries and only 19% of authors coming Canada.
Research questions should be posed within a theoretical framework and embedded in a holistic structure of research areas within a discipline. I hope my work is a good contribution towards such a map of DE research which helps to organize the body of knowledge in our field.
Current research is clearly dominated by studies on the micro level (teaching and learning in DE), whereas major gaps were found on the macro and meso level, especially on the institutional level of educational management.
I just came across a very interesting project, the DEHub project in Australia.
A research consortium was established in 2009 between 5 Australian universities:
University of New England (UNE), Charles Sturt University (CSU), Central Queensland University (CQUniversity), the University of Southern Queensland (USQ), and Massey University.
They received a massive grand from the Australien Government (I heard about 20 Million Dollars) to establish a distance education research institute and to develop a research agenda.
As you can see here, for their research agenda they used the classification system developed in our Delphi Study. The research levels and areas are almost excactly the same, they just added the theme of OER on the meso level.
The European DE research community is weak compared to North American institutions, at least in terms of the quantitative research outputs in leading DE journals. The UK is the only European country among the top 10 countries.
I think, we need a European research agenda for the next decade with a clear cut research profile to contribute to the consolidation of the research-based professional field of DE to inform practice, Governments and other stakeholders who might provide the project funds.
The priority areas in this research agenda could be based on – among other resources – on the Delphi study and the literature review I presented here – this might be a good starting point.
Maybe a European Association such as EDEN could be the facilitator of this process.
European DE journals should be strengthened, especially EURODL.
EURODL is an isolated journal in terms of citations. EURODL should be better integrated into the international journal network by publishing research papers with higher impact obligated to academic rigour. The goal has to be that EURODL is listed in major citation databases such as SCOPUS or the Web of Science.