This paper reports the findings of a Stage I meta-analysis exploring the effectiveness of online digital literacy practices performed through social media tools in higher education classrooms as measured by learning outcomes. An extensive literature search culled more than 500 potential articles and resulted in a sample of 51 representative quasi-experimental studies, consisting of 4,630 total participants. Fifty-one effect sizes were extracted and yielded a moderately positive statistically significant weighted average effect size of g+ = 0.315, k=51, p <. 01. This overall effect size suggests that integrating online digital literacy practices into the higher education classroom benefits students on measures of academic achievement and offers significant learning support. In addition, several moderator variables were tested to determine what factors and literacy practices impact learning outcomes; namely, field of study (STEM, ARTS), conceptualization of tool (social learning theory, delivery tool), peer interaction (yes, no), modality (blogs, collaborative communication text-based technologies, i.e. forums and wikis; podcasts, and virtual worlds), practice (consuming, prosuming behaviors), and the learning outcome measured (course project, knowledge-based exam); moderator effect sizes were statistically significant for the following variables: conceptualization of the tool, practice, and learning outcomes.

1. SAMPLE
RESULTS
More than 500 abstracts were reviewed for this
meta-analysis, leading to the retrieval of 325 full-
text articles and ultimately 90 primary research
articles for consideration, which were further refined
to the inclusion of 51 studies that yielded 51 effect
sizes, consisting of 4,630 participants.
•Reported research methods, including participants,
instructional interventions, instruments, research
design and results.
•Employed a quasi-experimental or experimental
research method.
•Study established pretest equivalence and employed
a statistical control for pretest differences.
•The experimental group utilized some form of online
digital literacy practice involving a social media
platform or Web 2.0 tool.
•Control groups received traditional instructional
delivery methods and practices.
•Examined the impact of online digital literacy
practices on learning outcomes.
•Included higher education participants.
•The research presented in one study did not overlap
research from another study.
•The article was written in English.
Introduction
ABSTRACT
This paper reports the findings of a Stage I
meta-analysis exploring the effectiveness of
online digital literacy practices performed
through social media tools in higher education
classrooms as measured by learning outcomes.
An extensive literature search culled more than
500 potential articles and resulted in a sample of
51 representative quasi-experimental studies,
consisting of 4,630 total participants. Fifty-one
effect sizes were extracted and yielded a
moderately positive statistically significant
weighted average effect size of g+ = 0.315,
k=51, p <. 01. This overall effect size
suggests that integrating online digital
literacy practices into the higher education
classroom benefits students on measures of
academic achievement and offers
significant learning support.
In addition, several moderator variables were
tested to determine what factors and literacy
practices impact learning outcomes; namely,
field of study (STEM, ARTS), conceptualization of
tool (social learning theory, delivery tool), peer
interaction (yes, no), modality (blogs,
collaborative communication text-based
technologies, i.e. forums and wikis; podcasts,
and virtual worlds), practice (consuming,
prosuming behaviors), and the learning outcome
measured (course project, knowledge-based
exam); moderator effect sizes were
statistically significant for the following
variables: conceptualization of the tool,
practice, and learning outcomes.
RESEARCH QUESTIONS
• What is the effect of digital literacy practices
on learning outcomes in formal educational
settings?
• How do pedagogical variables; such as field of
study, peer interaction, form of online
modality, practice, and learning outcome
moderate this effect?
The Impact of Digital Literacy Practices on Learning Outcomes inThe Impact of Digital Literacy Practices on Learning Outcomes in
Higher Education: A Meta-AnalysisHigher Education: A Meta-Analysis
Jeannette Novakovich, Graduate Student in PhD Program, Educational Technology, Concordia University, Montreal, QuebecJeannette Novakovich, Graduate Student in PhD Program, Educational Technology, Concordia University, Montreal, Quebec
Nicole Fournier-Sylvester, Graduate Student in PhD Program, Educational Studies, Concorida University, Montreal, QuebecNicole Fournier-Sylvester, Graduate Student in PhD Program, Educational Studies, Concorida University, Montreal, Quebec
Jeannettenovakovich@gmail.com, 514-755-3987Jeannettenovakovich@gmail.com, 514-755-3987

2. Introduction
DISCUSSION
The overall effect of digital literacy practices on learning outcomes
The overall results of this Stage 1 meta-analysis (g+ = 0.315, k=51, p < .01) indicate that on average, learners who
engage in digital literacy practices in the form of a social media technology in higher education achieve higher positive
learning outcomes than students who engage in traditional literacy practices. Our results are fairly close to those
reported by Richard Schmid et al (2014) in a meta-analysis on the overall effect of the impact of technology on learning
outcomes in higher education (g+ 0.27, k=879, p < .05) and a meta-analysis conducted by Robert Bernard et al (2014)
on the impact of blended learning or computer-supported online learning combined with classical instruction, (g+ =
0.334, k = 117, p< .001). Although these results cannot answer Tess’ (2013) question about whether or not digital
technologies represent a “fundamental shift in the way students learn,” (p. A65), they do support Bennett’s (2012)
contention that the content creation, collaboration and sharing that is facilitated by digital literacy practices through
online social media in higher education can have important learning benefits.
The impact of conceptualization, practice and measured learning outcome
First, the significance of the moderator variable Conceptualization suggests that the approach taken to the integration
of the practice will impact to some extent learning outcomes (g+ = 0.325, k=51, p < .002; Social learning theory
g+= .390, and Delivery tool, g+= -.013). For example, using social media as a delivery tool (ie. posting a podcast of a
lecture) had no impact on learning outcomes. Effective uses of social media in the classroom are thus grounded in a
socially situated and constructivist approach to teaching and learning.
To further support the finding of the variable conceptualization, the significance of the moderator variable Practice,
suggests that learning outcomes appear to be considerably higher when students are called upon to produce knowledge
rather than consume it: g+ = 0.326, k=51, p < .022, Prosuming g+= .504, and Consuming g+=.042. Consuming
practices have basically no impact on learning outcomes and register with an effect size close to zero or nil. This
moderator variable was chosen based on Lin et al (2010) theoretical framework for developing digital literacy practices
by distinguishing between function and critical “consuming” (reading/viewing) and “prosuming” (producing/creating) of
the works of others.
And lastly, the significance of the moderator variable Learning Outcome suggests that the content produced through
digital literacy practices exceeds content produced through traditional practice: g+ = 0.235, k=51, p < .02, Exam
g+= .168, and Project g+= .719. Our findings support Bennett’s (2012) findings that the main benefits that come from
the “effective use of Web 2.0” is “particularly through student content creation and sharing” (p. 533).
The effectiveness of the integration of social media in the classroom is dependent on how a teacher utilizes its capacity
for interaction and the development and sharing of projects. Effective uses of social media in the classroom are thus
grounded in a student-centered and constructivist approach to teaching and learning. Greenhow (2011) summarizes
that integrating social media tools in learning promotes a more student-centered course. Students are better able to
create their own understanding of content when creating with these tools.
Thus, the integration of social media also facilitates active engagement with the world. Learners create meaning, by
experiencing the world and engaging in meaningful ways.
References
Bennett, S., Bishop, A, Dalgarno, B., Waycott, J., & Kennedy, G. (2012). Implementing Web 2.0 technologies in higher
education: A collective case study. Computers & Education. 59: 524-534.
Bernard, R., Borokhovski, E, Schmid, R., Tamim, R. & Abrami, P. (2014) A meta-analysis of blended learning and
tcchnology use in higher education: from the general to the applied. Journal Computers in Higher Education. 26: 87-
122.
Greenhow, C., Robelia, B., & Hughes, J. E. (2009). Learning, teaching, and scholarship in a digital age. Educational
Researcher, 38(4), 246–259. doi:10.3102/0013189X09336671.
Lin, T., Li, J., Deng, F. & Lee, L. (2010). Understanding New Media Literacy: An Explorative Theoretical Framework.
Educational Technology & Society, 16(4), 160-170.
Schmid, R, Bernard, R, Borokhovski, E. Tamim, R., Abrami, P., Surkes, M, Wade, C., & Woods, J. (2014) The effects of
technology use in postsecondary education: A meta-analysis of classroom applications. Computers & Education. 72:
271-291
RESEARCH QUESTIONS
• What is the effect of digital literacy practices
on learning outcomes in formal educational
settings?
• How do pedagogical variables; such as field of
study, peer interaction, form of online
modality, practice, and learning outcome
moderate this effect?
DEFINITION OF
TERMS
Jeannettenovakovich@gmail.com, 514-755-3987Jeannettenovakovich@gmail.com, 514-755-3987
The following terms are essential for understanding the underpinnings of
the research questions:
Literacy event. Defined by Bhatt, (2012) “In analysis, researchers begin
with a literacy event as an empirical occasion involving interaction around a
written text, from which they then infer the literacy practices” (p. 291).
Literacy practices. This type of activity involves the creation of
knowledge, values, attitudes, beliefs and feelings correlated with the
reading and writing of texts within a social context (Street 1984, Baynham
1995, Barton & Hamilton 1998, Barton et al. 2000).
Digital literacy practices. As established for the purposes of this meta-
analysis, digital literacy practices are comprised of literacy activities that
involve interaction around digital content, including, texts, images, sound,
and video.
Prosuming. Digital literacy practices where knowledge is created or
generated (Lin et al, 2010).
Consuming. Digital literacy practices where knowledge is consumed.
Social media. “ Internet based applications that build on the ideological
and technological foundations of Web 2.0, and that allow the creation and
exchange of user generated content” (Kaplan & Haenlein, 2010, p. 61), and
encompass “(a) social networking sites, such as Facebook, Twitter, and
LinkedIn, (b) media sharing sites, such as YouTube and Flickr, (c) creation
and publishing tools, such as wikis and blogs, (d) aggregation and
republishing through RSS feeds, and (e) remixing of content and
republishing tools,” (Greenhow, 2011, p. 140).
Blogs. Individual digital spaces where individual learners can consume
and /or prosume knowledge and other learners can interact through a
commenting function.
Podcasts. Video and audio online digital content that learners can consume
and/or prosume.
Collaborative communication technology. Early and current online text-
based technologies that allow learners to communicate collaboratively,
including wikis, forums, and other online communication tools.
Virtual worlds. A digital literacy practice where learning takes place
through online simulated worlds designed for formal learning activities, such
as Second life.
Learning outcome. A measured gain of knowledge or skills by the
learner as measured by a valid instrument.
Project. Learning outcomes measured through the creation of course
content by a literacy event.
Exam. Learning outcomes measured through an examination of course
material.