1. An E-assessment Model for Supporting Collaborative
Knowledge Building in a Social Software
Ole C. Brudvik
Anders Nome
NKS Nettstudier
www.nks.no
2. NKS Nettstudier
• 4000 distance education students
• 100 years in 2014
• Total of 3 million students so far
• Located in Norway
• Focus on university/university college degrees
• Collaborate with many universities/university
colleges
• Main subject areas:
– Business
– Media & Communication
– Pedagogy & Society
– Health & Social Studies
3. Motivation for Study
“Remix means to take cultural artifacts and
combine and manipulate them into new kinds of
creative blends” (Knobel & Lankshear, 2008)
• Develop an e-assessment model for evaluating
students engagement in remixing new creative
knowledge products using social software.
• Employ this e-assessment model to evaluate
students engagement with knowledge building
using a social content network – www.diigo.com
4. Purpose of study
• Develop an e-assessment model for supporting
and assessing learning by knowledge building
using social software.
• Engage the students to work collaboratively with
problem-based knowledge building using social
software.
• The study evaluate how the e-assessment
support students use of social software for
knowledge building tasks.
• The study evaluate what affordances the social
software have for engaging in collaborative
knowledge building.
5. Web 2.0/Social Software - Spaces and Places for Knowing
• Explore in virtual and cross-cultural settings.
• Multimodal experience and conventions.
• Supports communities and collaboration.
• Manipulation and sharing.
• Consumers to become producers.
• Working with ideas and making links between
sources.
• Add and remix their own interpretations and
meanings.
(Owen and colleagues, 2006; Manovich, 2001;
Jenkins, 2006)
6. Knowledge in the Knowledge Society
• Weinberg (2006) describes knowledge as no
longer organized as trees, but as «pile of
leaves».
• The organization of knowledge in digital forms
allows the same knowledge to be organized by
many people in many ways through different
digital constructions.
• Relationships with ideas determined by context,
and each change in context can result in the
ideas being reworked to meet the particular
need.
(Weinberg, 2006; Siemens, 2006)
7. Knowledge
• The same idea might change its modality of
representation and its structure at different times
in different places (Siemens, 2006).
8. Instructional Method: Knowledge Building
• Based on the need to be able to work creatively with
knowledge.
• Challenge is to engage learners in a developmental
trajectory of creative knowledge production.
• Knowledge building defined as «the production and
continual improvement of ideas of value to a
community, through means that increases the
likelihood that what the community accomplishes will
be greater than the sum of individual contributions
and part of broader cultural efforts.»
(Bereiter & Scardamalia, 2003)
9. Instructional Method: Knowledge Building
• «The goal is to advance the frontiers of
knowledge as they perceive them.»
• Calls for «deep constructivism».
• «Learning is an internal, unobservable process
that results in changes in belief, attitude or skill.
Knowledge building, by contrast, results in the
creation or modification of public knowledge…»
(Bereiter & Scardamalia, 2003)
10. Dialogic Literacy – Fundamental for Work in Knowledge
Society
• «In every kind of knowledge-based, progressive
organization, new knowledge and new directions are
forged through dialogue….The dialogue in the
knowledge age organizations is not principally
concerned with narrative, exposition, argument, and
persuasion but with solving problems and developing
new ideas. Higher-prder knowledge age skills have to
do with collaboration, initiative, communication, and
creativity»
• Knowledge building has the potential to engage the
students in dialogic literacy: «the ability to engage
productively in discourse whose purpose is to
generate new knowledge and understanding»
(Bereiter & Scardamalia, 2005)
11. Diigo – Social Content Network www.diigo.com
The students used the browser toolbar to work with information resources on the web
The students used a
Diigo Education Group to
Collaborate on the knowledge
building task
12. Methodology
• Study conducted over 10 days with 1 learning
task.
• 4 participants volunteered.
• Purpose-based sampling (Miles & Huberman,
1984) based on the argument by Owen and
colleagues (2006) that no longer are learners
constrained by institutional borders, but they can
explore in virtual and cross-cultural settings.
• Data collection: screenshots of work in Diigo –
social content network. Work stored in the
students Diigo Education Group.
13. Participants
• Participants where recruited from PLN.
• 3 in Norway and 1 in Australia.
• All of them studying or/and working with
educational technology.
• The students did not know each other.
• 2 of them have used Diigo before.
The participants:
1. Thomas: a 21 years old student in the master in
ICT-supported Learning program at a University
College in Norway. Work as a teacher in a primary
school in Norway.
14. Participants
2. John: a 37 years old student in the master in
ICT-supported learning program at a University
College in Norway. Work at an e-learning company
and completing a Dr. Philos in instructional
technology.
3. Andrew: a 31 year old educational technology
researcher and a candidate to a PhD in Education at
a university in Australia.
4. Peter: a 47 years old history teacher at an upper
secondary school in Norway and a trainer for upper
secondary school teachers in pedagogical use of
technology.
15. Design of Learning Task and E-assessment Model
• Engage learners with ideas/resources on the
Web.
• Engage learners in dialogic literacy through
knowledge building using social content network.
• Open-ended task with multiple possible solutions.
16. Design of Learning Task and E-assessment Model
• Competence 1: ability to find and collect relevant
ideas/resources.
• Competence 2: ability to find, collect/sample,
organize and describe the content in the
ideas/resources by choosing descriptive tags of
content.
• Competence 3: ability to add own interpretation
and meaning to the content of the
ideas/resources.
• Competence 4: ability to combine and remix
ideas/resources into a new construction of
idea/proposal/solution to the task.
17. Design of Learning Task and E-assessment Model
Assessment model given to the students at the start of the learning task.
Purpose: 1. Support summative & formative assessment; 2. Self assessment;
3. Peer assessment; and 3. Transparency (Dalsgaard & Paulsen, 2009).
18. Results
• 30 bookmarks and 130 visits to the Diigo Group
• Most of the engagement on competence 3 with
21 occurrences:
Ability to add own interpretation and meaning
to the content of the ideas/resources.
19. Results: Competence 1
• The students bookmarks was initially not using descriptive tags of the
content in source.
• Formatively scaffold the use of tags with pre-defined tags.
• Edited one bookmark by adding tags to show how the tags could be
used to direct a focus on using the tags to describe the content in the
source.
• Purpose of this was to create an organized "pile of leaves" in the
knowledge folksonomy that the group was to work on further.
• After, descriptive tags were used more.
• A shared purpose for using the tags is needed!
• Scaffolding of using tags is a need!
21. Results: Competence 2
• The students used descriptive tags such as: game, addiction,
psychological obsessive-compulsory to direct focus to what content
in this resource is the focus in their work with this resource.
• The students sampled into the Diigo Group space the pieces of
content that the tags was linked to.
• Also as they worked with this they engaged with competence 3 as
they added their own interpretation by dialogically adding comments
to the tagged sampled content.
• The process of sampling content from sources did not need any
scaffolding and seemed to be an affordance for knowledge
building that Diigo social content network supports.
22. Results: Competence 2
The students engaging in competence 2 by tagging the content
in the source and sampling the content in focus.
23. Results: Competence 3
• The students started to engage with dialogic
literacy at this stage.
• They used comments and sticky notes to add
their own interpretation and meaning to the
tagged content in their own bookmarks and to
comments/sticky notes in other students
bookmarks.
• Hence, an affordance of using Diigo social
content network is in supporting the
students to engage in dialogic literacy by
adding interpretation and meaning to all the
bookmarks in the collaborative Diigo Group.
24. Results: Competence 3
The students collaborate on adding interpretation and meaning
to the content in the source:
25. Results: Competence 4
• The most challenging part of the learning task.
• The students need to switch focus from dialogically creating an
organized "pile of leaves" knowledge folksonomy by adding
interpretation and meaning to using this to propose a
solution/solutions to the task.
• Hence, only two students attempted to reach this level.
• The affordances of Diigo do not have good support for this
process!
• Hence, more scaffolding is needed to extend the bookmarks to
solutions or combine Diigo with another tool to do this part (e.g.,
“post to blog” from Diigo).
26. Competence Level 4
Attempt to use discussion Topic in Diigo Group for Competence 4.
The students did not choose to use this
28. Conclusions
• The e-assessment model provides a useful
analytical lens to evaluate the affordances of
using different social software for knowledge
building.
• It provides a useful framework for designing e-
assessment for supporting students working on
knowledge building task using different social
software.
• The e-assessment model in action could be better
as a summative assessment if the number of
occurrences of work with competence levels
would determine the final score instead of
whether or not they do a competence.
29. Conclusion
• The affordances of using Diigo – social content
network for knowledge building tasks:
– Require scaffolding a shared purpose and how to do the
tagging of content relevant to the task in the sources.
– The affordance of using Diigo is centered at competence
2 & 3: find, sample and add interpretation and meaning
– For the students to engage with competence 4 the
extension of the dialogic work on competence 3 need to
be scaffold for work on proposing solutions to task
– …or combine Diigo with another tool where the students
can bring in the work on competence 3 to create and
remix solutions to the task employing the bookmark
dialogues.
30. Further Research
• A study designed with the needed scaffolding
and/or tool combination for achieving task
completion and longer dialogic literacy
engagement in the knowledge building.
• This can extend the knowledge building activity
to a competence level 5:
Competence level 5: Dialogically analyze and
evaluate the proposed solutions up against each
other to collaboratively remix a comprehensive
shared solution to the task.
