KHOO, Kay (The University of Hong Kong) http://citers2012.cite.hku.hk/en/paper_546.htm

1. Title: The framework of viewing and
representing skills for the engagement with digital
texts
Kay Yong, Khoo
Faculty of Education, The University of Hong Kong
CITE Research Symposium 2012
(June 15th )

2. Literature Review
CITE Research Symposium,2012 2

3. The meaning of literacy in
the 21st century has
changed markedly with
emerging and now dominant
technologies: that is, the
move away from writing to
the new digital text mode,
and from the medium of the
books to that of the
computer screens(
Kress, 2010).
CITE Research Symposium,2012 3

4. The need to redefine notions of reading, speaking,
listening and writing as the basic skills in language learning
is critical, as these skills are not adequate to address
digital multimodal literacy.
CITE Research Symposium,2012 4

5. Young people are frequently described as digital
natives as a result of their extensive experience of
digital computer technology (Buckingham & Willett,
2006; Prensky, 2001).
CITE Research Symposium,2012 5

6. In “Growing Up Digital”, Tapscott (1998) states that
technology is changing the practices and
preferences of young people in a series of areas –
namely, cognition, play, learning and etc.
CITE Research Symposium,2012 6

7. There are skills to be derived from children’s
intuitive understandings arising from their natural
interactions with digital text.
CITE Research Symposium,2012 7

8. There is an acknowledgement that the English Curriculum has to
evolve according to the changing world to prepare children for the
opportunities and challenges of life in the 21st century.
Ministry of Education Australia has a long
MOE Singapore has (2006) of Ontario, history of incorporating In the UK, a new
also introduced new Canada, has multimodal texts into primary curriculum is
English Language incorporated the context of English being reviewed and this
Curriculum in 2010 for multimodal texts into learning (Curriculum will be implemented in
primary and secondary the curriculum for Corporation., 1994; New 2011. “Viewing”, is
schools to be young children as early South Wales Board of defined as a skill
implemented from 2010. as at stage one (p. 45). Studies, 1998). In the necessary for
In the new curriculum, Multimodal texts are recent outlined understanding and
viewing and representing referred to as media Australia National responding to
skills are introduced as texts in the curriculum, English Curriculum, information, and
receptive and productive and are introduced in systematic exploration “broadcasting” is
skills to incorporate a the English Curriculum and production of identified as one of the
wide range of literacy guide as one of four multimodal texts have key skills required to
information/functional strands: oral been introduced present ideas and
texts (Singapore, 2010) communication, throughout the school opinions. (Department
reading, writing and years (National for Children schools and
media literacy. Curriculum Board, families UK, 2009).
2008).
CITE Research Symposium,2012 8

9. However, in Hong Kong, the government-
mandated primary school’s curriculum of English
Language has not incorporated digital text as an
integral aspect of Language learning (
Education and Manpower Bureau HK, July 2004).
CITE Research Symposium,2012 9

10. There is an emerging gap between children’s out-
of-school digital practices and practices in the
primary school English Language classroom in
Hong Kong (which remain focused on linguistic
learning).
CITE Research Symposium,2012 10

11. The literature suggests four areas of digital
practices that impact the engagement on screens:
Mode Contextual Links Navigation Multi-modes
Affordances The ability to link The ability to The ability to
The ability to apply the elements of information navigate on screens recognize and create
affordances of modes in different modes (Leeuwen, 2005; elements in different
in meaning making ( contextually in spatial or modes (Jewitt, 2006;
Martinec & Leeuwen, 2009
Jewitt, 2006; temporal layouts ( ). Len Unsworth, 2008);
Kress, 2003; Kress, 2010;
Martinec & Leeuwen, 2009 Leeuwen, 2005);
);
CITE Research Symposium,2012 11

12. Theoretical
Background
CITE Research Symposium,2012 12

13. Cognitive theory of multimedia learning
Selecting relevant Organizing the
words from the selected words
presented text or into a coherent
narration verbal
representation
Organizing Integrating the
Selecting relevant selected images pictorial and
images from the into a coherent verbal
presented pictorial representations
illustrations representation and prior
knowledge
CITE Research Symposium,2012 13

14. Research
Questions
CITE Research Symposium,2012 14

15. With the new digital text practices that are found in
literature, the gaps of knowledge that need to be filled. The
gaps lead to three research questions:
1: What digital literacy practices emerge from the participating
children’s out-of-school technology use?
2: How have these practices been extended to school activities?
CITE Research Symposium,2012 15

16. Research
Methodology
CITE Research Symposium,2012 16

17. The study explored four cases of primary school-aged
children. The methodology of the study was qualitative
(Gay, Mills, & Airasian, 2006; Merriam, 1988, 1998a,
1998b, 2009; Yin, 1994).
CITE Research Symposium,2012 17

18. CITE Research Symposium,2012 18

19. Research
Results
CITE Research Symposium,2012 19

20. Results: The digital skills developed by the participants
Participant One: Angel, a multi-mode representing practitioner
Three digital artifacts were selected from Angel’s messages
listed in her Facebook. The researcher found that she had
acquired the following abilities:
She was competent in both To apply affordances of To link elements of To navigate information
recognizing and creating modes in meaning making. different modes on screens with
messages with elements of Images is to provide meaningfully in layouts. hypertexts. They afford
different modes -, i.e. video, depictive information The links integrate readers interaction with
verbal narrations, images (Kress, 2010). elements of information in text in a manner that is
and printed texts. Angel’s Writing affords the different modes meaningful to them (
three selected digital description of ideas (Kress, contextually in spatial or Jonassen, 1986;
artifacts, the messages 2010). temporal layouts ( Leeuwen, 2005;
consisted of elements in Videos provide information Kress,2010; Martinec & Leeuwen, 2009
different modes: i.e. that demand changes of Leeuwen, 2005); ). Angel could quickly
images, sound, verbal time or space, or adapt to the navigation
narrations, video and information that is potential and the
writing. perceived through sight and processing of digital
hearing (emotion, attitudes, texts of different modes.
and etc.).
CITE Research Symposium,2012 20

21. Affordances of Images Affordances of Writing Contextual Linking
Angel applied a newspaper Angel urged readers of her Angel demonstrated her
cutting to show the authentic message to tell more people (as understanding to elaborate on
aspect of the message as in in Figure 2) about the news she the title (writing) with the
Figure 2. had read that live puppies were newspaper cutting (an image)
used as shark bait. as in Figure 2.
Please help to disseminate…
It was very cruel and inhumane…Do you notice that
the dog is crying? Please disseminate this message!
The title of the newspaper cutting: “Live
puppies are used as shark bait”( 活小狗慘變
餌釣鯊魚 )
Figure 2. Angel's digital artifact with the title “Puppies
As Fishing Bait.”
CITE Research Symposium,2012 21

22. Affordances of Images Affordances of Writing Contextual Linking
In Figure 3, she used her She determined to change the Angel demonstrated her to
serious look photo to reflect image she was projecting of complement the body
her friends’ perception of her. herself by using smiley faces to message (writing) with a photo
get to know more friends (as in (an image) as in Figure 3.
Figure 3).
The title: “Your first impression for friends of
different gender (for girls only).“ 妳給異性的
第一印象 ( 女生玩 )”
The body message stated: “AC has completed
a psychological test. The test showed that she
felt she always needed to insert a smiley face
so that she is viewed as approachable by
friends of different gender.”
Figure 3. The digital artifact by Angel with
the title: A Psychological Test
CITE Research Symposium,2012 22

23. Affordances of Videos
Angel applied a video clip that showed a series of images with different narration for each, “The Handsome Guys
and The Ugly Guys (http://www.youtube.com/watch?v=vZhSvMRiXtE)”. The narrations with illustrations compared
how people perceived handsome and ugly boys performing a series of heterogeneous actions. For example: when a
handsome boy reads a book, people would praise him as “a decent boy”, but an ugly boy reading a book would be
termed a “bookworm”.
CITE Research Symposium,2012 23

24. On top of the four digital practices that have been noted from the literature,
two additional practices emerged specifically from the Stage One study.
Angel had also acquired the following abilities:
To assimilate digital functionalities in as a part of engagement.
and
Some clear patterns emerged from Angel’s viewing engagement with these elements.
There were two macro processes in Angel’s self-regulated engagement with these
elements, she selected elements and she integrated them with different links to form a
semantic path of engagement.
CITE Research Symposium,2012 24

25. Table 2.
Viewing and representing skills developed from children’s out-of-school digital
practices
Skill Area Skill
One Multi-mode The skill to recognize or to create elements of various modes.
Two Affordances of mode The skill to apply or to engage with the affordances of various modes.
Three Contextual Link The skill to interpret and create contextual links (in spatial/temporal layouts)
between elements.
Four Navigation The skill to move around a screen to engage with information.
Five Digital functionality The skill to assimilate digital functionalities in one’s engagement on screens.
In this study, five viewing and representing skills are identified (as in Table 2)
in response to the first research question. Four of these skills - skills 1,2,3 and
4 - were derived from practices identified in the literature review before the
data collection (Jewitt, 2006; Kress, 2010; Kress & Leeuwen, 2007;
Leeuwen, 2005; Martinec & Leeuwen, 2009). An additional skill emerged from
the study.
CITE Research Symposium,2012 25

26. The different processes were identified when Angel engaged with digital
text by using the five skills. Analysing these processes can lead to the
development of the conceptual overview to frame the other five skills (as
in
i Table 3).
Table 3.
The framework of viewing and representing skills
Macro Process Element Selection Element Integration
Multi-mode Contextual Link
The skill to recognize and create elements of The skill to interpret and create contextual links (in
different modes to form information. spatial/temporal layouts) with different elements to
form information.
Skills Affordances of mode Navigation
The skill to apply and engage with the The skill to move around a screen to integrate
affordances of different modes in elements to different elements to form information.
form information.
Digital functionality
The skill to assimilate digital functionalities The skill to assimilate digital functionalities to
in elements to form information. integrate elements to form information.
CITE Research Symposium,2012 26

27. Discussion
of results and the research’s
implication.
CITE Research Symposium,2012 27

28. A pedagogical implication arises from this research. Each participant
had different preferences in relation to the ways in which they engaged
with digital texts.
• Multimodal practitioner
• Multimodal written text representing
• practitioner Digital savvy practitioner
• Viewing practitioner
This finding implies that education should adopt more individualized
approaches to allow learners to design their own style and practice in
learning.
CITE Research Symposium,2012 28

29. The researcher found that skills
1,4 5 & 6 were utilized, while
skills 2 & 3 were not in
evidence. The second research
question will be answered
based on these findings.
Although skill one (multi-mode) is
extended as images in formal
learning, the affordances of
images (skill two) and the
contextual links between images
and writing mode (skill three)
were not utilized. The result is
consistent with the findings of
other studies where images were
found to be not utilized
meaningfully in learning (
Daly & Unsworth, 2011;
Len Unsworth, 2008).
The children extended skills
4,5 & 6 intuitively into their
formal learning on screens.
Furthermore, feedback from
the teachers indicated the fact
that they were also aware of
the skills.
CITE Research Symposium,2012 29

30. The studies from the four cases
show that English language
learning at school focuses on
linguistic development (such as
grammar and reading
comprehension) as observed in
classroom observations and
the interviews with the
teachers.
The children extended skills 1,
2 & 3 into their independent
schoolwork. Besides creating
their messages meaningfully
with images and writing, they
were also skilled at creating
meaning with contextual links
with non-linear spatial layouts
(skill three).
However, Skills 4,5 & 6 were
not in evidence in their
independent schoolwork: There
was no teacher-led promotion
of schoolwork to be completed
on screens with digital texts;
the teachers requested that
independent schoolwork be
printed to hard copy.
CITE Research Symposium,2012 30