2. 1. Introduction
2. The Background this research
3. The Objective this research
4. Research Questions
5. Sample and context
6. Research Design
・ In-class Website Analysis Assignment
・ Actual Experiment
・ Data Analysis – Use of Coders
7. Findings
8. Conclusions
3. Literature on computer assisted language
learning is mostly silent on how web-based
design analysis could be effectively used as a
tool and framework for developing critical
thinking skills and language proficiency in an
EFL classroom.
This article reported on
how EFL learners perform
with English website analysis
tasks in a language reception
and product context.
4. There is substantial research in language studies and
cognition that establishes cognition and language
development to be closely related
Educators have identified multiple features and
elements of reading and writing to have always
influenced thinking skills to a large extent.
There has always been a strong appeal to promote
higher order thinking in ESL and EFL classrooms, and
research has clearly focused on the need to foster
critical thinking in a foreign language classroom
However, unfortunately language learning and thinking
skills were almost always treated as independent
processes
5. This study focused on extensive and sustained
content analysis using information technology
resources.
This study establishes that such attempt helps
with both linguistic and cognitive information
processing ability.
This study is also influenced by the fact that we
have shifted from Web 1.0 towards Web 2.0
where there is an increased emergence of
computer-mediated communication, social
networking and active interaction between the
user and the web environment.
6. How did the EFL readers perform with
various design questions as asked during
website analysis?
Is there any significant difference
between coders who graded
responses to the design
questions suggesting significant
difference between responses to
a design question and/or suggesting that
one or more coder(s) have not understood
the questions and responses correctly and
resultantly could not use the assessment
rubric correctly for grading the design
responses?
7. Participants (N=17) are junior level students (age
group: 18-20 years) in their third year
undergraduate program specializing in computer
science in a Japanese technical university
With this specific elective course named Writing
and Design for World Wide Web, students mostly
focused on the process of online writing,
designing and analyzing
websites based on design
principles, besides designing
concept maps on websites
they analyzed.
8. As part of the website analysis assignment,
students were asked to study a specific
website in a chosen domain (e.g., education,
entertainment, government, tourism, sports
etc)
Students provide open-ended responses to 8
standard questions asked of them.
9. 1. Explain whether the organization of information in the site is user-friendly or not?
2. Explain whether the presentation of content is appealing or not?
3. Explain whether the effective use of technology is demonstrated?
4. Who is the target audience? Is the website appropriate for the projected audience?
5. Explain the quality of the text content.
6. Is the information accessible?
7. Explain whether the resources use real-world situations.
8. Here are some common reasons for building this website. Rank them in order of
importance to you. Do you have a reason that is not listed?
Open-ended Design Questions asked of the Participants
10. The experiment was conducted in a controlled
environment as an in-class activity, and over two
weeks.
students analyzed the Belize tourism website
based on the 8 open-ended questions asked
during the same assignment that happened over
the previous weeks
Students had one-week to complete the analysis,
besides the 90 minutes of class time where they
could consult their friends.
To encourage writing and proper explanation,
the minimum word limit for the assignment was
set at 500 words.
11. Three undergraduate students (not part of the class
with the sample) who took the same class at an
earlier semester were appointed as coders with the
task of grading the first week assignment where
readers participated in an open-ended evaluation of
the Belize tourism website.
The coders were given a set of criteria on the basis of
which they graded each open-ended response, for all
the 8 questions assigned.
Each of the 8 open-ended responses for each of the
17 participants was rated thrice, once by each of the
three coders.
The group (including the project supervisor and the
three coders) then discussed each grade for each
question and criterion.
13. N Minimum Maximum Sum of Mean Scores Std. Deviation
Question1
18 9 16 14.56 2.479
Question2
18 3 16 12.56 3.110
Question3
18 0 14 11.56 3.585
Question4
18 3 15 11.50 4.148
Question5
18 3 15 11.33 3.804
Question6
18 2 15 11.28 4.254
Question7
18 0 15 10.33 4.366
Question8
18 6 15 11.67 2.990
Valid N (list
wise)
18
Mean and SD Values for Each Question on 6 Criteria and Each Criteria Scored Thrice by Three Different Coders
14. Descriptive Statistics for Student Score Calculated over Six Criteria by Coder A
N
Minimum Mean
Score
Maximum
Mean Score
Mean of Total
Mean Score Std. Deviation
S1 6 3 7 5.33 1.366
S2 6 3 7 5.67 1.506
S3 6 2 7 4.00 2.098
S4 6 2 8 4.17 2.639
S5 6 1 6 3.50 2.258
S6 6 5 8 7.00 1.549
S7 6 1 6 3.33 2.582
S8 6 1 1 1.00 .000
S9 6 3 8 5.00 2.098
S10 6 1 8 4.00 3.098
S11 6 3 8 5.83 2.317
S12 6 3 7 4.17 1.602
S13 6 3 8 7.00 2.000
S14 6 2 8 6.33 2.422
S15 6 1 8 3.17 2.639
S16 6 3 8 4.83 1.941
Valid N
(list wise) 6
15. Descriptive Statistics for Student Score Calculated over Six Criteria by Coder B
N
Minimum Mean
score
Maximum Mean
Score
Mean of Total Mean
Score Std. Deviation
S1 6 3 8 6.00 2.098
S2 6 4 8 6.50 1.643
S3 6 4 7 5.67 1.033
S4 6 3 8 5.83 2.137
S5 6 5 8 6.83 1.472
S6 6 6 8 7.50 .837
S7 6 3 8 5.67 2.338
S8 6 1 2 1.50 .548
S9 6 3 8 6.17 2.483
S10 6 2 8 6.00 2.449
S11 6 3 8 6.50 1.871
S12 6 4 8 6.50 1.761
S13 6 6 8 7.67 .816
S14 6 3 8 6.17 1.941
S15 6 4 8 6.33 1.633
S16 6 5 8 6.83 1.169
Valid N (list
wise) 6
16. Descriptive Statistics for Student Score Calculated over Six Criteria by Coder C
N
Minimum Mean
Score
Maximum Mean
Score
Mean of Total Mean
Score Std. Deviation
S1 6 6 8 6.83 .753
S2 6 7 8 7.67 .516
S3 6 5 7 6.67 .816
S4 6 6 8 7.50 .837
S5 6 6 8 7.50 .837
S6 6 5 8 7.50 1.225
S7 6 6 8 7.50 .837
S8 6 1 1 1.00 .000
S9 6 5 8 6.33 1.033
S10 6 7 8 7.83 .408
S11 6 8 8 8.00 .000
S12 6 6 8 7.67 .816
S13 6 6 8 7.50 .837
S14 6 6 8 7.50 .837
S15 6 5 8 7.00 1.549
S16 6 6 8 7.67 .816
Valid N (list
wise) 6
17. This study, Coders were a test of not only
understanding what constitutes valid
information, and good organization of
response, but it also required ability to read
through the criteria rubric used in the study
demonstrate at least moderate levels of
language proficiency.
