Why are Assessments in Indonesia are directed tothe Higher Order Thinking Skills (HOTS)? First: Internal & External Challenge Internal Challenge: 8 (eight) National Education Standards including management standards, cost standards, infrastructure standards, educator standards and education personnel, content standards, process standards, assessment standards, and graduate competency standards. External Challenge: Environment, Technology and Information, Creative Industries and International Education Development...etc

1. MDELT:
Study Pack and High Order Thinking Skills
and Evaluation
By:
Ari Famigia Torang (17716251001)
Elsa Maria Marahati (17716251018)
Emeral (17716251017)

2. Presentation Outline:
Curriculum 2013 and High Order Thinking Skills (HOTS)
What is HOTS?
Characteristics of HOTS Questions
Types of HOTS Questions
Steps in Creating Questions based on HOTS
Study Pack in the frame of Curriculum 2013
What is Common Core Standards?
Standards in Common Core
HOTS in Common Core Standards

3. The Example of HOTS in Reading
The Example of HOTS in Speaking and Listening
The Example of HOTS in Writing
The Example of HOTS in Assessment
Study Pack in the frame of Common Core Standards

4. Why are Assessments in Indonesia are directed to
the Higher Order Thinking Skills (HOTS)?
 First: Internal & External Challenge
Internal Challenge:
8 (eight) National Education Standards including management
standards, cost standards, infrastructure standards, educator
standards and education personnel, content standards, process
standards, assessment standards, and graduate competency
standards.
External Challenge:
Environment, Technology and Information, Creative Industries
and International Education Development

5.  Second:
Based on the results of the international study, the
Program for International Student Assessment (PISA)
shows reading literacy achievement, mathematical
literacy, and scientific literacy achieved by Indonesian
learners is very low.
In general, the ability of Indonesian learners is very low
in: (1) understanding complex information;
(2) theory, analysis, and problem solving;
(3) the use of tools, procedures and problem solving; and
(4) conduct an investigation.

6.  Therefore, based on the facts, there is a need for system
changes in learning and assessment. Assessment
developed by teachers is expected to encourage the
improvement of high-level thinking skills, increase
creativity, and build the independence of learners to
solve problems.

7. A. What is HOTS?
 HOTS is a concept of education reform based on learning
taxonomies or Bloom’s Taxonomy. HOT takes thinking to higher
levels than just restating the facts and requires students to do
something with the facts – understand them, infer from them,
connect them to other facts and concepts, categorize them,
manipulate them, put them together in new ways, and apply them
as we seek new solution to new problems.
 HOTS measures the ability to analyze (C4), evaluate (C5), and
create (C6).

8. Revision of Bloom's Taxonomy by Anderson & Krathwohl (2001) :

9. B. Characteristics HOTS Questions
 Measuring High-Level Thinking Skills
 Contextual Based Problems
 Using Various Types of Question

10. Stimulus:
 Stimulus is the basis for making questions. The stimulus
presented should be contextual and appealing. Stimulus is
from global issues such as information technology, science,
economics, health, education and infrastructure issues.
 Stimulus can be derived from the problems that exist in the
environment around educational units such as culture,
customs, cases in the region, or various advantages that
exist in certain areas. The creativity of a teacher greatly
affects the quality and variation of the stimulus used in
writing about HOTS.

11. Stimulus
Contextual
global issues
the problems that
exist
in the
environment
Appealing

12. C. Types of Questions in HOTS
Multiple
Choice
True or False/
Yes or No
Missing
Words
Short
Answer
Essay

13. D. Steps in creating Questions based on HOTS
 Analyze Basic Competence (KD)
 Arrange the questions grid
 Choosing an appealing and contextual stimulus
 Write the item of questions according to the question grid
 Create scoring guidelines (rubrics) and answer keys

14. E. Study Pack
Study Pack
Curriculum 2013 Common Core
Standard

15. F. Outline of Study Pack in the frame of
Curriculum 2013
 The Study Pack is designed based on Curriculum 2013.
 The Target Group is Senior High School Grade XI Social Science
Program.
 The tasks consisting of 3 Pre Tasks and 7 Task Cycle respectively.
 Assessing Students on Asking and Giving Opinion and Discussion
Text.
 Basic Competence of Asking and Giving Opinion is clearly stated in
the syllabus below, however, the discussion text is not available in
the syllabus.

16. Syllabus of Curriculum 2013 for Senior High School Grade
XI Social Science Program.

18. G. Evaluation of Study Pack of Curriculum 2013
Task 1:
The question “what video
might about?’ has the
students to simply guess
and state their opinion
about what they see. The
question refers to C1,
Remember.
So, this question is not
HOTS question.

19. Task 2:
Here, the students are
asked to tick items that
they know, and list the
items they do not know
and want to know further.
This activity is part of C1 as
as students still try to build
the concept of the topic by
listing and writing what
they know and want to
know. So, it is not HOTS
questions.

20. Task 3:
After watching the video, the
students answer the two following
questions based on captured
moments in video:
1. What number is the best
moment of the video?
2. What is your opinion about the
man who help the young girl?
The first question stimulates
students to recall what they have
watched, thus it is categorized into
remembering.
The second question asks the
students to explain their opinion
related to the character in video.
Thus, it is classified as C2,
understanding.

21. Task 4:
This task is categorized in C 1
about how to ask and give
opinion with teacher
guidance. There is no
questions involved.

22. Task 5:
As the follow-up of task 4, in the fifth
task, the students are asked to fill the
incomplete dialogue to make sure that
they really understand how to ask and
give opinion. This is categorized into
C3, applying, in where the students are
are intended to implement and use the
the knowledge they have understood
in answering the task. Still, it is not
HOTS questions.

23. Task 6:
The questions “What is your
and your friend’s opinion about
situation in the picture” is
aimed to make students ask and
and give their opinion orally.
Since the expressions of asking
and giving opinion are used, this
this task is classified into the
stage of applying.

24. Task 7:
In this task, the students are asked to
take position on an issue and argue
why they agree or disagree on that
issue by supporting their idea with
their knowledge and presenting it
orally. Since this activity has the
students to justify a stand, argue,
defend and support their idea, this
task refers to evaluation stage in
HOTS.
In addition, a new stimulus is
presented here that is based on the
existing problem as required in the
characteristics of HOTS question.

25. Task 8, 9, 10
The tasks are not
relevant to the
Syllabus as there is no
no Basic Competence
(KD) for writing a
Discussion Text. So, it
does not fulfill the
characteristics of HOT
Questions

26. H. Based on the Study Pack of Curriculum 2013, it can be
inferred that:
 The question of HOTS in this Study Pack is derived from contextual and
appealing stimulus since the topic is based on existing problem.
 On the other hand, The Study Pack is about Asking and Giving Opinion
& Discussion Text, however in the syllabus of Curriculum 2013 for XI
grade students of Social Science, there is no Discussion Text. So, the
Discussion Text is not relevant to the Study Pack. In addition, there are
LOTS questions than HOTS questions provided in the study pack.
Furthermore, there are some misspelled words in this study pack. Thus,
there are a lot of works needed on creating and providing HOTS
Questions in the Study Pack

27. What is Common Core Standards?
 Set of standards which describe the knowledge and skills
high school graduates expected to develop in order to be
successful in college and careers.
 Applied for kindergarten students until High school
students
 It is research and evidence-based and align with work
expectations
 Emphasize on critical thinking and problem solving skills.
 Set requirements not only for English language arts (ELA)
but also for literacy in history/social studies, science, and
technical subjects.

28. Standards in Common Core
 Reading: Text complexity and the growth of comprehension
 Writing: Text types, responding to reading, and research
 Speaking and Listening: Flexible communication and
collaboration
 Language: Conventions, effective use, and vocabulary

29. HOTS in Common Core Standards
HOTS
- critical thinking
- problem solving
skill
Standards
Assessment

30. The Example of HOTS in Reading
(Analyzing)
Standards in Reading for Grade 12:
 Analyze the structure of texts, including how specific sentences,
paragraphs, and larger portions of the text (e.g., a section,
chapter, scene, or stanza) relate to each other and the whole.
 Analyze how two or more texts address similar themes or topics
in order to build knowledge or to compare the approaches the
authors take.

31. The Example of HOTS in Listening and Speaking
(Evaluating)
Standards in Speaking and listening:
 Integrate and evaluate information presented in diverse
media and formats, including visually, quantitatively, and
orally.
 Evaluate a speaker’s point of view, reasoning, and use of
evidence and rhetoric.

32. The Example of HOTS in Writing (Creating)
Standard of Writing in CCS:
 Write narratives to develop real or imagined experiences or
events using effective technique, well-chosen details, and
well-structured event sequences.
 Write arguments to support claims in an analysis of
substantive topics or texts, using valid reasoning and relevant
evidence.
 Write informative/explanatory texts to examine and convey
complex ideas and information clearly and accurately
through the effective selection, organization, and analysis of
content.

33. The Example of HOTS in Assessments
(Grade 11)

34. Sample Performance Tasks for English Language and Arts
 Students compare two or more recorded or live productions of
Arthur Miller’s Death of a Salesman to the written text, evaluating
how each version interprets the source text and debating which
aspects of the enacted interpretations of the play best capture a
particular character, scene, or theme. [RL.11–12.7]
 Students provide an objective summary of F. Scott’s Fitzgerald’s
Great Gatsby wherein they analyze how over the course of the text
different characters try to escape the worlds they come from,
including whose help they get and whether anybody succeeds in
escaping. [RL.11–12.2]

35. Summary of Study Pack
 Thereotical Framework
It discusses Common Core Standards (CCS) and the reason why the
author is interested in developing study pack based on CCS.
 Target Group
Study pack is designed for language teachers of a general English Course
for adult learners.
 Approach
Approach used in this study pack is learning-centered pedagogist

36.  Organizations of activities in study pack
The activities are divided into three parts, namely pre-viewing activity, while viewing
activity and post viewing. The video that students view is a Thailand video entitled
“You Can Be Hero Too”.
 Objectives of study pack
- Linguistic competence : develop language skills (speaking, reading listening and
writing) with the aid of audiovisual material and visual. ii) practice their spoken
fluency through active participation in classroom task with their peers.
- Functional competence: develop confidence to communicate and interact in
English

37. - Social competence: appreciating their peers contribution to language learning
process through collaborative activity.
- Media development competence : increase understanding of media used in study
pack.
 Pre-Viewing activities
It includes prediction task, listening and reading activity
 While Viewing activities
It includes writing activity (note-taking and describing physical appearances) and
observation activity.
 Post-Viewing activities
It includes collaborative writing activity, talkshow (class discussion), individual
writing activity, finding information from internet and group project.

38. Evaluation of Study Pack (2)

39. Pre-viewing activity
Activity 1: Prediction Task
evaluation:
- Impossible time limitation
students are asked to work in pairs to
discuss a synopsis of a video they have
not watched yet for five minutes

40. Activity 2: Listening
In this activity, teacher reads a
synopsis and students take notes.
Then, students form groups and
reconstruct the synopsis. This activity
refers to creating stage in HOTS.
Activity 3: Reading
Teacher asks students to look up
unknown words in the dictionary
Then after watching video, students
write expressions found in the video
and find their meanings. This activity
refers to remembering stage in low
order thinking skills as the students
recall vocabulary they find in the
dictionary in order to translate the
expressions.

41. While Viewing Activity
Activity 4: Sequencing
evaluation:
This activity enables the students work
work together with their peers to
write plot of the film they have just
watched by numbering the sequence
and put into chronological order. This
activity aligns with CCS which
emphasizes sequencing technique in
narrative writing. In addition, this
activity refers to creating stage in
HOTS as it encourages students to
produce something.

42. Activity 5: Observation Activity
Students are asked to take notes while
watching the video. After that, they
work in group and write five difficult
questions for other groups to answer.
Groups that can answer correctly get
the highest scores.
This activity is not clear at all because
the author does not state what kind of
questions students should ask or
whether the questions should be
related to the video or not. In addition,
this activity does not refer to HOTS.

43. Activity 6: Eye Witness
students watch a film, make notes
of physical appearances of the
characters. Then, students work in
group and discuss their list of
descriptions. This activity refers to
low order of thinking
(understanding) as the students
discuss and explain their ideas
related to character descriptions
of the video to their peers.

44. Post viewing activity
Activity 7: Collaborative Writing Activity
Student work in groups and write the
frame of the film plot. The groups deal
with different contents, half of the groups
deal with what happened before and the
other half deals with what happens
afterwards. Then students read all texts
from other groups and choose the best
“after story”. This activity refers to creating
creating stage in HOTS as students
construct the frame of film plot. However,
the division of contents the students
should write is not stated clearly in this
activity.

45. Activity 8: Talkshow
in this activity, the class setting
will be like a talkshow where
there are some speakers and
audiences. In the end of the
discussion they make
conclusion together. This
activity does not refer to HOTS
as the topic of discussion is not
clearly stated.

46. Activity 9: writing an article
Students are asked to produce an
article using the plot of the film
as any information gained during
activity seven and eight. Students
should share their notes with one
another. Students’ activity of
producing an article refers to
creating stage in HOTS.

47. Activity 10: film funding
(modelling)
This activity enables students to
find information how to apply for
film funding. This activity does
not refer to any stages in HOTS

48. Activity 11: making your own
short film
This activity, students work in
group to make video that has
similar concept with “You Can
Be a Hero Too” video.
This activity refers to the
creating stage in HOTS.

49. Based on the evaluation, the weaknesses identified from this
study pack are:
 Important points are not clearly stated, such as the target
group and the main material.
 It does not fully align with common core standards which
emphasize on critical thinking and problem-solving skills.
 Some of the activities do not encourage HOTS
However, the strength of this study pack is in the authors’
selection of video which teaches moral values to students.

50. References
An Overview of Common Core Assessments. (n.d.) In Thoughtco. Retrieved in November,
20 2017, from https://www.thoughtco.com.
Common Core State Standards for English Language Arts & Literacy In History/Social
Studies, Science, and Technical Subjects. (n.d.) In Core Standard. Retrieved in November,
18 2017 www.corestandards.org/assets/CCSSI_ELA%20Standards.pdf
Pros and Cons of Common Core Standards. (n.d.) In Thoughtco. Retrieved in November,
20 2017, from https://www.thoughtco.com.
Warisdiono, E. et al (2017). Modul Penyusunan Soal High Order Thinking (HOTS)
Direktorat Pembinaan Sekolah Menegah Atas Direktorat Jenderal Pendidikan Dasar
dan Menengah Kementerian Pendidikan dan Kebudayaan. Jakarta: DIREKTORAT
PEMBINAAN SMA DIREKTORAT JENDERAL PENDIDIKAN DASAR DAN MENENGAH
DEPARTEMEN PENDIDIKAN DAN KEBUDAYAAN.

51. Thank You 