IJLTER.ORG Vol 20 No 2 February 2021

ijlterorg

This study measured the structural and organizational changes in the knowledge schema of human cognition in response to the learning achieved by 48 students enrolled in the second year of a psychology degree. Two studies were carried out based on the Chronometric Constructive Cognitive Learning Evaluation Model. This article deals only with the first one, which consisted of a conceptual definition task designed in line with the Natural Semantic Network technique. Participants defined ten target concepts with verbs, nouns, or adjectives (definers), and then weighed the grade of the semantic relationship between the definers and the target concepts. The data indicate that the initial knowledge structures had been modified towards the end of the course. The participants’ human cognition schema presented changes in terms of content, organization, and structure. This evidence supports the idea that the acquisition and transformation of the schemata learned in academic environments may be observed through cognitive science indicators.

International Journal
of
Learning, Teaching
And
Educational Research
p-ISSN:
1694-2493
e-ISSN:
1694-2116
IJLTER.ORG
Vol.20 No.2
International Journal of Learning, Teaching and Educational Research
(IJLTER)
Vol. 20, No. 2 (February 2021)
Print version: 1694-2493
Online version: 1694-2116
IJLTER
International Journal of Learning, Teaching and Educational Research (IJLTER)
Vol. 20, No. 2
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically those of translation, reprinting, re-use of illustrations,
broadcasting, reproduction by photocopying machines or similar means, and storage in data banks.
Society for Research and Knowledge Management
International Journal of Learning, Teaching and Educational Research
The International Journal of Learning, Teaching and Educational
Research is a peer-reviewed open-access journal which has been
established for the dissemination of state-of-the-art knowledge in the
fields of learning, teaching and educational research.
Aims and Objectives
The main objective of this journal is to provide a platform for educators,
teachers, trainers, academicians, scientists and researchers from over the
world to present the results of their research activities in the following
fields: innovative methodologies in learning, teaching and assessment;
multimedia in digital learning; e-learning; m-learning; e-education;
knowledge management; infrastructure support for online learning;
virtual learning environments; open education; ICT and education;
digital classrooms; blended learning; social networks and education; e-
tutoring: learning management systems; educational portals, classroom
management issues, educational case studies, etc.
Indexing and Abstracting
The International Journal of Learning, Teaching and Educational
Research is indexed in Scopus since 2018. The Journal is also indexed in
Google Scholar and CNKI. All articles published in IJLTER are assigned
a unique DOI number.
Foreword
We are very happy to publish this issue of the International Journal of
Learning, Teaching and Educational Research.
The International Journal of Learning, Teaching and Educational
Research is a peer-reviewed open-access journal committed to
publishing high-quality articles in the field of education. Submissions
may include full-length articles, case studies and innovative solutions to
problems faced by students, educators and directors of educational
organisations. To learn more about this journal, please visit the website
http://www.ijlter.org.
We are grateful to the editor-in-chief, members of the Editorial Board
and the reviewers for accepting only high quality articles in this issue.
We seize this opportunity to thank them for their great collaboration.
The Editorial Board is composed of renowned people from across the
world. Each paper is reviewed by at least two blind reviewers.
We will endeavour to ensure the reputation and quality of this journal
with this issue.
Editors of the February 2021 Issue
VOLUME 20 NUMBER 2 February 2021
Table of Contents
Chronometric Constructive Cognitive Learning Evaluation Model: Measuring the Construction of the Human
Cognition Schema of Psychology Students .........................................................................................................................1
Guadalupe Elizabeth Morales-Martinez, Janneth Trejo-Quintana, David Jose Charles-Cavazos, Yanko Norberto Mezquita-
Hoyos, Miriam Sanchez-Monroy
Portfolio as a Tool for Self- Reflection and Professional Development for Pre-Service Teachers.............................. 22
Arlinda Beka, Ganimete Kulinxha
The Efficacy of Habits of Mind in the Inculcation of Self-Directed Learning Skills in Pre-Service Teachers ........... 36
Gideon Petrus Van Tonder, Byron John Bunt, Alvine Petzer, Hermann Davin Bosch, Nicolaas Van Deventer, Amanda
Gerber, Lizelle Van Schadewijk
The Impact of Professional Development, Modern Technologies on Lecturers’ Self-Efficacy: Implication for
Sustainable Science Education in Developing Nations....................................................................................................61
David Agwu Udu, Benson Ikechukwu Igboanugo, John Nmadu, Chidebe Chijioke Uwaleke, Benjamin Chukwunonso
Okechineke, Adaora Phina Anudu, Precious Chisom Attamah, David Onyemaechi Ekeh, Mercy Ifunanya Ani
A Competence Model to Assess and Develop Designing Competence Assessment Tool .......................................... 81
Do Tra Huong, Nguyen Thi Dieu Linh
Cultural Language Learning Approach (CLLA): An Approach for Learners’ Integrated Language Use
Acculturation....................................................................................................................................................................... 104
Hermayawati .
The Impact of Using VAKT Strategy on Oral Reading and Reading Comprehension Skills of Elementary Students
with Dyslexia....................................................................................................................................................................... 121
Ayed H. Ziadat
The Influence of Architecture Students’ Learning Approaches on their Academic Performance in Two Nigeria
Universities.......................................................................................................................................................................... 137
Gabriel Sen, Albert Adeboye, Oluwoye Alagbe
Blended Learning in Rural Primary ESL Classroom: Do or Don’t............................................................................... 152
Melanie Jerry, Melor Md Yunus
Developing Interaction in ESL Classes: An Investigation of Teacher-Student Interaction of Teacher Trainees in a
Sri Lankan University......................................................................................................................................................... 174
Abdul Majeed Mohamed Navaz
Teaching with Relevance: Saudi Students’ Perceptions of a Foundation Course in Communication Skills .......... 197
Philline M. Deraney
Students’ Soft Skills, Hard Skills, and Competitiveness (SHC): A Suggested Model for Indonesian Higher
Education Curriculum........................................................................................................................................................ 218
Hadiyanto ., Noferdiman ., Syamsurizal ., Muhaimin ., Ina Krisantia
Decision-making by Heads of Academic Department using Student Evaluation of Instruction (SEI) ................... 235
Mohammed Saleh Alkathiri
Fostering Media Literacy Skills in the EFL Virtual Classroom: A Case Study in the COVID-19 Lockdown Period
............................................................................................................................................................................................... 251
Marina Bilotserkovets, Tatiana Fomenko, Oksana Gubina, Tetiana Klochkova, Oksana Lytvynko, Maryna Boichenko, Olena
Lazareva
Analysis of Future Teachers’ Perceptions on the Evaluation of Learning: A Case Study of Chile .......................... 270
José Hernández Sepúlveda, Rodrigo Panes Chavarría, Karla Rosalia Morales Mendoza
Career Readiness of Graduating Office Technology and Management Students in Polytechnics in Southeast
Nigeria.................................................................................................................................................................................. 290
Lynda Chineze Ezechukwu, Oliver Okechukwu Okanazu, Muhammadu Madu-Saba Babalulu, Ayoola Abimbola Arowolo,
Chijioke Jonathan Olelewe
Information Technology Integration Perception on Ghanaian Distance Higher Education: A Comparative
Analysis................................................................................................................................................................................ 304
Albert Arthur Qua-Enoo, Brandford Bervell, Paul Nyagorm, Valentina Arkorful, John K. E. Edumadze
ESL Teachers’ Challenges in Implementing E-learning during COVID-19 ................................................................ 330
Brenda Anak Lukas, Melor Md Yunus
1
©Authors
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
International License (CC BY-NC-ND 4.0).
International Journal of Learning, Teaching and Educational Research
Vol. 20, No. 2, pp. 1-21, February 2021
https://doi.org/10.26803/ijlter.20.2.1
Chronometric Constructive Cognitive Learning
Evaluation Model: Measuring the Construction of the
Human Cognition Schema of Psychology Students
Guadalupe Elizabeth Morales-Martinez and Janneth Trejo-Quintana
Cognitive Science Laboratory, IISUE,
National Autonomous University of Mexico, Mexico City, Mexico
https://orcid.org/0000-0002-4662-229X
https://orcid.org/0000-0002-7701-6938
David Jose Charles-Cavazos
TecMilenio University, Mexico City, Mexico
https://orcid.org/0000-0002-3445-9026
Yanko Norberto Mezquita-Hoyos
Autonomous University of Yucatán, Yucatan, Mexico
https://orcid.org/0000-0001-6305-7440
Miriam Sanchez-Monroy
Tecnologico Nacional de Mexico-Instituto Tecnologico de Merida, Yucatan, Mexico
https://orcid.org/0000-0001-5263-1216
Abstract. This study measured the structural and organizational changes
in the knowledge schema of human cognition in response to the learning
achieved by 48 students enrolled in the second year of a psychology
degree. Two studies were carried out based on the Chronometric
Constructive Cognitive Learning Evaluation Model. This article deals
only with the first one, which consisted of a conceptual definition task
designed in line with the Natural Semantic Network technique.
Participants defined ten target concepts with verbs, nouns, or adjectives
(definers), and then weighed the grade of the semantic relationship
between the definers and the target concepts. The data indicate that the
initial knowledge structures had been modified towards the end of the
course. The participants’ human cognition schema presented changes in
terms of content, organization, and structure. This evidence supports the
idea that the acquisition and transformation of the schemata learned in
academic environments may be observed through cognitive science
indicators.
Keywords: cognitive evaluation; knowledge schema; learning; NSN;
psychology students
2
http://ijlter.org/index.php/ijlter
1. Introduction
Assessing academic learning is one of the most significant challenges for
educators in the twenty-first century. This is supported by William (2011), who
asserted that assessing learning is a central activity in the instruction-learning
process. There is a great diversity of learning measurement tools, especially given
the development of new technology, which has opened up new possibilities in
this field. However, there is still no consensus on the most convenient way to
assess student learning. This problem means that although there is a diversity of
tools with which to measure academic learning, there is no agreement about the
best way to determine what and how much content a student has learned during
a course. El-Yassin (2015) remarked that there is no right or wrong way to evaluate
student learning since each instrument can inspect a specific learning aspect. In
addition, William (2011) pointed out that although the sequence of presentation,
quality, and even teaching in a class is the same for all students, they understand
what they learn in the classroom differently and may even learn different things
to what they are taught.
This variability in students’ academic learning has long been considered a barrier
to teaching rather than a source of enrichment within the classroom. In this regard,
William (2011) discussed how for many years, those involved in the educational
field assumed that the quality of instruction alone would be enough for students
to learn, and failure to learn in spite of effective instruction was attributed to the
students’ cognitive characteristics. Currently, the educational community is
beginning to raise awareness about the role played by an individual student’s
needs and cognitive characteristics in the design of teaching-learning sequences.
Regarding the above, in the 1980s, Messick (1984) stressed that the interpretation
of achievement measures should be carried out in the context of the style of
instruction and learning to reduce errors in the interpretation of academic
performance and students’ functioning within specific learning environments.
Although this proposal sounds obvious, Messick explained that measuring
learning in such an all-encompassing way is rarely feasible due to the complexity
of the information that needs to be extracted at different levels of student life.
In general, learning assessment can be very complex due to the broad spectrum
of factors involved. According to Muskin (2015), the evaluation of learning
implies using a means to determine what a person knows in conceptual or
procedural terms. In this regard, Messick (1984) pointed out that school learning
not only involves the content that a student can store in their memory, but also
how the student structures or restructures their knowledge and cognitive skills
according to their level of academic development (beginner, intermediate, or
advanced).
Messick (1984) suggested that any learning measurement should take account of
the state of academic development of each student to establish the cognitive
functioning level at which the learning assessment will be carried out. For
example, Messick proposed that with students in an initial learning phase, the
objective should be to acquire information. At this level, information-retrieval
3
http://ijlter.org/index.php/ijlter
recognition assessments could be used. In contrast, at a more advanced level,
student learning should manifest itself in the restructuring of schemata and the
flexible use of schemata to solve problems. However, Messick saw the application
of such a proposal as very forward-looking rather than being based on the reality
of developing performance tests.
Currently, most evaluation instruments are focused on performance
measurement. In this regard, Banister (2004) pointed out that in psychology, the
most commonly used instruments to measure learning are exams, practical tests,
and empirical dissertations. These kinds of tools are used as summative
assessments of student performance. The tests provide valuable information
about aspects of students’ knowledge of the information evaluated in the test.
However, they are not planned to have implications for the design of instruction
techniques (Arieli-Attali, 2013). Exams have been criticized for being indirect
measures that do not take into account context and that are more oriented towards
obtaining a product rather than understanding the learning process (Sadeghi &
Rahmati, 2017).
Summative assessments are useful in this sense as they are used for what they
were designed. However, when the main objective is to provide information on
the processing of the information inputs that students receive in the classroom,
rather than on the performance (the output from the process), then the necessary
use of alternative tools to measure the cognitive processes of assimilation and
accommodation of information as a result of learning becomes evident.
Nevertheless, scientific exploration of the use and impact of evaluation tools to
assess cognitive changes and provide useful indicators to correct or promote the
restructuring of a learned schema is still an underexplored field.
One way to approximate this learning-evaluation challenge is to include cognitive
psychology tools to measure the human mind. This scientific discipline has high
potential to evaluate skills (Embretson, 1999) and the formation of knowledge
structures, and can thus be applied to different aspects of the learning process. For
example, Marzano’s Learning Dimensions Model identifies five kinds of thinking
involved in the learning process: a) attitudes and perceptions, b) acquisition and
integration of knowledge, c) extending and refining knowledge, d) the
meaningful use of knowledge, and e) mental habits (Marzano & Pickering, 1997).
The measurement of these dimensions can be approximated with the paradigms
and research techniques involved in human cognition science. For example, the
research techniques used to explore human memory can be extrapolated to
studying the cognitive mechanisms involved in dimensions b, c, and d of the
Learning Dimensions Model.
Arieli-Attali (2013) stated that the idea of including advances in cognitive science
to develop new forms of measurement or complement psychometric means of
evaluation is not new. Initiatives have been emerging since the last century to link
advances in cognitive psychology to the measurement of abilities. For example,
the Air Force Human Resources Lab carried out the Learning Abilities
Measurement Project (LAMP) (Kyllonen & Christal, 1988), which sought to
4
http://ijlter.org/index.php/ijlter
identify indicators of student learning and achievement, taking into account
measures for processing capacity, speed of processing, knowledge, and skills. The
results of this seminal effort demonstrated that cognitive measures could
successfully predict performance in learning tasks and even do so with greater
precision than some instruments already available. Later initiatives such as the
Cognitive Design System (CDS) (Embreston, 1999) or Evidence-centered Design
(ECD) (Mislevy, Steinberg & Almond, 2003) have continued to promote the
concept of using cognitive tools within the assessment of learning.
The Chronometric Constructive Cognitive Learning Evaluation Model (C3-LEM)
by Lopez and Morales (Lopez et al., 2014; Morales-Martinez & Lopez-Ramirez,
2016; also see Morales-Martínez, 2020; Morales-Martinez et al., 2017; Morales-
Martinez, Lopez-Ramirez & Lopez-Gonzalez, 2015) is a recent initiative to
promote the use of cognitive measurement tools to evaluate academic learning.
This evaluation model is based on applying the laws and principles for how the
human mind selects, stores, and retrieves information.
From cognitive psychology, the human mind is seen as a producer of cognitive
structures called schemata. These mental structures are formed with the
knowledge that people store in their memories. Schemata possess properties
relating to their flexibility and stability. In the educational field, the students form
schemata from materials learned on a course or in a career. These schemata can
remain or be modified over time, depending on how students store, organize and
structure their learning.
Keeping the above idea in mind, Lopez (1989) proposed an academic-failure-rate
predictor system based on evaluation techniques derived from the Theory of
Human Information Processing (HIP) and the Theory of Parallel Distributed
Processing (PDP). Lopez attempted to show that the study techniques from these
areas allow the properties of learned-knowledge schemata to be observed and
measured in the same way that general knowledge schemata can be observed. He
tested this idea in his doctoral thesis, by designing and applying the Semantic
Analyzer of Schemata Organization (SASO). This system allowed him to explore
knowledge schemata in human memory (Lopez, 1996; Lopez & Theios, 1992).
Later, Lopez et al. (2014) used this model to create a new system by which to
evaluate learning. This learning-evaluation system was the origin of the Cognitive
Evaluator (known in Spanish as EVCOG), which is a computerized system that
assesses academic learning, and which gave rise to the C3-LEM developed by
Morales-Martinez & Lopez-Ramirez (2016; also see Morales-Martinez et al. 2017,
Morales-Martinez, Ángeles-Castellanos et al. 2020).
The C3-LEM (Figure 1) offers an alternative way to measure various aspects of
mental representation of the knowledge students learn in academic courses. For
example, this model allows indicators on the schematic organization of
knowledge to be obtained. Arieli-Attali (2013) pointed out that measuring the
conceptual understanding advances of students during a course can provide
useful information to support the design of teaching and learning strategies that
help students learn the knowledge and skills necessary to adapt to an
5
http://ijlter.org/index.php/ijlter
environment whose economy is based precisely on information and knowledge
management.
Figure 1. Phases and components of the C3-LEM
Note: From “Cognitive e-tools for diagnosing the state of medical knowledge in students
enrolled for a second time in an anatomy course,” by Morales-Martinez, Ángeles-
Castellanos et al., 2020, International Journal of Learning, Teaching and Educational Research,
19(9), p. 346 (https://doi.org/10.26803/ijlter.19.9.18). Copyright 2020 by the authors and
IJLTER.ORG.
Figure 1. illustrates the phases and components that make up the C3-LEM. In
general, this evaluation model promotes the combined and intertwined use of
mental representation techniques, computational simulation tools, and
chronometric cognitive measurement techniques to assess the modifications in the
organization and mental structure of knowledge, as well as the dynamics and
temporal changes in the learned schemata (Morales-Martinez, Ángeles-
Castellanos et al., 2020; Morales-Martinez, Lopez-Perez et al., 2020).
C3-LEM studies are based on the EVCOG procedure, which consists of two
phases: constructive cognitive evaluation and chronometric cognitive evaluation
(Figure 1). Together, these two approaches provide indicators of students’
cognitive mechanisms in terms of their ability to select, elaborate on, and build
knowledge from the information obtained from an academic course. This article
focuses on using the constructive cognitive evaluation of knowledge since it
illustrates the first step for evaluating learning with C3-LEM. The objective is to
contribute empirical evidence on the usefulness of cognitive techniques for
measuring organization and structural changes in students’ knowledge schemata
due to the learning process in a human cognition course.
1.1. Constructive Cognitive Evaluation of Knowledge Schemata Learned
during an Academic Course
The constructive cognitive evaluation of learning involves measuring the
knowledge schema’s properties through a mental representation technique and
computer simulations. The central idea is to observe the conceptual changes that
occur in the student’s memory due to the learning process.
6
http://ijlter.org/index.php/ijlter
Typically, the first step consists of applying the Natural Semantic Network (NSN)
technique at the beginning and the end of the academic year (see the Methodology
section), although any other technique that allows organization indicators and
conceptual structure to be extracted can be used. Figueroa, Gonzalez & Solis
(1976) proposed the NSN as a mental representation technique to explore
meaning formation. According to Figueroa-Nazuno (2007), the construction of
meaning depends entirely on the person who constructs it. The person elaborates
and interprets knowledge through a constructive and reconstructive process of
memory. So, from this conceptualization of cognitive functioning, the formation
of meaning goes beyond free association.
Mental representation studies based on the C3-LEM have provided evidence that
students construct or reconstruct their declarative knowledge schemata as a result
of the learning obtained during a course. For example, Morales-Martinez, Lopez-
Perez et al. (2020) applied the NSN technique to measure the knowledge schema
arising from a course on the Computational Theory of Mind. They observed that
students enter the course with a pre-schema. However, no conceptual
organization could be identified between the pre-schema nodes. After the course,
the students had assimilated new concepts, eliminated some information nodes,
and established an organization amongst the conceptual nodes they had learned
during the course. These results agree with Bower’s (1975) seminal idea that the
acquisition of declarative schemata embraces the incorporation of new
information nodes.
Moreover, the studies using NSN have been able to identify limitations in the
knowledge structures of students, relating to each individual’s level of academic
development in terms of the subject they are learning. Morales-Martínez,
Mezquita-Hoyos et al. (2018) noted that students who did not achieve passing
grades on the computational usability course had fractured knowledge schemata
at the end of the course. Morales-Martinez, Angeles-Castellanos et al. (2020)
reported similar data in their cognitive diagnostic study on the structure and
organization of the human anatomy knowledge schema amongst first-year
medical students. The data from this study pointed to fractured cognitive
structure in the schema and difficulties with conceptual organization.
Some reasons for schematic fragmentation include the relevance weight given to
the different topics within a course or a lack of emphasis on establishing the
relationships or connections between the topics reviewed during the academic
course (Morales-Martinez, Ángeles-Castellanos et al., 2020). Fragmented
knowledge structures are also observed in students starting a course to review a
new topic (Morales-Martínez, López-Pérez et al., 2020; Urdiales-Ibarra et al.,
2018).
Information integration strategies influence the formation or correction of
integration limitations in knowledge structures such as those mentioned above.
In this regard, Morales-Martínez, Mezquita-Hoyos et al. (2018) reported that
engineering students with a fractured schema at the end of their course managed
7
http://ijlter.org/index.php/ijlter
to integrate information from the computational usability schema after attending
a corrective course on the subject.
In general, NSN provides information on how the student’s mind organizes and
structures knowledge schemata according to the learning experiences during
academic courses. Few studies exist which have used the C3-LEM approach to
explore the knowledge domain in psychology. Specifically, the topics covered to
date using C3-LEM relate to the Piagetian Theory schema and the Computational
Theory of Mind (e.g., Morales-Martínez, López-Pérez et al., 2020). The results of
these studies suggested that students start the courses with vague but
pre-organized ideas about the knowledge that they will review throughout the
course. At the end of the course, students with passing grades had acquired new
information nodes in the cognitive structures related to their knowledge.
Additionally, they had established new relationships between concepts and
reconstructed or reorganized their schemata based on their learning experiences.
However, more investigations offering empirical evidence on the learning
properties of knowledge schemata in psychology are necessary to build a solid
theory about the behavior of schemata in this field of knowledge. The present
study contributes new information on the organization and schematic behavior of
the knowledge structures acquired in one of the most relevant fields of
psychology science, human cognition.
2. Methodology
2.1. Study Overview
This research measured the state of knowledge on the human cognition schema
amongst students enrolled in the second year of a psychology degree at the
beginning and end of a course. The state of knowledge refers to the set of cognitive
properties (organizational, structural, temporal, and dynamic) that characterizes
students’ knowledge schemata in any academic course. For example, at the
beginning of a course, students present less semantic richness than at the end of
the course. In addition, throughout the course, students judge the semantic
relevance of concepts in different ways. Moreover, the recognition pattern for
schematic words is different at the beginning, during, and at the end of the course.
Thus, this study explored the changes in the organization and structure of the
human cognition schema experienced by students as a result of the learning
acquired during a cognition course. The authors designed an NSN study that
included a conceptual definition task related to the human cognition schema.
2.2. Participants
The participants were 48 second-year psychology students enrolled in a course on
human cognition. Their ages ranged from 19 to 34 years old (M = 20.3, SD = 2.58).
Overall, 79% (38) were women and 21% (10) were men. The authors selected
participants using a convenience sampling technique. Potential participants were
included in the study only if they took part voluntarily and signed the informed
consent. Participants who did not finish the two application phases or did not
follow the instructions were excluded from the study.
8
http://ijlter.org/index.php/ijlter
2.3. Study Design
The study design was based on the EVCOG sequence proposed in the C3-LEM.
The researchers designed a mental representation study based on the modified
NSN from Lopez and Theios (1992) and Lopez (1996). The objective was to
measure the cognitive properties of the content, organization, and structure of the
human cognition schema.
2.4. Instruments and Materials
To build the NSN instrument, the researchers selected ten target concepts from
the Protocol for the Collection of Target Concepts and Central and Deferred
Definers (Morales-Martinez, 2015). This protocol guides the teacher or knowledge
domain expert in terms of identifying the most relevant conceptual targets for the
course. The resulting ten concepts were considered to be the evaluated schema
concepts. The ten conceptual targets selected by the teacher were: cognition,
cognitive psychology, perception, attention, consciousness, memory,
representation of knowledge, reasoning, problem-solving, and decision-making.
The researchers used EVCOG software to design and apply the cognitive studies
of mental representation. Additionally, this software allowed the capture and
analysis of data based on the C3-LEM (Morales-Martínez, López-Pérez et al.,
2020).
2.5. Procedure
In this study, the constructive cognitive evaluation of learning comprised the
application of a task based on the NSN technique at the beginning and end of the
course. First, the researchers invited students who were enrolled in a course on
human cognition to participate in the research. Subsequently, the students who
agreed to participate received information about the study and their rights as
participants, and gave their informed consent. After this, they performed an
exercise to familiarize themselves with the task. Finally, the NSN study was
applied.
During the NSN study, each participant observed the target concepts one by one
on a computer screen. The task was to define the targets using verbs, nouns,
adjectives, and pronouns as definers. The production criterion for definers was
that they had to be directly related to their course content on human cognition.
Phrases, articles, and prepositions were not allowed to be used for the definitional
task. The participants had 60 seconds to define each target. Subsequently, they
rated each definer using a scale from 1 to 10; 1 meant that the evaluated definer
chosen was not very related to the target concept, and 10 indicated that the definer
was significantly related to the target concept. The time to complete the entire task
varied from 15 to 20 minutes, depending on each participant.
3. Data Analysis
In this study, the authors undertook three analyses of the NSN data. The first
analysis was a traditional mental representation analysis using the EVCOG
system. This software allows several NSN values, proposed by Figueroa et al.
(1976) and described by Lopez (1996) and Lopez and Theios (1992), to be
9
http://ijlter.org/index.php/ijlter
computed. This analysis involved various elements which are described below.
The indicators for the analysis included, firstly, semantic richness (J value),
generated for each target concept through the total number of different definers.
Secondly, semantic relevance (M value) was obtained from the score consciously
given by the participants for each target definer, expressed as the sum of all the
weights assigned by the participants to each definer. The ten most relevant
defining concepts were also identified to build the meaning of the target concept
of the network. This group of definers is known as the SAM group (Semantic
analysis of M value or SAM) and is made up of the ten definers with the highest
M values for each target concept. Another indicator was semantic distance (FMG
value) between the given definer and the target concept that was defined. This is
computed using the percentage range corresponding to the M value of each of the
definers obtained for the SAM group in relation to the highest M value obtained
in the group. Finally, semantic density (G value) was calculated.
The second analysis was undertaken using the EVCOG system. This software
allowed the extraction of the association matrix. This matrix is called the SASO
connectivity matrix, which is calculated using a Bayesian formula proposed by
Lopez and Theios (1992). According to these authors, this equation is a
modification of that by Rumelhart et al. (1986). Lopez and Theios’s equation is
given below:
WIJ = -1n{[p(X = 0 & Y = 1) p(X = 1 & Y = 0)]*[p(X = 1 & Y = 1) p(X = 0 & Y = 0)]-1}
[1]
This equation calculates the co-occurrence probability amongst pairs of concepts
(X and Y) throughout the NSN. Firstly, p(X = 0 & Y = 1) refers to the joint
probability that Y appears but X does not appear in a SAM group. Similarly,
p(X = 1 & Y = 0) denotes the joint probability that X appears but Y does not appear
in a SAM group, and p(X = 1 & Y = 1) was computed in the same manner. The
calculation of p(X = 1 & Y = 1) involved the hierarchical modulation of M values
in the SAM groups.
The SASO connectivity matrix was used to feed the Gephi software to obtain a
graphical representation of the accommodation of schema concepts (see Figure 3).
Finally, the authors used STATISTIC software (version 7) to apply a
multidimensional scaling on the NSN data. To this end, the authors considered
the co-occurrence of definer concepts for each target concept.
4. Findings/Results
4.1. Lopez and Theios’s Analysis of NSN Data
The NSN data obtained before (Table 1) and after (Table 2) the course were
analyzed based on the procedure described by Lopez and Theios (1992).
10
http://ijlter.org/index.php/ijlter
Table 1. SAM groups for the human cognition schema obtained from the participants
before the course
Note: J = semantic richness, G = semantic density, F = occurrence frequency,
M = semantic weight, IRT = inter-response time
F Definer M IRT F Definer M IRT F Definer M IRT
9 Cognitive process 158 18 9 Cognitive process 159 22 2 Senses 97 26
3 Mind 115 23 3 Mind 147 18 2 Interpret 67 35
5 Memory 110 27 5 Memory 141 36 2 Stimuli 63 23
3 Learning 88 32 5 Thought 93 26 9 Cognitive process 54 33
5 Thought 79 20 3 Learning 75 24 1 Feel 51 24
4 Attention 64 32 4 Attention 73 35 4 Attention 43 22
4 Perception 44 38 4 Perception 72 37 3 Brain 36 54
5 Capacity 40 41 1 Study 69 16 1 Observe 34 22
1 Processing 36 33 1 Behavior 64 29 4 Information 33 34
3 Brain 35 31 1 Science 47 18 1 Reality 26 41
F Definer M IRT F Definer M IRT F Definer M IRT
9 Cognitive process 129 24 3 Mind 78 23 1 STM 127 26
1 Focus 91 27 9 Cognitive process 55 30 1 LTM 105 29
5 Capacity 73 31 1 Mind state 45 11 1 Store 94 14
2 Stimuli 66 19 5 Thought 44 38 3 Learning 91 25
1 Concentrate 58 23 4 Attention 42 27 9 Cognitive process 88 28
5 Memory 38 24 3 Brain 40 44 1 Memories 86 24
2 Senses 36 38 1 Vigil 39 29 1 Remember 67 14
1 Selective attention 33 20 1 Internal 33 42 1 WM 65 30
4 Perception 31 47 2 Cognition 31 41 4 Information 63 32
2 Cognition 30 72 3 Reasoning 30 23 1 Retrieve 59 33
J-value: 205 J-value: 174 J-value: 282
F Definer M IRT F Definer M IRT F Definer M IRT
1 Schemata 95 22 3 Thinking 117 16 3 Reasoning 64 18
1 Image 81 20 9 Cognitive process 72 20 9 Cognitive process 56 28
1 Symbols 43 15 1 Human 47 22 3 Thinking 52 18
1 Models 39 21 5 Thought 46 25 2 Reason 51 16
1 Mental 36 15 3 Analysis 45 33 5 Memory 46 27
4 Perception 33 32 1 Logic 43 19 5 Capacity 44 19
4 Information 31 31 1 Consciousness 32 33 3 Analysis 41 28
1 Object 28 27 5 Capacity/Ability 31 27 2 Choice 39 36
5 Memory 27 39 4 Information 29 30 2 Options 38 41
1 Concepts 27 37 2 Interpretation 27 46 5 Thought 38 32
J-value: 175 J-value: 200 J-value: 192
F Definer M IRT
2 Choice 104 17
9 Cognitive process 73 34
2 Options 58 30
1 Evaluation 53 39
2 To reason 42 18
1 Solutions 38 31
5 Capacity 34 33
3 Reasoning 33 26
3 Analysis 31 40
3 Thinking 27 50
Cognition Cognitive psychology Perception
J-value: 218 G-value 12.30 J-value: 258 G-value: J-value: 217 G-value: 7.10
Attention Consciousness Memory
G-value: 9.90 G-value: 4.80 G-value: 6.80
Representation Reasoning Problem solving
G-value: 6.80 G-value: 9.00 G-value: 2.60
Decision making
J-value: 212 G-value: 4.60
11
http://ijlter.org/index.php/ijlter
Table 2. SAM groups for the human cognition schema obtained from the participants
after the course
Note: J = semantic richness, G = semantic density, F = occurrence frequency,
M = semantic weight, IRT = inter-response time
Table 1 shows that the definers (cognitive process, mind, memory,
short-term memory (STM), thinking, long-term memory (LTM), choice, senses,
schemata) with the highest M in each SAM group before the course were mostly
general. At the end of the course, however, most of the concepts with the highest
F Definer M IRT F Definer M IRT F Definer M IRT
10 Cognitive process 259 13 10 Cognitive process 230 26 1 Sensation 182 13
3 Information 98 28 1 Science 167 20 1 Interpret 152 22
2 Mind 89 22 1 Neisser 113 23 1 Threshold 125 29
7 Memory 80 33 2 Cognition 92 30 2 Stimuli 116 27
1 Cold cognition 79 31 7 Memory 79 31 10 Cognitive process 110 32
3 Attention 68 33 1 HIP 78 29 1 Direct perception 84 26
1 Psychology 63 43 3 Information 76 31 1 Illusion 69 30
1 Hot cognition 54 29 1 Representation 63 41 1 Senses 62 24
1 Human 49 42 3 Attention 62 36 4 Perception 55 24
1 Processing 49 32 4 Perception 61 32 1 Gestalt 49 44
F Definer M IRT F Definer M IRT F Definer M IRT
1 Filter 211 22 3 Attention 134 25 1 Store 286 20
10 Cognitive process 190 20 10 Cognitive process 127 24 1 Retrieve 258 30
1 Selective attention 124 23 1 Become aware 94 17 1 LTM 232 32
1 Divided attention 95 31 7 Memory 83 35 1 STM 231 27
2 Stimuli 93 39 4 Perception 67 36 1 SM 230 26
1 Attenuation model 82 25 2 Knowledge 62 11 1 Encoding 147 29
1 Sustained attention 81 23 1 Unconscious 49 43 10 Cognitive process 112 22
1 Capacity 78 28 1 Explicit 48 42 1 WM 96 34
4 Perception 71 25 1 Reflector 41 36 1 Implicit 77 25
1 Focus 68 28 2 Cognition 41 49 2 Semantics 74 41
J-value: 360 J-value: 279 J-value: 411
F Definer M IRT F Definer M IRT F Definer M IRT
1 Schemata 261 23 1 Reasoning 239 20 1 Objective 114 30
2 Mind 198 19 1 Conclusion 216 25 10 Cognitive process 103 19
1 Concepts 142 26 1 Inductive 201 20 1 Problem 100 25
10 Cognitive process 99 25 1 Syllogism 175 26 2 Reasoning 87 30
7 Memory 98 33 1 Analogical 110 27 1 Goal 81 17
1 Images 74 20 10 Cognitive process 109 23 1 Heuristics 77 37
1 Imagine 73 14 1 Premises 86 24 7 Memory 76 48
2 Knowledge 59 24 3 Information 78 25 2 Decision 70 38
2 Semantics 43 42 7 Memory 66 34 1 Strategies 56 33
1 Absence 41 18 1 Logic 60 26 1 Initial state 55 29
J-value: 332 J-value: 344 J-value: 311
F Definer M IRT
1 Choice 222 14
1 Alternative 112 19
10 Cognitive process 105 29
1 Evaluation 94 19
1 Experience 80 25
2 Reasoning 79 37
2 Decision 78 18
1 Options 62 29
7 Memory 56 32
1 Normative theories 23 35
Cognition Cognitive psychology Perception
J-value: 373 G-value: 21.00 J-value: 378 G-value: 16.90 J-value: 336 G-value: 13.30
Attention Consciousness Memory
G-value: 14.30 G-value: 9.30 G-value: 21.20
Representation Reasoning Problem solving
G-value: 22.00 G-value: 17.90 G-value: 5.90
Decision making
J-value: 331 G-value: 5.90
12
http://ijlter.org/index.php/ijlter
M in each SAM group were specific (cognitive process, sensation, filter, attention,
store, schema, reasoning, objective, choice), as shown in Table 2. Besides, when
comparing Tables 1 and 2, it can be observed that the students at the end of the
course included new definers or information nodes, rearranged some definers, or
eliminated concepts in the definitions of some targets. For example, the following
definers for cognition: thought, capacity, perception, learning and brain were
removed, and definers such as information, cold cognition, psychology, hot cognition,
and human were included (Figure 2).
Figure 2. Conceptual changes in the target cognition
In general terms, cognitive process (M value = 159) was the definer with the greatest
semantic weight in the entire network before the course (Table 1), whilst after the
course, it was store (M value = 286) (Table 2). Additionally, cognitive process was
13
http://ijlter.org/index.php/ijlter
the definition with the highest appearance frequency at the beginning of the
course (F = 9) and also at the end of the course (F = 10). The M value average for
cognitive process at the beginning of the course was 93.77, whereas at the end of the
course, it had increased to 144.4.
4.2. Gephi Analysis of NSN Data
The researchers carried out a graphical analysis of the changes in the organization
and structure of the NSN using the Gephi system (Bastian, Heymann & Jacomy,
2009). Gephi is open-access software which explores the properties of networks.
At the beginning of the course, the participants’ knowledge schema on human
cognition was made up of four large modules of concepts (Figure 3). The first
(blue) included memory-related definers (memories, learning, remembering, storing,
retrieval, working memory (WM), short-term memory, long-term memory, information).
The second group (purple) consisted of definers related to cognitive psychology
as a science (science, study, cognition, mental state, observing, wakefulness, feeling,
internal, reality, interpretation, reasoning, attention, senses, mind, thought, cognitive
process, stimuli, processing, brain, behavior). The third grouping (orange) embraced
definers related to decision-making (solutions, reasoning, thinking, evaluation,
analysis, ability, options, choice, consciousness, human, logic). The fourth group of
definers (green) was made up of concepts relating to cognitive processes (memory,
selective attention, concentration, symbols, perception, focus, image, schemata, models,
mind).
After the course, the participants rearranged the human cognition schema into
seven conceptual modules (Figure 3). The first module embraced definers
associated with perception (orange) (senses, interpretation, illusion, sensation,
threshold, direct perception, Gestalt). The second module (light green) included
definers related to consciousness and attention (sustained attention, divided
attention, selective attention, capacity, attenuation model, unconscious, filter, focus,
realize, reflector, explicit). Module 3 (pink) grouped concepts related to problem-
solving (initial state, strategies, problem, goal, heuristics, objective). Conceptual
group 4 (dark green) encompassed definers on decision-making (alternative,
options, choice, evaluation, experience). Module 5 (purple) concentrated concepts
related to three objectives: cognitive psychology, cognition, and mental
representation (schemata, images, absence, concepts, cold cognition, imagine, mind,
processing, hot cognition, human, reasoning, cognition, stimuli, decision, semantics,
memory, psychology, memory, Neisser, cognitive process, mental representation, HIP,
science, attention, knowledge). Module 6 (brown) included definers on reasoning
(deductive reasoning, premises, conclusion, logic, inductive, analogical, information,
syllogism). The last module (blue) involved definers related to memory (sensory
memory, short-term memory, long-term memory, retrieve, store, encoding, working
memory, implicit).
Additionally, the Gephi analysis pointed out changes in the conceptual
organization. The conceptual connections of definers had changed at the end of
the course. To illustrate these changes, observe in Figure 3 that at the beginning
of the course, cognitive process was a central definer concept in the primary schema
that participants brought about human cognition, although it did not have a
14
http://ijlter.org/index.php/ijlter
connection with all the schema modules. At the end of the course, the concept of
cognitive process retained its quality as a central conceptual node yet now also fully
connected with all the targets and all the conceptual modules.
Figure 3. Gephi analysis of the NSN data obtained before and after the course
15
http://ijlter.org/index.php/ijlter
4.3. Multidimensional Scaling of NSN Data
The researchers applied multidimensional scaling to the NSN data to examine the
general structure of the human cognition schema. The analysis showed changes
in the arrangement of target conceptual nodes due to the learning achieved during
the course (Figure 4).
Figure 4. Multidimensional scaling analysis of the target concepts
16
http://ijlter.org/index.php/ijlter
The multidimensional scaling graph shows that the participants started the course
without a specific structure in mind for the objective concepts, whilst at the end
of the course, they had rearranged the objective concepts based on two
dimensions. The first related to categorizing cognitive processes in terms of basic
and higher order cognition (horizontal axis). Although the definition of the second
dimension is not clear, in general, this dimension seems to be related to the use of
knowledge structures (vertical axis). Note that the target concept for reasoning
does not appear alongside targets such as problem-solving or decision-making, even
though all of these processes involve making use of knowledge structures from
memory.
5. Discussion
This study has explored changes in the knowledge schema due to the learning
process during a course on human cognition taken by second-year psychology
students. First, the authors determined whether a human cognition schema
existed before the course. The NSN and Gephi analyses indicated that the
participants entered the course with a previous-knowledge schema or a
knowledge pre-schema (see Table 1 and Figure 3). The existence of a knowledge
pre-schema has been observed in other studies (e.g., Morales-Martinez, Lopez-
Perez et al., 2020); however, the organization and structure are rudimentary. This
finding suggests that students generally have a vague schema about the
knowledge they will acquire in their courses, and it is based on this schema that
they reorganize and reconfigure the information they will learn in class.
As psychology teachers, the authors have observed that the use of general
schemata and previous learning to begin a new knowledge schema is a common
phenomenon observed in the classroom. Students generally comment that they
have come across certain information about the topic. It was therefore not
unexpected that the participants in this study commented that they were slightly
familiar with the topics. They had reviewed readings on cognitive processes in
other courses, although this had not been from the perspective of the field of
cognitive psychology.
The authors hypothesize that students use their previous learning experiences to
form a general schema or make inferences about information related to the course
in which they are enrolled. In this way, they have a conceptual basis from which
to form a more sophisticated outline of the information they cover during the
course. From a cognitive perspective, students can use or create a rudimentary
cognitive structure that allows them to guide the reorganization and restructuring
of their knowledge based on the new information inputs that they acquire through
the course. If the cognitive structure is sufficiently broad and general, it will be
flexible enough to undergo modifications due to the new learning experiences.
Interestingly, although the initial schema with which the participants in this study
entered the course was very general, their schema was not fractured as has been
observed in other courses where students start on a topic for the first time (e.g.,
Urdiales-Ibarra et al., 2018). This result may be because the participants in this
study had reviewed cognition materials the previous year when taking different
17
http://ijlter.org/index.php/ijlter
courses, meaning that they had had previous information about the topic. At the
start of their degree, the participating students were enrolled in a course where
they reviewed some of the concepts included in the course on human cognition
and obtained a passing grade on this initial course. Thus, they had general and
pre-organized ideas about the meaning of some important target concepts in NSN
studied in this research. Other studies have indicated that students who do not
obtain a passing grade for a course have a fragmented schema at the end of the
course compared to those who end the course with a passing grade (Morales-
Martinez, Angeles-Castellanos et al., 2020; Morales-Martinez, Mezquita-Hoyos et
al., 2018).
In this study, at the end of the course, the authors explored the changes that had
taken place in the participants’ pre-knowledge schema of human cognition due to
the learning acquired through the course. The analysis of the organization of the
schematic knowledge indicated that the participants had established new
relationships between the concepts. This result is consistent with Bower’s (1975)
idea that the acquisition of declarative schemata necessarily involves
incorporating new information nodes and new connections between these nodes.
The reader can compare the definers included in Tables 1 and 2 and observe that
at the beginning of the course, for some target concepts, some of the definers were
global concepts on the topic of human cognition. Meanwhile, at the end of the
course, the definers were more specific and theoretically closer to the target
evaluated. For example, for the initial conceptual definition of human cognition
(Figure 2), half of the concepts were categorical (memory, thought, attention,
perception, learning), and the other half were schematic (cognitive process, capacity,
mind, brain, processing). At the end of the course, however, the participants
included a greater number of schematic-type definers (cognitive process,
information, mind, cold cognition, psychology, hot cognition, human, processing).
The change in predominance from categorical to schematic relationships in
knowledge structures suggests that the participants had developed more
sophisticated schemata. That is, instead of using as many exemplification
schemata, their perception had changed and they were using more probabilistic
schemata. It is possible that, when students start learning a knowledge domain,
learning by exemplification dominates most of their knowledge acquisition
process. As participants in this study acquired new knowledge and refined it, they
began to use or establish other semantic relationships amongst the concepts. It
would be useful to carry out further research to explore this phenomenon since
there has been no discussion of this issue in previous research with C3-LEM to
date (e.g., Morales-Martinez, Angeles-Castellanos et al., 2020; Morales-Martinez
et al., 2020; Morales-Martínez, Mezquita-Hoyos et al., 2018; Urdiales-Ibarra et al.,
2018).
Another modification in the knowledge organization, which is of note, was the
change in the degree of generality with regard to the human cognition schema. At
the beginning of the course, the participants formed some groups that included
general definers and even incorporated information from other knowledge
18
http://ijlter.org/index.php/ijlter
schemata. For instance, module 2 of the Gephi analysis shows that before the
course, participants included definitions of various target concepts (cognitive
psychology, cognition, mental representation, perception) in the same group of
concepts and included definitions of other knowledge schemata learned for other
topics. For example, participants recovered conceptual nodes from the
behaviorism field as stimuli instead of inputs or behavior instead of cognitive patterns
(Figure 3).
The previous results indicate that at the end of the course, the participants were
able to extend and refine their knowledge about human cognition, thus placing
them at level three of Marzano’s Dimensions of Learning Model (Marzano &
Pickering, 1997). On the other hand, according to Messick (1984), the participants
in the present study would be in an intermediate stage of academic development
in terms of the development of the knowledge schema on human cognition
because indicators observed included not just the retrieval of information but a
restructuring of their schema. In congruence with this idea, the analysis of the
structural changes in the knowledge schema indicated a reconfiguration of the
schema structure by the end of the course. In this regard, Figure 3 shows how the
initial schema’s definers were arranged into four large modules, whilst the
definers for the final schema were restructured into seven conceptual modules.
Changes in the configuration of the schematic structure have been observed in
other studies that have used the C3-LEM (Morales-Martinez, Lopez-Perez et al.,
2020; Morales-Martínez, Mezquita-Hoyos et al., 2018; Urdiales-Ibarra et al., 2018).
From the point of view of cognitive psychology, changes in schematic
configuration patterns are an indication of learning. In this study, the changes to
the schema’s configurational arrangement suggest that participants had rebuilt
their structures based on the new meanings that they had acquired during the
course.
The multidimensional analysis (Figure 4) on the target concepts showed that at
the beginning of the course, the participants did not have a clear idea of how the
course’s target concepts could form a wholly organized knowledge schema. At
the end of the course, the participants organized the ten target concepts into two
dimensions, the first one relating to the cognitive nature of the processes (basic vs.
higher order cognition) and the second associated with the use of knowledge
structures. Although some concepts such as reasoning were not correctly located
in this second dimension, in general terms, this result suggests that the
participants had understood the structure of knowledge underlying the course’s
thematic organization, using the information implicit in the same target concepts.
Since this is a seminal intent of introducing a new way to analyze the results from
C3-LEM, more evidence about this phenomenon is needed to explore and explain
this kind of implicit cognitive change in the knowledge schema as a learning
product.
In summary, the study results indicated that there were changes in the
organization and structure of the human cognition knowledge schema of the
participants. They had reconfigured their old four-module schema on human
19
http://ijlter.org/index.php/ijlter
cognition into a new one which included seven modules. The participants
included new information nodes, eliminated conceptual nodes that belonged to
other disciplines, and established new relationships between the old and new
concepts.
6. Conclusions
In conclusion, the results of the present investigation have implications in three
areas. At a theoretical level, the study generated empirical evidence that supports
the idea that students enter courses with prior knowledge of the subject they are
going to study. For example, the study data indicated that the participants
possessed a macro-schema of human cognition at the beginning of the course. This
finding is relevant because it suggests that cognitive techniques such as those
contemplated in the C3-LEM can help diagnose preconceived ideas. It opens up
the possibility of correcting inaccurate information held by students when starting
a course. The measurement of this type of pre-schema would empower the teacher
to decide whether it is necessary to demystify some information or whether
modifications are required in the application of the established work program to
provide continuity or correct the knowledge structures held by students when
starting the course.
In addition, the results demonstrated that the learning process involves the
assimilation of new information and the elimination of specific conceptual nodes,
as well as the restructuring of schematic information. Furthermore, evidence from
the NSN study indicated that this type of technique can provide information on
students’ academic development level in a course. This finding has important
implications at the applied level. For example, how a student configures their
knowledge can also be taken as an indicator of mastery of the course knowledge.
Consequently, the C3-LEM could be a valuable tool in the formative assessment
of students. However, since the sample in this study was small and only
addressed one domain of knowledge, new explorations must be carried out in
other fields, such as the area of exact sciences (e.g., mathematics, chemistry,
physics), to calibrate the scope and implications of this evaluation model in the
design of new forms of educational evaluation and intervention.
Finally, at a methodological level, the study’s data supported the idea that mental
representation studies from the C3-LEM perspective may help assess cognitive
changes in the organization and structure of knowledge schemata.
7. References
Arieli-Attali, M. (2013, October 20−25). Formative assessment with cognition in mind: The
cognitively based assessment of, for and as learning [Paper presentation]. 39th Annual
Conference: Educational Assessment 2.0: Technology in Educational Assessment.
Tel Aviv, Israel. https://www.iaea.info/conference-proceedings/
Banister, P. (2004). Assessment as a tool for fostering key skills. Psychology Learning &
Teaching, 3(2), 109–113. https://doi.org/10.2304/plat.2003.3.2.109
Bastian, M., Heymann, S., & Jacomy, M. (2009, May 17−20). Gephi: An open source software
for exploring and manipulating networks [Paper presentation]. Association for the
Advancement of Artificial Intelligence, Third International AAAI Conference on
20
http://ijlter.org/index.php/ijlter
Weblogs and Social Media. San Jose California, United States.
https://gephi.org/publications/gephi-bastian-feb09.pdf
Bower, G. H. (1975). Cognitive psychology: An introduction. In W. K. Estes (Ed.), Handbook
of learning and cognitive processes: Vol. 1. Introduction to concepts and issues (pp. 25–
80). Lawrence Erlbaum Associates.
El-Yassin., H. D. (2015). Integrated assessment in medical education. Journal of
Contemporary Medical Sciences, 1(4), 36–38.
http://www.jocms.org/index.php/jcms/article/view/51
Embretson, S. E. (1999). Cognitive psychology applied to testing. In F. T. Durso (Ed.),
Handbook of applied cognition (pp. 629−660). John Wiley & Sons.
Figueroa-Nazuno, J. G. (2007, October 24−25). El significado de las Redes Semánticas
Naturales: Y la tradición oral, 20 años después [The meaning of natural semantic
networks: And oral tradition, 20 years later] [Paper presentation]. Primer
Simposium Internacional: Cognición y Representación del Conocimiento.
Monterrey, Nuevo Leon, Mexico.
Figueroa, J. G., Gonzalez, E. G., & Solis, V. M. (1976). An approach to the problem of
meaning: Semantic networks. Journal of Psycholinguistic Research, 5(2), 107−115.
https://doi.org/10.1007/BF01067252
Kyllonen, P. C., & Christal, R. E. (1988). Cognitive modeling of learning abilities: A status
report of LAMP (Learning Abilities Measurement Program). In R. Dillon & J. W.
Pellegrino (Eds.), Testing: Theoretical and applied issues. Freeman.
Lopez, R. E. O. (1989, December 1−6). Sistema predictor de indice reprobatorio (SPIR) [Failure
rate predictor system (SPIR)] [Paper presentation]. IX Coloquio de Investigación.
ENEP Iztacala, Mexico city, Mexico.
Lopez, E. O. (1996). Schematically related word recognition (Publication No. 9613356)
(Doctoral dissertation). University of Wisconsin-Madison, Madison, Wisconsin.
ProQuest Dissertations & Theses Global.
Lopez, E. O, & Theios, J. (1992). Semantic analyzer of schemata organization (SASO).
Behavior Research Methods, Instruments, & Computers, 24(2), 277−285.
https://link.springer.com/content/pdf/10.3758/BF03203508.pdf
Lopez, E. O., Morales, G. E., Hedlefs, I., & Gonzalez, C. J. (2014). New empirical directions
to evaluate online learning. International Journal of Advances in Psychology, 3(2),
40−47. https://doi.org/10.14355&ijap.2014.0302.03
Marzano, R. J., & Pickering, D. J. (1997). Dimensions of learning: Teacher’s manual (2nd ed.).
ASCD.
Messick, S. (1984). The psychology of educational measurement. Journal of Educational
Measurement, 21(3), 215–237. https://doi.org/10.1111/j.1745-3984.1984.tb01030.x
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational
assessments. Measurement: Interdisciplinary research and perspectives, 1(1), 3−62.
Morales-Martinez, G. E. (2015). Protocolo para la recolección de conceptos objetivo y definidores
centrales y diferidos (PRECODECD): Un sistema de codificación de conceptos extraídos
de las redes semánticas naturales [Protocol for the collection of objective concepts
and central and deferred definers (PRECODECD): A coding system for concepts
extracted from natural semantic networks] (Unpublished manuscript). Institute of
Research on the University and Education, National Autonomous University of
Mexico, Mexico.
Morales-Martínez G. E. (2020). Sistema de evaluación cognitiva constructiva cronométrica del
aprendizaje en línea y presencial [Online and face-to-face learning’s constructive-
chronometric cognitive assessment system] (Manuscript submitted for
publication). Institute of Research on the University and Education, National
Autonomous University of Mexico, Mexico.
21
http://ijlter.org/index.php/ijlter
Morales-Martinez, G., & Lopez-Ramirez, E. (2016). Cognitive responsive e-assessment of
constructive e-learning. Journal of e-Learning and Knowledge Society (Je-LKS), 12(4),
39−49. http://www.je-lks.org/ojs/index.php/Je-LKS_EN/article/view/1187
Morales-Martinez, G. E., Lopez-Ramirez, E. O., Castro-Campos, C., Villarreal-Treviño, M.
G., & Gonzales-Trujillo, C. J. (2017). Cognitive analysis of meaning and acquired
mental representations as an alternative measurement method technique to
innovate e-assessment. European Journal of Educational Research, 6(4), 455−464.
https://www.eu-jer.com/EU-JER_6_4_455_Morales-Martinez_etal.pdf
Morales-Martínez, G. E., Lopez-Perez, R. M., Garcia-Collantes, A., & López-Ramírez, E. O.
(2020). Evaluación constructiva cronométrica para evaluar el aprendizaje en línea y
presencial [Chronometric constructive assessment to assess online and face-to-face
learning]. Tecnología, Ciencia y Educación, 15(1), 105−124.
https://www.tecnologia-ciencia
educacion.com/index.php/TCE/article/view/371
Morales-Martinez, G. E., Lopez-Ramirez, E. O., & Lopez-Gonzalez, A. E. (2015). New
approaches to e-cognitive assessment of e-learning. International Journal for e-
Learning Security (IjeLS), 5(2), 449−453.
https://doi.org/10.20533/ijels.2046.4568.2015.0057
Morales-Martinez, G. E., Ángeles-Castellanos, A. M., Ibarra-Ramírez, V. H., & Mancera-
Rangel, M. I. (2020). Cognitive e-tools for diagnosing the state of medical knowledge in
students enrolled for a second time in an anatomy course. International Journal of
Learning, Teaching and Educational Research, 19(9), 341−362.
https://doi.org/10.26803/ijlter.19.9.18
Morales-Martinez, G. E., Mezquita-Hoyos, Y. N., Gonzalez-Trujillo, C. J., Lopez-Ramirez,
E. O., & Garcia-Duran, P. J. (2018). Formative e-assessment of schema acquisition in
the human lexicon as a tool in adaptive online instruction. In R. Lopez-Ruiz (Ed.),
From natural to artificial intelligence: Algorithms and application (pp. 69−88).
IntechOpen. http://doi.org/10.5772/intechopen.81623
Muskin, J. A. (2015). Student learning assessment and the curriculum: Issues and
implications for policy, design and implementation (Current and critical issues in
the curriculum and learning). UNESCO International Bureau of Education.
http://www.ibe.unesco.org/sites/default/files/resources/ipr1-muskin-
assessmentcurriculum_eng.pdf
Rumelhart, D., Smolensky, P., McClelland, J., & Hinton, G. (1986). Schemata and
sequential thought processes in PDP models. In J. McClelland, D. Rumelhart, &
the PDP Research Group (Eds.), Parallel distributed processing: Explorations in the
microstructure of cognition: Vol. 2. Psychological and biological models (pp. 7−57). MIT
Press.
Sadeghi, K., & Rahmati. T. (2017). Integrating assessment as, for, and of learning in a large-
scale exam preparation course. Assessing Writing, 34, 50−61.
https://doi.org/10.1016/j.asw.2017.09.003
Urdiales-Ibarra, M. E., Lopez-Ramirez, E. O., Castro-Campos, C., Villarreal-Treviño, M.
G., & Carrillo-Colon, J. E. (2018). Biology schemata knowledge organization and
meaning formation due to learning: A constructive-chronometric approach to
concept mapping usability. Creative Education, 9(16), 2693−2706.
https://doi.org/10.4236/ce.2018.916203
Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1),
3–14. https://doi.org/doi:10.1016/j.stueduc.2011.03.001
22
©Authors
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
International License (CC BY-NC-ND 4.0).
International Journal of Learning, Teaching and Educational Research
Vol. 20, No. 2, pp. 22-35, February 2021
https://doi.org/10.26803/ijlter.20.2.2
Portfolio as a Tool for Self- Reflection and
Professional Development for Pre-Service Teachers
Arlinda Beka
Faculty of Education, University of Prishtina ‘Hasan Prishtina’, Kosovo
https://orcid.org/0000-0002-9871-783X
Ganimete Kulinxha
Faculty of Education, University of Prishtina ‘Hasan Prishtina’, Kosovo
https://orcid.org/0000-0003-4461-8841
Abstract. Self-reflections, especially those done through the usage of
portfolios, are an essential component of teaching. Portfolio reflections
allow pre-service teachers to set professional goals. This research was
conducted using qualitative methodology, and all semi-structured
interviews were organised with students and graduates from the Early
Childhood Department-Faculty of Education, University of Prishtina.
The research results show that a portfolio provides future teachers with
the opportunity to organise their work better while enabling cooperation
between pre-service teachers, which ergo, directly affects their
professional competencies. Owing to its benefits, portfolio should be
integrated by the Faculty of Education of the University of Prishtina as
an integral part of its work with students.
Keywords: portfolio; pre-service preschool teachers; teacher professional
development
1. Introduction
Self-reflection is one of the most important competencies that a teacher should
possess. Reflection directly enables the identification of things that not only are a
strong part of professional work but that also need to be advanced and to allow a
higher level of professionalism. In terms of taking individual responsibility for
thought and action processes, self-reflection is a required condition for assessment
and regulation (Dilger, 2007). After completing a given task, one tries to reflect on
the process and the work accomplished; however, sometimes one’s attention is
drawn to the sections that have been the most successful or most challenging,
leaving out many other aspects that may be equally important for enhancing the
quality of one’s work and for continuing professional development.
Portfolios are a proper way of recording our accomplished work and the
challenges and successes encountered during the process. The record of such data
23
http://ijlter.org/index.php/ijlter
allows one to go back in time to see what one went through, how one did one’s
activities, how one conceived things or what challenges were faced during a given
period. The portfolio technique is viewed as particularly huge with regard to self-
intelligent learning. The technique forms a significant part in the advancement of
polished skill through reflection and self-reflection (Riebenbauer, Dreisiebner &
Stock, 2017).
Through the use of portfolios, pre-service teachers can succeed in developing a
professional working plan, reflect on their work, develop additional skills in the
use of technology for professional development and have a clear review of their
work (Beka & Gllareva, 2016). Portfolios for self-reflection in pre-service teacher
training provide vital input for trainers since they represent both the strengths
and weaknesses of students (Cimermanová, 2019).
Portfolios play a significant role in not only the process of preparing teachers for
employment but also in their licensing as teachers and coaches and their state-
level performance appraisals. Being reflective is a key aspect of any educational
experience. It involves the continuous educational growth of both professional
educators and students (Webster & Whelen, 2019).
However, portfolio is often regarded as a collection of facts and documents that
testify teachers’ professional development and achievement. In some cases, it is
considered as an important part where the teacher must present the work planned
for certain periods, including the reflection on commitments which he/she has
accomplished. Therefore, the teachers make the portfolio in terms of the purpose
that it will serve them. Similarly, during pre-service studies, future teachers are
sometimes required to have more reflections on their portfolio. In some cases,
their portfolio is also required to include facts, arguments and goals that students
aim to achieve during their studies (Stock & Winkelbauer, 2012).
2. Development of portfolios and the arguments
The use of portfolios in pre-service teachers’ professional development and
professional assessment has become widespread around the world. As a result,
several contemporary studies and research projects are being conducted by
various scholars and educational scientists regarding the importance of portfolios,
their meaning and their relevance to teachers’ professional development. It is
often thought that portfolio has begun to be affirmed in the last two decades as a
contemporary approach to professional development. Portfolio, as a concept in
education, has been developed in the last decades of XX century in the United
States of America and the German-speaking countries (Slepcevic-Zach & Stock,
2018).
For a clear understanding of the concept of portfolio usage among teachers
whether in pre-service or in-service, we need to determine the purpose of the
portfolio and the way in which it acts as a professional aid to the teacher.
According to various authors, there is a discrepancy between the portfolio and the
way it should be filled in, i.e., whether the portfolio will focus more on reflection
or on the documentation of professional work.
24
http://ijlter.org/index.php/ijlter
In his paper, ‘Do Portfolios Have a Future?’, Driessen (2017) divides portfolio into
two categories, namely reflective portfolios and comprehensive portfolios.
Reflective portfolio is primarily intended to help students improve their reflective
abilities. It requires students to write a reflection on an aspect of their learning or
career that is significant to them. Reflections can be found in comprehensive
portfolios, but their material is much more varied than that of reflective portfolios
(Driessen, 2017). Baumgartner (2009) classifies portfolio into the three following
types depending on their purpose: reflection portfolio, development portfolio and
presentation portfolio.The e-portfolio is a flexible tool that can be used for several
purposes, including learning, evaluation/assessment and employability/career
growth (Ciesielkiewicz, 2019). Trávníčková and Puhrová (2020) state that portfolios
can be encountered in the following terms: ‘professional teaching portfolio’,
‘professional development portfolio’, ‘professional practice portfolio’ and even
the relatively new term of ‘eportfolio’.
All the above types of classifications from different authors can be considered as
being similar due to their common elements. The reflective portfolio, which is
considered a crucial part of teachers’ professional development, is common in the
classifications. The aim of comprehensive portfolios is to assist students in their
learning and to evaluate their progress. Reflections can be found in
comprehensive portfolios, but their material is much more varied than that of
reflective portfolios (Driessen, 2017).
Since education methods, such as reporting, contemplation and representation of
one’s learning activity, were major issues in the 1980s, the portfolio idea might
have been a pillar in the German debate about educational reform (Häcker, 2009).
Like in Germany, in almost all European countries and beyond, the portfolio is
beginning to be considered a significant part of pre-service and pre-service teacher
work. Nowadays, portfolios are seen as
“rich, contextual, highly personalised documentaries of one’s learning
journey.” (Jones & Shelton, 2011, p.21).
There are numerous reasons why a portfolio is considered an important tool for
teacher development and professional development. Portfolios are considered
beneficial for pre-service and in-service teachers
“as they promote autonomous learning and autonomous learners are often
motivated and reflective learners.” (Fernández, 2014).
In almost all professions, especially the teaching profession, reflection is a crucial
part. According to Lynch (2000), reflection is considered to be an important part
of a person’s skill set. Through reflection, each teacher undergoes a long and
complicated process to reach his/her desired stage of professional development.
Within the scope of the portfolio-work, not only the result (item) is critical but
mostly the method that led to this improvement (Kerr, 2007). Therefore, the
documentation of professional experiences through the use of a portfolio is an
asset for every pre-service teacher. Usually, the portfolio’s documentation is
structured and deliberately designed as
“they contain purposefully organised documentation that demonstrates
specific knowledge, skills, dispositions, and accomplishments achieved
25
http://ijlter.org/index.php/ijlter
over time. Portfolios represent connections made between actions and
beliefs, thinking and doing, and evidence and criteria. They are a medium
for reflection through which the builder constructs meaning, makes the
learning process transparent and learning visible, crystallises insights,
and anticipates future direction.” (Jones & Shelton, 2011, p.22)
Despite the different definitions of portfolio which depend on their purpose,
objective and nature, certain theories consider portfolio as a personal ownership,
personal record structured collection of material and so on. According to Paris
and Ayres (1994),
“The overarching purpose of portfolios is to create a sense of personal
ownership over one’s accomplishments because ownerships engender
feelings of pride responsibility and dedication (p.10).”
McKimm (2014) defined teaching portfolios in her book ‘A Handbook for
Teaching & Learning in Higher Education’ as
“A teaching portfolio (sometimes referred to in this chapter as a
‘portfolio’) is a personal record of achievement and professional
development as a university teacher. It is a carefully selected and
structured collection of material that may demonstrate a level of
attainment, a range of skills and activity, and/or progression.(p.473)”
Kayler (2004) states that:
“Teacher education institutions need to make the often-invisible
professional growth of classroom teachers more visible to other teachers
and programs as a way to foster and expand the teacher knowledge base
(p.267).”
According to Fernández (2014), portfolios play a significant role in stimulating
and inspiring teachers as well as students. Learning becomes more effective and
motivating with a portfolio since the students take ownership of their learning
process and develop strategies to plan, monitor and evaluate their learning.
Teachers benefit from portfolios in their professional development since
portfolios promote autonomous learning, and autonomous learners are often
motivated (Fernández, 2014). Portfolios can also be considered as a source of data,
whether qualitative or statistical, and are an important tool for evaluation.
In terms of assessment, Belgrad, Burke and Fogarty (2008) consider that portfolios
(and e-portfolios) can become excellent tools for both, formative and summative
assessments. Martin-Kniep (1999) considers portfolios as an instrument that
recounts the account of the educator whose objective is to make guidance and
evaluation rehearses something very similar and focused on critical learning
results. The portfolio uncovers the educator’s extraordinary capacity to apply
educational plan and appraisal-related ideas and abilities to the advancement of
exercises and evaluations that target obviously characterised results. The
instructor has provided all drafts of appraisals, enabling the user to notice the
changes and upgrades made to apply plan standards and better adjust
educational program, guidance and evaluation. The teachers can utilise their
portfolios to report their expert history, the inquiries and worries that drive their
26
http://ijlter.org/index.php/ijlter
work, their endeavours to plan or carry out better or diverse learning encounters
for understudies, their investigation of basic issues with at least one understudy
or the settings that encompass them, their expert work with partners, local area
individuals or bosses and their examination of and reflection on proficient assets
or critical writings. Notwithstanding the specialised parts of portfolios, it should
be remembered that the
“essential crowd for an educator portfolio is the instructor who readies the
portfolio. Simultaneously, the portfolio can help instructors screen the
change from the exercise as-wanted to the exercises educated, to the
exercise as-learned. At last, portfolios can help screen educators’
endeavours to fuse new information and abilities into their expert
collections. Albeit quite a bit of this work is a private undertaking,
educators, as different experts, need to take part in a discussion with their
partners to really comprehend and approve their work. Portfolios can fill
in as a springboard for such conversations.” (Martin-Kniep, 1999, p.15)
This is very well explained in ‘The Portfolio Connection: Student Work Linked to
Standards’, in which Belgrad et al. (2008) states that:
“Portfolios as the outcome of a continuous, student-centred process help
students learn in a variety of ways. One remarkable by-product of the
portfolio process in the classroom is increased student awareness of and
responsibility for learning objective (p.15).”
The portfolio presents pre-service teacher’s information that is directly related to
the personality, mindset, worldviews, mission and vision approaches that pre-
service and in-service teachers create for themselves, whether on a personal or
professional level. Portfolio creation represents the nature of the individual’s
personality, his/her level of knowledge and the goals that he/she wishes to
achieve in his/her professional field.
This approach will help teachers in their awareness of professional development.
They will manage to present the goals for their work and the path that they will
follow. Moreover, portfolios will act as an open statement that they will follow to
be more successful. In this way, they manage to develop critical thinking about
themselves and their profession and begin to be more creative regarding their
plan to achieve their professional goals. They also become aware of a sustainable
professional investment that will serve their work in the long term. The portfolio
also presents the skills that pre-service teachers have built up during their studies
and becomes a bridge between them and the work that they will aspire to have,
which enables teachers to have personal and professional competence.
Through reflective learning, pre-service teachers can learn from their previous
experiences either in cognitive or practical terms, enabling them to make decisions
for change and professional development. Pre-service teachers can also share
information with their peers and obtain feedback from their colleagues, thus
having an excellent opportunity to be open to others, share ideas and collaborate
in the professional field.
27
http://ijlter.org/index.php/ijlter
Creating a portfolio during the study period enables pre-service teachers to
establish a culture of presenting ideas, approaches and understandings in the
personal and professional field. By establishing strategies for presenting their
work in a broader professional environment, they continue to present their work
in the same way when they look for a job or even after they are hired. For teachers
in the portfolio, a service is a great tool that can serve them for the evaluation
performed either by their institution or even at a wider level.
Therefore, in terms of teacher professional development, regardless of in-service
or pre-service teacher training, portfolio is a considerable tool for documentation,
assessment and evaluation of instructors’ work. They can be pre-designed and
pre-structured to collect and document information as required for a specific task
or profession, i.e., pre-service or in-service teaching. Portfolios enable teachers to
document the evolution of different areas of their work, such as curriculum and
assessment development, professional development, inquiry/research and
teaching and learning in general.
When used by students, portfolios are developed to display materials that
represent the abilities and accomplishments during a specified period. According
to Henderson, Davis & Day (2015), students’ presentation of materials over an
extended time helps to chronicle their improvement and competence that cannot
be measured by paper and pencil tests. One of the primary purposes of a portfolio
is to provide a more reliable and comprehensive picture of student growth and
achievement.
By using pre-service teachers’ portfolios, instructors will have better
opportunities for their personal and professional development and will create a
professional culture for their work with children. In Kosovo, as in several other
nations, one of the vital competencies of instructors is to reflect on the work they
do, to archive their completed exercises and to show the goals they wish to attain
in a given period. Typically, the employment of a portfolio plays a critical role in
teacher instruction in Kosovo or any other nations because it makes a difference
in creating the key competencies for educator teachers and pre-service instructors
(Beka, 2015).
According to the implemented laws and educational policies, all the pre-service
and in-service teacher trainings in Kosovo are organised by public universities,
particularly by their faculties of education (Ministry of Education, Science and
Technology [MEST], 2011). The Faculty of Education of the University of
Prishtina, being the oldest one, carries out the most significant role in teacher
training. It provides pre-service and in-service teacher programs from
undergraduate to doctoral level (University of Prishtina, 2004).
As part of the Strategic Development Plan of the Faculty of Education,
improvement of quality teaching programs and teacher professional development
are considered crucial factors for enhancing our education system (Faculty of
Education, 2013). Professional portfolios of pre-service teacher are becoming now
an integrated part of the teaching curriculum in all of its study programs. The
Faculty of Education has had in the past few years several in-service training
28
http://ijlter.org/index.php/ijlter
programs for teachers’ professional development besides the regular study
programs. Those programs have been supported by the MEST and different
donors-education partners aiming to advance teachers’ professional
development. Examples of these programs are: Advancement of Qualifications of
Teachers Project, Capacity Building for Teacher Mentors, Re-Qualification of
Pedagogical Student Graduates, and so on. So far, in- service training programs
have not been part of the systematic policy of Teacher Professional Development
(TPD). Rather, they were a need for the re-qualification of teachers in their
teaching degree or in the new teaching strategies (MEST, 2015). With the new
strategic documents in place, in-service training is expected to play a vital role in
TPD, leading to teachers’ career advancement (MEST, 2016). A professional
portfolio is a required tool from the pre-service studies as it will present ideas,
professional goals and the ways of achieving these goals in a period of time. It will
help teachers to self-reflect on what they have learned and how to improve their
professional practice. Reflecting on their accomplished work is significant for the
role and the well-being of the activities that they should perform on campus and
in internships. However, not all students find the portfolio relevant to their
professional development. Junior students treat it more as a task that they should
complete during their studies, whereas senior students understand the
importance of the portfolio and view it as a tool that will help them to enhance
the quality of their work and to promote continuous professional development.
At the same time, senior students consider the portfolio as a way to achieve the
goals that they set for professional development.
3. Research methodology
This research paper presents the current state of usage of portfolio as a tool for
teachers’ professional development in pre-service teacher training at the Faculty
of Education of the University of Prishtina. The research was conducted using
qualitative methodology. Interviews were organised with students and graduates
of the Early Childhood Department-Faculty of Education, University of Prishtina.
Students were randomly selected; every fifth student was selected for each year,
while alumni were purposely selected due to their easier access and the possibility
of their inclusion in this research. Five students were selected for each year of
studies, starting with the second year of studies, third and fourth, as well as five
graduates. A total of 20 respondents were included in this survey.
The interviews were organised with each student and graduate separately. Each
of the interviews lasted approximately 45-50 min. Students and graduates have
voluntarily participated in the interviews. The difference between sophomores
and third- and fourth-year students was that sophomores felt more limited in
giving answers due to their beliefs about the portfolio and its relevance –
reflection, while third- and fourth-year students felt more confident in giving
answers since they already had a clear portfolio effect on their professional
development as it was their reflection on the quality of the work they did. The
graduates were clear about the importance of reflection through the portfolio.
With the permission of students and graduates, all interviews were recorded and
transcribed. The final transcription report was provided to the participants for
verification and signature. By signing these reports, they have confirmed that
29
http://ijlter.org/index.php/ijlter
their answers were correct and the meaning they wanted to give has not changed
through these statements.
The research was conducted to answer the following research questions:
1. ‘How will a portfolio help pre-service teachers in their professional
development’?
2. ‘How do pre-service teachers understand the importance of a portfolio at
different stages of studies’?
3. ‘What impact does pre-service teacher alumni portfolio have on
instructors’ employment’?
4. ‘In what way does self-reflection orient pre-service teachers towards
motivation for professional development’?
5. ‘How does self-reflection affect in-service teachers for continuous
professional development’?
6. ‘How appropriate is the portfolio as a tool to develop self-reflection in pre-
service teachers’?
The questions asked in the interview were semi-structured and intended to
obtain the students’ opinions on the following:
• The importance of portfolio preparation and reflection for pre-service
preschool teachers;
• The benefits of portfolio preparation;
• The portfolio’s significance for professional development; and
• The portfolio’s importance in employment.
4. Research outcomes
4.1 The importance of portfolio preparation and reflection for pre-service
preschool teachers
Based on interviews regarding students’ opinions on the preparation of portfolios,
it is evident that learners had very different perceptions. Since the concept of
portfolio creation is perceived differently by students, varying in various levels of
study, second-year students are found to be more inclined to viewing it as a chore
that should be checked off of their list of obligations;
“I spend a lot of time preparing my portfolio…it takes hard work.”
Nevertheless, third-year students seem to understand its importance in their
professional upgrading. They begin to consider portfolios as a useful tool which
they will continue to use at their jobs even after graduation. A third-year student
stated that:
“In the beginning, I thought it was not important but now it seems to be
very helpful.”
By contrast, fourth-year students or alumni do not view portfolios as a task but
instead as a tool for summarising and evidencing works and professional ideas.
Similar to the previous statement, a fourth-year student declared that:
“By using the portfolio, I know what I’m doing…selection of materials is
important for ongoing development.”
30
http://ijlter.org/index.php/ijlter
Meanwhile, the alumni expressed that:
“The portfolio is a necessary tool as it helps to reflect on professional
development.”
Based on these answers, it can be concluded that the portfolio is seen as an
exceedingly important tool, necessary for the ongoing professional development
of graduates who have entered the teaching profession.
4.2 The benefits of portfolio preparation
There is a discrepancy between second-year students and other years students’
opinions regarding the benefits of preparing a portfolio. A second-year student
stated that:
“Portfolio helps me to get a good grade.”
Meanwhile, third-year students stated that:
“The portfolio helps me recognise my weaknesses…”
This attitude is directly related to the reflection of the pre-service teacher related
to his/her work. Through portfolios, students manage to self-evaluate their work
and professional development. On the other hand, a fourth-year student notes the
portfolio’s importance as an opportunity for the development of critical and
creative thinking. In this regard, she declares that:
“The portfolio helped me to be a critical thinker in my work and
professional development. By using the portfolio, I can easily identify my
strengths and my weaknesses.”
Moreover, an alumni student considers portfolios as a tool that helps her to focus
on completing her objectives. In this regard, she stated:
“The portfolio has helped me become more efficient, improve my quality
of work in planning and my cooperation with colleagues.”
Based on the statements provided by the students, it has been concluded that
second-year students consider the portfolio as a means to achieve the highest
grades, while third-year, fourth-year and alumni students view it as an essential
implement for professional development. Their opinions have developed
precisely from their experiences with portfolio preparation. Self-reflection on the
professional development of pre-service teachers is more witnessed among the
fourth-year students than the students of previous years. They have already
realised the importance of reflecting on their work, and through this, they begin
to make ongoing plans for their professional development. The reflection itself is
even more emphasized among the graduates. Through statements, they have
emphasized that through this form, they become more efficient and focus to a
greater extent on the quality of their work. Reflections has allowed them to
improve their professional practices and knowledge, thus making them better
prepared teachers.
4.3 The portfolio’s significance for professional development
Students have expressed the significant contribution of portfolios to their
professional development as well as their reasoning behind it. Even in the case of
31
http://ijlter.org/index.php/ijlter
second-year students, it is evident that their lack of experience makes them more
inclined to consider portfolio as a task through which they can learn. At this level,
they have not yet fully established the importance of portfolios in their personal
and professional development. Therefore, by continuing to think that the
employment of portfolios enables them to learn new things and compare their
work with others, the portfolio becomes more appealing. A second-year student
declared:
“I do research so that I can have a good portfolio, which in return will help
me improve my work. The portfolio also helps me to compare my work
with others.”
Meanwhile, the opinions of third-year students begin to present a higher level of
advancement regarding the importance of portfolios in their professional
development. Hence, third-year students are more inclined to consider it as an
encouraging tool for cooperation with others and as something that helps them
feel secure. In this regard, a third-year student stated:
“The portfolio has helped me to cooperate with others. Cooperation is a
job-requested skill…”
In contrast to the second and third-year students, the opinions of the fourth-year
students hold the portfolio as a source of new ideas, an instrument that provides
them with security, and more importantly, an opportunity for self-evaluation. A
fourth-year student declared:
“The preparation of portfolios has given me more confidence and I do
better quality work. I am more confident in what I do”
This allows for a better understanding of the fourth-year students’ view because
through their experience, whether they have been taught on campus or
internships, they have found the portfolio to have a significant impact on their
preparation for the teaching profession. Therefore, their opinions at this stage are
also more constructive and based on the achieved results. Meanwhile, graduates
see the importance of the portfolio as an evidence of their qualitative work with
which they can secure their job more easily, which encourages them to apply this
kind of assessment with the children involved in their work. In this regard, an
alumni declared:
“The portfolio is the summary of my work. Additionally, the portfolio has
enabled me to get a job at a private preschool.”
Based on the students and alumni’s answers, we can conclude that portfolios are
a critical tool in the professional development of pre-service teachers. By using
portfolios, pre-service teachers understand goals and ways to achieve those goals.
The portfolio is a testimony of the acquired knowledge and skills of pre-service
teachers, which consequently enables them to focus on their professional
development.
4.4 The portfolio’s importance in employment
Due to the portfolio being one of the most important tools for their professional
development, students have shared their opinions on the importance of the
portfolio in their employment phase. This question has been answered by third-
32
http://ijlter.org/index.php/ijlter
year students, fourth-year students and alumni since their chances of obtaining a
job are higher than those of second-year students. The students have described
their opinions about the professional preparation that they have achieved through
the development of portfolios. They explained how much the preparation of
portfolios has helped or will help them to find employment in their professional
field. Third-year students have said that the portfolio has helped them to love
their future profession and at the same time has helped them to develop
presentational skills. As a result, they can better present their work, and according
to them, these are the benefits that will give them advantage over others. One of
the opinions of the third-year student in this regard was:
“The portfolio has helped me to further develop my presentational skills
as I had to present my work. It has helped me to love my future profession
more…”
A fourth-year student stated that throughout the portfolio preparation process,
they are taught to organise and document their work, which is of great advantage
to all those who work. The student also said that developing portfolios requires
creativity and the use of new ideas and different materials, especially those that
are recycled so that the portfolio is more meaningfully presented to others.
“The portfolio helped me understand that it is necessary to be organised
at work… Portfolio preparation pushed me to be more creative and to use
recycled and reusable materials.”
Given the above-mentioned characteristics, students are convinced that they will
find jobs more easily since those aspects are required in the first interview.
Graduates have stated that during their studies, in certain cases, they did not
consider portfolio as useful work. However, when they applied for work, they
realised that a portfolio is an extraordinary tool used to present their professional
preparation:
“In the beginning, I thought I am doing work that nobody appreciates,
but now I see that the portfolio is an amazing testimony. Each time I look
at my portfolio, I feel like a teacher, and I will create a portfolio for my pre-
school students.”
Based on the interviewers’ opinions, we can conclude that the portfolio also plays
an important role in pre-service teacher employment. Through portfolios, pre-
service teachers can easily prove their professional development and present their
work to children. It is of utmost importance that they apply the same kind of
assessment to pre-school children; the application of formative assessments and
the documenting of the level of childhood achievement. Furthermore, they are
able to document the ideas, approaches and goals related to their teaching
profession.
5. Conclusions
According to the research data, portfolio is a siginificant factor in educators’
professional development. This instrument provides future teachers with the
opportunity to learn to self-reflect and to simultaneously organise their work
better. To create skills in view of collaborating with others directly and to reflect
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021
IJLTER.ORG Vol 20 No 2 February 2021

Recomendados

IJLTER.ORG Vol 20 No 11 November 2021 von
IJLTER.ORG Vol 20 No 11 November 2021IJLTER.ORG Vol 20 No 11 November 2021
IJLTER.ORG Vol 20 No 11 November 2021ijlterorg
7 views529 Folien
IJLTER.ORG Vol 21 No 4 April 2022 von
IJLTER.ORG Vol 21 No 4 April 2022IJLTER.ORG Vol 21 No 4 April 2022
IJLTER.ORG Vol 21 No 4 April 2022ijlterorg
29 views405 Folien
IJLTER.ORG Vol 20 No 7 July 2021 von
IJLTER.ORG Vol 20 No 7 July 2021IJLTER.ORG Vol 20 No 7 July 2021
IJLTER.ORG Vol 20 No 7 July 2021ijlterorg
4 views371 Folien
IJLTER.ORG Vol 19 No 5 May 2020 von
IJLTER.ORG Vol 19 No 5 May 2020IJLTER.ORG Vol 19 No 5 May 2020
IJLTER.ORG Vol 19 No 5 May 2020ijlterorg
16 views455 Folien
IJLTER.ORG Vol 20 No 4 April 2021 von
IJLTER.ORG Vol 20 No 4 April 2021IJLTER.ORG Vol 20 No 4 April 2021
IJLTER.ORG Vol 20 No 4 April 2021ijlterorg
3 views346 Folien
IJLTER.ORG Vol 20 No 6 June 2021 von
IJLTER.ORG Vol 20 No 6 June 2021IJLTER.ORG Vol 20 No 6 June 2021
IJLTER.ORG Vol 20 No 6 June 2021ijlterorg
3 views432 Folien

Más contenido relacionado

Similar a IJLTER.ORG Vol 20 No 2 February 2021

IJLTER.ORG Vol 19 No 10 October 2020 von
IJLTER.ORG Vol 19 No 10 October 2020IJLTER.ORG Vol 19 No 10 October 2020
IJLTER.ORG Vol 19 No 10 October 2020ijlterorg
8 views397 Folien
IJLTER.ORG Vol 19 No 8 August 2020 von
IJLTER.ORG Vol 19 No 8 August 2020IJLTER.ORG Vol 19 No 8 August 2020
IJLTER.ORG Vol 19 No 8 August 2020ijlterorg
7 views480 Folien
IJLTER.ORG Vol 21 No 5 May 2022 von
IJLTER.ORG Vol 21 No 5 May 2022IJLTER.ORG Vol 21 No 5 May 2022
IJLTER.ORG Vol 21 No 5 May 2022ijlterorg
16 views498 Folien
IJLTER.ORG Vol 21 No 6 June 2022 von
IJLTER.ORG Vol 21 No 6 June 2022IJLTER.ORG Vol 21 No 6 June 2022
IJLTER.ORG Vol 21 No 6 June 2022ijlterorg
11 views351 Folien
IJLTER.ORG Vol 22 No 2 February 2023 von
IJLTER.ORG Vol 22 No 2 February 2023IJLTER.ORG Vol 22 No 2 February 2023
IJLTER.ORG Vol 22 No 2 February 2023ijlterorg
6 views545 Folien
IJLTER.ORG Vol 21 No 8 August 2022 von
IJLTER.ORG Vol 21 No 8 August 2022IJLTER.ORG Vol 21 No 8 August 2022
IJLTER.ORG Vol 21 No 8 August 2022ijlterorg
11 views452 Folien

Similar a IJLTER.ORG Vol 20 No 2 February 2021(20)

IJLTER.ORG Vol 19 No 10 October 2020 von ijlterorg
IJLTER.ORG Vol 19 No 10 October 2020IJLTER.ORG Vol 19 No 10 October 2020
IJLTER.ORG Vol 19 No 10 October 2020
ijlterorg8 views
IJLTER.ORG Vol 19 No 8 August 2020 von ijlterorg
IJLTER.ORG Vol 19 No 8 August 2020IJLTER.ORG Vol 19 No 8 August 2020
IJLTER.ORG Vol 19 No 8 August 2020
ijlterorg7 views
IJLTER.ORG Vol 21 No 5 May 2022 von ijlterorg
IJLTER.ORG Vol 21 No 5 May 2022IJLTER.ORG Vol 21 No 5 May 2022
IJLTER.ORG Vol 21 No 5 May 2022
ijlterorg16 views
IJLTER.ORG Vol 21 No 6 June 2022 von ijlterorg
IJLTER.ORG Vol 21 No 6 June 2022IJLTER.ORG Vol 21 No 6 June 2022
IJLTER.ORG Vol 21 No 6 June 2022
ijlterorg11 views
IJLTER.ORG Vol 22 No 2 February 2023 von ijlterorg
IJLTER.ORG Vol 22 No 2 February 2023IJLTER.ORG Vol 22 No 2 February 2023
IJLTER.ORG Vol 22 No 2 February 2023
ijlterorg6 views
IJLTER.ORG Vol 21 No 8 August 2022 von ijlterorg
IJLTER.ORG Vol 21 No 8 August 2022IJLTER.ORG Vol 21 No 8 August 2022
IJLTER.ORG Vol 21 No 8 August 2022
ijlterorg11 views
IJLTER.ORG Vol 20 No 10 October 2021 von ijlterorg
IJLTER.ORG Vol 20 No 10 October 2021IJLTER.ORG Vol 20 No 10 October 2021
IJLTER.ORG Vol 20 No 10 October 2021
ijlterorg11 views
IJLTER.ORG Vol 21 No 7 July 2022 von ijlterorg
IJLTER.ORG Vol 21 No 7 July 2022IJLTER.ORG Vol 21 No 7 July 2022
IJLTER.ORG Vol 21 No 7 July 2022
ijlterorg6 views
IJLTER.ORG Vol 19 No 3 March 2020 von ijlterorg
IJLTER.ORG Vol 19 No 3 March 2020IJLTER.ORG Vol 19 No 3 March 2020
IJLTER.ORG Vol 19 No 3 March 2020
ijlterorg8 views
IJLTER.ORG Vol 22 No 1 January 2023 von ijlterorg
IJLTER.ORG Vol 22 No 1 January 2023IJLTER.ORG Vol 22 No 1 January 2023
IJLTER.ORG Vol 22 No 1 January 2023
ijlterorg21 views
IJLTER.ORG Vol 19 No 7 July 2020 von ijlterorg
IJLTER.ORG Vol 19 No 7 July 2020IJLTER.ORG Vol 19 No 7 July 2020
IJLTER.ORG Vol 19 No 7 July 2020
ijlterorg5 views
IJLTER.ORG Vol 19 No 4 April 2020 von ijlterorg
IJLTER.ORG Vol 19 No 4 April 2020IJLTER.ORG Vol 19 No 4 April 2020
IJLTER.ORG Vol 19 No 4 April 2020
ijlterorg11 views
IJLTER.ORG Vol 22 No 3 March 2023 von ijlterorg
IJLTER.ORG Vol 22 No 3 March 2023IJLTER.ORG Vol 22 No 3 March 2023
IJLTER.ORG Vol 22 No 3 March 2023
ijlterorg4 views
IJLTER.ORG Vol 22 No 4 April 2023 von ijlterorg
IJLTER.ORG Vol 22 No 4 April 2023IJLTER.ORG Vol 22 No 4 April 2023
IJLTER.ORG Vol 22 No 4 April 2023
ijlterorg6 views
IJLTER.ORG Vol 20 No 5 May 2021 von ijlterorg
IJLTER.ORG Vol 20 No 5 May 2021IJLTER.ORG Vol 20 No 5 May 2021
IJLTER.ORG Vol 20 No 5 May 2021
ijlterorg5 views
IJLTER.ORG Vol 20 No 1 January 2021 von ijlterorg
IJLTER.ORG Vol 20 No 1 January 2021IJLTER.ORG Vol 20 No 1 January 2021
IJLTER.ORG Vol 20 No 1 January 2021
ijlterorg4 views
IJLTER.ORG Vol 22 No 5 May 2023 von ijlterorg
IJLTER.ORG Vol 22 No 5 May 2023IJLTER.ORG Vol 22 No 5 May 2023
IJLTER.ORG Vol 22 No 5 May 2023
ijlterorg4 views
IJLTER.ORG Vol 20 No 3 March 2021 von ijlterorg
IJLTER.ORG Vol 20 No 3 March 2021IJLTER.ORG Vol 20 No 3 March 2021
IJLTER.ORG Vol 20 No 3 March 2021
ijlterorg4 views
IJLTER.ORG Vol 20 No 9 September 2021 von ijlterorg
IJLTER.ORG Vol 20 No 9 September 2021IJLTER.ORG Vol 20 No 9 September 2021
IJLTER.ORG Vol 20 No 9 September 2021
ijlterorg7 views
IJLTER.ORG Vol 19 No 9 September 2020 von ijlterorg
IJLTER.ORG Vol 19 No 9 September 2020IJLTER.ORG Vol 19 No 9 September 2020
IJLTER.ORG Vol 19 No 9 September 2020
ijlterorg6 views

Más de ijlterorg

IJLTER.ORG Vol 21 No 12 December 2022 von
IJLTER.ORG Vol 21 No 12 December 2022IJLTER.ORG Vol 21 No 12 December 2022
IJLTER.ORG Vol 21 No 12 December 2022ijlterorg
11 views299 Folien
IJLTER.ORG Vol 21 No 11 November 2022 von
IJLTER.ORG Vol 21 No 11 November 2022IJLTER.ORG Vol 21 No 11 November 2022
IJLTER.ORG Vol 21 No 11 November 2022ijlterorg
17 views470 Folien
IJLTER.ORG Vol 21 No 10 October 2022 von
IJLTER.ORG Vol 21 No 10 October 2022IJLTER.ORG Vol 21 No 10 October 2022
IJLTER.ORG Vol 21 No 10 October 2022ijlterorg
21 views463 Folien
IJLTER.ORG Vol 21 No 9 September 2022 von
IJLTER.ORG Vol 21 No 9 September 2022IJLTER.ORG Vol 21 No 9 September 2022
IJLTER.ORG Vol 21 No 9 September 2022ijlterorg
14 views517 Folien
IJLTER.ORG Vol 19 No 11 November 2020 von
IJLTER.ORG Vol 19 No 11 November 2020IJLTER.ORG Vol 19 No 11 November 2020
IJLTER.ORG Vol 19 No 11 November 2020ijlterorg
8 views302 Folien
IJLTER.ORG Vol 19 No 6 June 2020 von
IJLTER.ORG Vol 19 No 6 June 2020IJLTER.ORG Vol 19 No 6 June 2020
IJLTER.ORG Vol 19 No 6 June 2020ijlterorg
3 views414 Folien

Más de ijlterorg(10)

IJLTER.ORG Vol 21 No 12 December 2022 von ijlterorg
IJLTER.ORG Vol 21 No 12 December 2022IJLTER.ORG Vol 21 No 12 December 2022
IJLTER.ORG Vol 21 No 12 December 2022
ijlterorg11 views
IJLTER.ORG Vol 21 No 11 November 2022 von ijlterorg
IJLTER.ORG Vol 21 No 11 November 2022IJLTER.ORG Vol 21 No 11 November 2022
IJLTER.ORG Vol 21 No 11 November 2022
ijlterorg17 views
IJLTER.ORG Vol 21 No 10 October 2022 von ijlterorg
IJLTER.ORG Vol 21 No 10 October 2022IJLTER.ORG Vol 21 No 10 October 2022
IJLTER.ORG Vol 21 No 10 October 2022
ijlterorg21 views
IJLTER.ORG Vol 21 No 9 September 2022 von ijlterorg
IJLTER.ORG Vol 21 No 9 September 2022IJLTER.ORG Vol 21 No 9 September 2022
IJLTER.ORG Vol 21 No 9 September 2022
ijlterorg14 views
IJLTER.ORG Vol 19 No 11 November 2020 von ijlterorg
IJLTER.ORG Vol 19 No 11 November 2020IJLTER.ORG Vol 19 No 11 November 2020
IJLTER.ORG Vol 19 No 11 November 2020
ijlterorg8 views
IJLTER.ORG Vol 19 No 6 June 2020 von ijlterorg
IJLTER.ORG Vol 19 No 6 June 2020IJLTER.ORG Vol 19 No 6 June 2020
IJLTER.ORG Vol 19 No 6 June 2020
ijlterorg3 views
IJLTER.ORG Vol 19 No 2 February 2020 von ijlterorg
IJLTER.ORG Vol 19 No 2 February 2020IJLTER.ORG Vol 19 No 2 February 2020
IJLTER.ORG Vol 19 No 2 February 2020
ijlterorg4 views
IJLTER.ORG Vol 19 No 1 January 2020 von ijlterorg
IJLTER.ORG Vol 19 No 1 January 2020IJLTER.ORG Vol 19 No 1 January 2020
IJLTER.ORG Vol 19 No 1 January 2020
ijlterorg5 views
IJLTER.ORG Vol 21 No 2 February 2022 von ijlterorg
IJLTER.ORG Vol 21 No 2 February 2022IJLTER.ORG Vol 21 No 2 February 2022
IJLTER.ORG Vol 21 No 2 February 2022
ijlterorg4 views
IJLTER.ORG Vol 20 No 12 December 2021 von ijlterorg
IJLTER.ORG Vol 20 No 12 December 2021IJLTER.ORG Vol 20 No 12 December 2021
IJLTER.ORG Vol 20 No 12 December 2021
ijlterorg6 views

Último

Jibachha publishing Textbook.docx von
Jibachha publishing Textbook.docxJibachha publishing Textbook.docx
Jibachha publishing Textbook.docxDrJibachhaSahVetphys
51 views14 Folien
REPRESENTATION - GAUNTLET.pptx von
REPRESENTATION - GAUNTLET.pptxREPRESENTATION - GAUNTLET.pptx
REPRESENTATION - GAUNTLET.pptxiammrhaywood
138 views26 Folien
Women from Hackney’s History: Stoke Newington by Sue Doe von
Women from Hackney’s History: Stoke Newington by Sue DoeWomen from Hackney’s History: Stoke Newington by Sue Doe
Women from Hackney’s History: Stoke Newington by Sue DoeHistory of Stoke Newington
163 views21 Folien
Psychology KS5 von
Psychology KS5Psychology KS5
Psychology KS5WestHatch
119 views5 Folien
Psychology KS4 von
Psychology KS4Psychology KS4
Psychology KS4WestHatch
98 views4 Folien
UNIDAD 3 6º C.MEDIO.pptx von
UNIDAD 3 6º C.MEDIO.pptxUNIDAD 3 6º C.MEDIO.pptx
UNIDAD 3 6º C.MEDIO.pptxMarcosRodriguezUcedo
134 views32 Folien

Último(20)

REPRESENTATION - GAUNTLET.pptx von iammrhaywood
REPRESENTATION - GAUNTLET.pptxREPRESENTATION - GAUNTLET.pptx
REPRESENTATION - GAUNTLET.pptx
iammrhaywood138 views
Psychology KS5 von WestHatch
Psychology KS5Psychology KS5
Psychology KS5
WestHatch119 views
Psychology KS4 von WestHatch
Psychology KS4Psychology KS4
Psychology KS4
WestHatch98 views
Pharmaceutical Inorganic Chemistry Unit IVMiscellaneous compounds Expectorant... von Ms. Pooja Bhandare
Pharmaceutical Inorganic Chemistry Unit IVMiscellaneous compounds Expectorant...Pharmaceutical Inorganic Chemistry Unit IVMiscellaneous compounds Expectorant...
Pharmaceutical Inorganic Chemistry Unit IVMiscellaneous compounds Expectorant...
Ms. Pooja Bhandare133 views
11.28.23 Social Capital and Social Exclusion.pptx von mary850239
11.28.23 Social Capital and Social Exclusion.pptx11.28.23 Social Capital and Social Exclusion.pptx
11.28.23 Social Capital and Social Exclusion.pptx
mary850239312 views
Narration lesson plan von TARIQ KHAN
Narration lesson planNarration lesson plan
Narration lesson plan
TARIQ KHAN61 views
When Sex Gets Complicated: Porn, Affairs, & Cybersex von Marlene Maheu
When Sex Gets Complicated: Porn, Affairs, & CybersexWhen Sex Gets Complicated: Porn, Affairs, & Cybersex
When Sex Gets Complicated: Porn, Affairs, & Cybersex
Marlene Maheu85 views
Pharmaceutical Inorganic chemistry UNIT-V Radiopharmaceutical.pptx von Ms. Pooja Bhandare
Pharmaceutical Inorganic chemistry UNIT-V Radiopharmaceutical.pptxPharmaceutical Inorganic chemistry UNIT-V Radiopharmaceutical.pptx
Pharmaceutical Inorganic chemistry UNIT-V Radiopharmaceutical.pptx
Ms. Pooja Bhandare113 views
Monthly Information Session for MV Asterix (November) von Esquimalt MFRC
Monthly Information Session for MV Asterix (November)Monthly Information Session for MV Asterix (November)
Monthly Information Session for MV Asterix (November)
Esquimalt MFRC72 views
How to empty an One2many field in Odoo von Celine George
How to empty an One2many field in OdooHow to empty an One2many field in Odoo
How to empty an One2many field in Odoo
Celine George87 views
Solar System and Galaxies.pptx von DrHafizKosar
Solar System and Galaxies.pptxSolar System and Galaxies.pptx
Solar System and Galaxies.pptx
DrHafizKosar106 views

IJLTER.ORG Vol 20 No 2 February 2021

  • 1. International Journal of Learning, Teaching And Educational Research p-ISSN: 1694-2493 e-ISSN: 1694-2116 IJLTER.ORG Vol.20 No.2
  • 2. International Journal of Learning, Teaching and Educational Research (IJLTER) Vol. 20, No. 2 (February 2021) Print version: 1694-2493 Online version: 1694-2116 IJLTER International Journal of Learning, Teaching and Educational Research (IJLTER) Vol. 20, No. 2 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically those of translation, reprinting, re-use of illustrations, broadcasting, reproduction by photocopying machines or similar means, and storage in data banks. Society for Research and Knowledge Management
  • 3. International Journal of Learning, Teaching and Educational Research The International Journal of Learning, Teaching and Educational Research is a peer-reviewed open-access journal which has been established for the dissemination of state-of-the-art knowledge in the fields of learning, teaching and educational research. Aims and Objectives The main objective of this journal is to provide a platform for educators, teachers, trainers, academicians, scientists and researchers from over the world to present the results of their research activities in the following fields: innovative methodologies in learning, teaching and assessment; multimedia in digital learning; e-learning; m-learning; e-education; knowledge management; infrastructure support for online learning; virtual learning environments; open education; ICT and education; digital classrooms; blended learning; social networks and education; e- tutoring: learning management systems; educational portals, classroom management issues, educational case studies, etc. Indexing and Abstracting The International Journal of Learning, Teaching and Educational Research is indexed in Scopus since 2018. The Journal is also indexed in Google Scholar and CNKI. All articles published in IJLTER are assigned a unique DOI number.
  • 4. Foreword We are very happy to publish this issue of the International Journal of Learning, Teaching and Educational Research. The International Journal of Learning, Teaching and Educational Research is a peer-reviewed open-access journal committed to publishing high-quality articles in the field of education. Submissions may include full-length articles, case studies and innovative solutions to problems faced by students, educators and directors of educational organisations. To learn more about this journal, please visit the website http://www.ijlter.org. We are grateful to the editor-in-chief, members of the Editorial Board and the reviewers for accepting only high quality articles in this issue. We seize this opportunity to thank them for their great collaboration. The Editorial Board is composed of renowned people from across the world. Each paper is reviewed by at least two blind reviewers. We will endeavour to ensure the reputation and quality of this journal with this issue. Editors of the February 2021 Issue
  • 5. VOLUME 20 NUMBER 2 February 2021 Table of Contents Chronometric Constructive Cognitive Learning Evaluation Model: Measuring the Construction of the Human Cognition Schema of Psychology Students .........................................................................................................................1 Guadalupe Elizabeth Morales-Martinez, Janneth Trejo-Quintana, David Jose Charles-Cavazos, Yanko Norberto Mezquita- Hoyos, Miriam Sanchez-Monroy Portfolio as a Tool for Self- Reflection and Professional Development for Pre-Service Teachers.............................. 22 Arlinda Beka, Ganimete Kulinxha The Efficacy of Habits of Mind in the Inculcation of Self-Directed Learning Skills in Pre-Service Teachers ........... 36 Gideon Petrus Van Tonder, Byron John Bunt, Alvine Petzer, Hermann Davin Bosch, Nicolaas Van Deventer, Amanda Gerber, Lizelle Van Schadewijk The Impact of Professional Development, Modern Technologies on Lecturers’ Self-Efficacy: Implication for Sustainable Science Education in Developing Nations....................................................................................................61 David Agwu Udu, Benson Ikechukwu Igboanugo, John Nmadu, Chidebe Chijioke Uwaleke, Benjamin Chukwunonso Okechineke, Adaora Phina Anudu, Precious Chisom Attamah, David Onyemaechi Ekeh, Mercy Ifunanya Ani A Competence Model to Assess and Develop Designing Competence Assessment Tool .......................................... 81 Do Tra Huong, Nguyen Thi Dieu Linh Cultural Language Learning Approach (CLLA): An Approach for Learners’ Integrated Language Use Acculturation....................................................................................................................................................................... 104 Hermayawati . The Impact of Using VAKT Strategy on Oral Reading and Reading Comprehension Skills of Elementary Students with Dyslexia....................................................................................................................................................................... 121 Ayed H. Ziadat The Influence of Architecture Students’ Learning Approaches on their Academic Performance in Two Nigeria Universities.......................................................................................................................................................................... 137 Gabriel Sen, Albert Adeboye, Oluwoye Alagbe Blended Learning in Rural Primary ESL Classroom: Do or Don’t............................................................................... 152 Melanie Jerry, Melor Md Yunus Developing Interaction in ESL Classes: An Investigation of Teacher-Student Interaction of Teacher Trainees in a Sri Lankan University......................................................................................................................................................... 174 Abdul Majeed Mohamed Navaz Teaching with Relevance: Saudi Students’ Perceptions of a Foundation Course in Communication Skills .......... 197 Philline M. Deraney Students’ Soft Skills, Hard Skills, and Competitiveness (SHC): A Suggested Model for Indonesian Higher Education Curriculum........................................................................................................................................................ 218
  • 6. Hadiyanto ., Noferdiman ., Syamsurizal ., Muhaimin ., Ina Krisantia Decision-making by Heads of Academic Department using Student Evaluation of Instruction (SEI) ................... 235 Mohammed Saleh Alkathiri Fostering Media Literacy Skills in the EFL Virtual Classroom: A Case Study in the COVID-19 Lockdown Period ............................................................................................................................................................................................... 251 Marina Bilotserkovets, Tatiana Fomenko, Oksana Gubina, Tetiana Klochkova, Oksana Lytvynko, Maryna Boichenko, Olena Lazareva Analysis of Future Teachers’ Perceptions on the Evaluation of Learning: A Case Study of Chile .......................... 270 José Hernández Sepúlveda, Rodrigo Panes Chavarría, Karla Rosalia Morales Mendoza Career Readiness of Graduating Office Technology and Management Students in Polytechnics in Southeast Nigeria.................................................................................................................................................................................. 290 Lynda Chineze Ezechukwu, Oliver Okechukwu Okanazu, Muhammadu Madu-Saba Babalulu, Ayoola Abimbola Arowolo, Chijioke Jonathan Olelewe Information Technology Integration Perception on Ghanaian Distance Higher Education: A Comparative Analysis................................................................................................................................................................................ 304 Albert Arthur Qua-Enoo, Brandford Bervell, Paul Nyagorm, Valentina Arkorful, John K. E. Edumadze ESL Teachers’ Challenges in Implementing E-learning during COVID-19 ................................................................ 330 Brenda Anak Lukas, Melor Md Yunus
  • 7. 1 ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0). International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 1-21, February 2021 https://doi.org/10.26803/ijlter.20.2.1 Chronometric Constructive Cognitive Learning Evaluation Model: Measuring the Construction of the Human Cognition Schema of Psychology Students Guadalupe Elizabeth Morales-Martinez and Janneth Trejo-Quintana Cognitive Science Laboratory, IISUE, National Autonomous University of Mexico, Mexico City, Mexico https://orcid.org/0000-0002-4662-229X https://orcid.org/0000-0002-7701-6938 David Jose Charles-Cavazos TecMilenio University, Mexico City, Mexico https://orcid.org/0000-0002-3445-9026 Yanko Norberto Mezquita-Hoyos Autonomous University of Yucatán, Yucatan, Mexico https://orcid.org/0000-0001-6305-7440 Miriam Sanchez-Monroy Tecnologico Nacional de Mexico-Instituto Tecnologico de Merida, Yucatan, Mexico https://orcid.org/0000-0001-5263-1216 Abstract. This study measured the structural and organizational changes in the knowledge schema of human cognition in response to the learning achieved by 48 students enrolled in the second year of a psychology degree. Two studies were carried out based on the Chronometric Constructive Cognitive Learning Evaluation Model. This article deals only with the first one, which consisted of a conceptual definition task designed in line with the Natural Semantic Network technique. Participants defined ten target concepts with verbs, nouns, or adjectives (definers), and then weighed the grade of the semantic relationship between the definers and the target concepts. The data indicate that the initial knowledge structures had been modified towards the end of the course. The participants’ human cognition schema presented changes in terms of content, organization, and structure. This evidence supports the idea that the acquisition and transformation of the schemata learned in academic environments may be observed through cognitive science indicators. Keywords: cognitive evaluation; knowledge schema; learning; NSN; psychology students
  • 8. 2 http://ijlter.org/index.php/ijlter 1. Introduction Assessing academic learning is one of the most significant challenges for educators in the twenty-first century. This is supported by William (2011), who asserted that assessing learning is a central activity in the instruction-learning process. There is a great diversity of learning measurement tools, especially given the development of new technology, which has opened up new possibilities in this field. However, there is still no consensus on the most convenient way to assess student learning. This problem means that although there is a diversity of tools with which to measure academic learning, there is no agreement about the best way to determine what and how much content a student has learned during a course. El-Yassin (2015) remarked that there is no right or wrong way to evaluate student learning since each instrument can inspect a specific learning aspect. In addition, William (2011) pointed out that although the sequence of presentation, quality, and even teaching in a class is the same for all students, they understand what they learn in the classroom differently and may even learn different things to what they are taught. This variability in students’ academic learning has long been considered a barrier to teaching rather than a source of enrichment within the classroom. In this regard, William (2011) discussed how for many years, those involved in the educational field assumed that the quality of instruction alone would be enough for students to learn, and failure to learn in spite of effective instruction was attributed to the students’ cognitive characteristics. Currently, the educational community is beginning to raise awareness about the role played by an individual student’s needs and cognitive characteristics in the design of teaching-learning sequences. Regarding the above, in the 1980s, Messick (1984) stressed that the interpretation of achievement measures should be carried out in the context of the style of instruction and learning to reduce errors in the interpretation of academic performance and students’ functioning within specific learning environments. Although this proposal sounds obvious, Messick explained that measuring learning in such an all-encompassing way is rarely feasible due to the complexity of the information that needs to be extracted at different levels of student life. In general, learning assessment can be very complex due to the broad spectrum of factors involved. According to Muskin (2015), the evaluation of learning implies using a means to determine what a person knows in conceptual or procedural terms. In this regard, Messick (1984) pointed out that school learning not only involves the content that a student can store in their memory, but also how the student structures or restructures their knowledge and cognitive skills according to their level of academic development (beginner, intermediate, or advanced). Messick (1984) suggested that any learning measurement should take account of the state of academic development of each student to establish the cognitive functioning level at which the learning assessment will be carried out. For example, Messick proposed that with students in an initial learning phase, the objective should be to acquire information. At this level, information-retrieval
  • 9. 3 http://ijlter.org/index.php/ijlter recognition assessments could be used. In contrast, at a more advanced level, student learning should manifest itself in the restructuring of schemata and the flexible use of schemata to solve problems. However, Messick saw the application of such a proposal as very forward-looking rather than being based on the reality of developing performance tests. Currently, most evaluation instruments are focused on performance measurement. In this regard, Banister (2004) pointed out that in psychology, the most commonly used instruments to measure learning are exams, practical tests, and empirical dissertations. These kinds of tools are used as summative assessments of student performance. The tests provide valuable information about aspects of students’ knowledge of the information evaluated in the test. However, they are not planned to have implications for the design of instruction techniques (Arieli-Attali, 2013). Exams have been criticized for being indirect measures that do not take into account context and that are more oriented towards obtaining a product rather than understanding the learning process (Sadeghi & Rahmati, 2017). Summative assessments are useful in this sense as they are used for what they were designed. However, when the main objective is to provide information on the processing of the information inputs that students receive in the classroom, rather than on the performance (the output from the process), then the necessary use of alternative tools to measure the cognitive processes of assimilation and accommodation of information as a result of learning becomes evident. Nevertheless, scientific exploration of the use and impact of evaluation tools to assess cognitive changes and provide useful indicators to correct or promote the restructuring of a learned schema is still an underexplored field. One way to approximate this learning-evaluation challenge is to include cognitive psychology tools to measure the human mind. This scientific discipline has high potential to evaluate skills (Embretson, 1999) and the formation of knowledge structures, and can thus be applied to different aspects of the learning process. For example, Marzano’s Learning Dimensions Model identifies five kinds of thinking involved in the learning process: a) attitudes and perceptions, b) acquisition and integration of knowledge, c) extending and refining knowledge, d) the meaningful use of knowledge, and e) mental habits (Marzano & Pickering, 1997). The measurement of these dimensions can be approximated with the paradigms and research techniques involved in human cognition science. For example, the research techniques used to explore human memory can be extrapolated to studying the cognitive mechanisms involved in dimensions b, c, and d of the Learning Dimensions Model. Arieli-Attali (2013) stated that the idea of including advances in cognitive science to develop new forms of measurement or complement psychometric means of evaluation is not new. Initiatives have been emerging since the last century to link advances in cognitive psychology to the measurement of abilities. For example, the Air Force Human Resources Lab carried out the Learning Abilities Measurement Project (LAMP) (Kyllonen & Christal, 1988), which sought to
  • 10. 4 http://ijlter.org/index.php/ijlter identify indicators of student learning and achievement, taking into account measures for processing capacity, speed of processing, knowledge, and skills. The results of this seminal effort demonstrated that cognitive measures could successfully predict performance in learning tasks and even do so with greater precision than some instruments already available. Later initiatives such as the Cognitive Design System (CDS) (Embreston, 1999) or Evidence-centered Design (ECD) (Mislevy, Steinberg & Almond, 2003) have continued to promote the concept of using cognitive tools within the assessment of learning. The Chronometric Constructive Cognitive Learning Evaluation Model (C3-LEM) by Lopez and Morales (Lopez et al., 2014; Morales-Martinez & Lopez-Ramirez, 2016; also see Morales-Martínez, 2020; Morales-Martinez et al., 2017; Morales- Martinez, Lopez-Ramirez & Lopez-Gonzalez, 2015) is a recent initiative to promote the use of cognitive measurement tools to evaluate academic learning. This evaluation model is based on applying the laws and principles for how the human mind selects, stores, and retrieves information. From cognitive psychology, the human mind is seen as a producer of cognitive structures called schemata. These mental structures are formed with the knowledge that people store in their memories. Schemata possess properties relating to their flexibility and stability. In the educational field, the students form schemata from materials learned on a course or in a career. These schemata can remain or be modified over time, depending on how students store, organize and structure their learning. Keeping the above idea in mind, Lopez (1989) proposed an academic-failure-rate predictor system based on evaluation techniques derived from the Theory of Human Information Processing (HIP) and the Theory of Parallel Distributed Processing (PDP). Lopez attempted to show that the study techniques from these areas allow the properties of learned-knowledge schemata to be observed and measured in the same way that general knowledge schemata can be observed. He tested this idea in his doctoral thesis, by designing and applying the Semantic Analyzer of Schemata Organization (SASO). This system allowed him to explore knowledge schemata in human memory (Lopez, 1996; Lopez & Theios, 1992). Later, Lopez et al. (2014) used this model to create a new system by which to evaluate learning. This learning-evaluation system was the origin of the Cognitive Evaluator (known in Spanish as EVCOG), which is a computerized system that assesses academic learning, and which gave rise to the C3-LEM developed by Morales-Martinez & Lopez-Ramirez (2016; also see Morales-Martinez et al. 2017, Morales-Martinez, Ángeles-Castellanos et al. 2020). The C3-LEM (Figure 1) offers an alternative way to measure various aspects of mental representation of the knowledge students learn in academic courses. For example, this model allows indicators on the schematic organization of knowledge to be obtained. Arieli-Attali (2013) pointed out that measuring the conceptual understanding advances of students during a course can provide useful information to support the design of teaching and learning strategies that help students learn the knowledge and skills necessary to adapt to an
  • 11. 5 http://ijlter.org/index.php/ijlter environment whose economy is based precisely on information and knowledge management. Figure 1. Phases and components of the C3-LEM Note: From “Cognitive e-tools for diagnosing the state of medical knowledge in students enrolled for a second time in an anatomy course,” by Morales-Martinez, Ángeles- Castellanos et al., 2020, International Journal of Learning, Teaching and Educational Research, 19(9), p. 346 (https://doi.org/10.26803/ijlter.19.9.18). Copyright 2020 by the authors and IJLTER.ORG. Figure 1. illustrates the phases and components that make up the C3-LEM. In general, this evaluation model promotes the combined and intertwined use of mental representation techniques, computational simulation tools, and chronometric cognitive measurement techniques to assess the modifications in the organization and mental structure of knowledge, as well as the dynamics and temporal changes in the learned schemata (Morales-Martinez, Ángeles- Castellanos et al., 2020; Morales-Martinez, Lopez-Perez et al., 2020). C3-LEM studies are based on the EVCOG procedure, which consists of two phases: constructive cognitive evaluation and chronometric cognitive evaluation (Figure 1). Together, these two approaches provide indicators of students’ cognitive mechanisms in terms of their ability to select, elaborate on, and build knowledge from the information obtained from an academic course. This article focuses on using the constructive cognitive evaluation of knowledge since it illustrates the first step for evaluating learning with C3-LEM. The objective is to contribute empirical evidence on the usefulness of cognitive techniques for measuring organization and structural changes in students’ knowledge schemata due to the learning process in a human cognition course. 1.1. Constructive Cognitive Evaluation of Knowledge Schemata Learned during an Academic Course The constructive cognitive evaluation of learning involves measuring the knowledge schema’s properties through a mental representation technique and computer simulations. The central idea is to observe the conceptual changes that occur in the student’s memory due to the learning process.
  • 12. 6 http://ijlter.org/index.php/ijlter Typically, the first step consists of applying the Natural Semantic Network (NSN) technique at the beginning and the end of the academic year (see the Methodology section), although any other technique that allows organization indicators and conceptual structure to be extracted can be used. Figueroa, Gonzalez & Solis (1976) proposed the NSN as a mental representation technique to explore meaning formation. According to Figueroa-Nazuno (2007), the construction of meaning depends entirely on the person who constructs it. The person elaborates and interprets knowledge through a constructive and reconstructive process of memory. So, from this conceptualization of cognitive functioning, the formation of meaning goes beyond free association. Mental representation studies based on the C3-LEM have provided evidence that students construct or reconstruct their declarative knowledge schemata as a result of the learning obtained during a course. For example, Morales-Martinez, Lopez- Perez et al. (2020) applied the NSN technique to measure the knowledge schema arising from a course on the Computational Theory of Mind. They observed that students enter the course with a pre-schema. However, no conceptual organization could be identified between the pre-schema nodes. After the course, the students had assimilated new concepts, eliminated some information nodes, and established an organization amongst the conceptual nodes they had learned during the course. These results agree with Bower’s (1975) seminal idea that the acquisition of declarative schemata embraces the incorporation of new information nodes. Moreover, the studies using NSN have been able to identify limitations in the knowledge structures of students, relating to each individual’s level of academic development in terms of the subject they are learning. Morales-Martínez, Mezquita-Hoyos et al. (2018) noted that students who did not achieve passing grades on the computational usability course had fractured knowledge schemata at the end of the course. Morales-Martinez, Angeles-Castellanos et al. (2020) reported similar data in their cognitive diagnostic study on the structure and organization of the human anatomy knowledge schema amongst first-year medical students. The data from this study pointed to fractured cognitive structure in the schema and difficulties with conceptual organization. Some reasons for schematic fragmentation include the relevance weight given to the different topics within a course or a lack of emphasis on establishing the relationships or connections between the topics reviewed during the academic course (Morales-Martinez, Ángeles-Castellanos et al., 2020). Fragmented knowledge structures are also observed in students starting a course to review a new topic (Morales-Martínez, López-Pérez et al., 2020; Urdiales-Ibarra et al., 2018). Information integration strategies influence the formation or correction of integration limitations in knowledge structures such as those mentioned above. In this regard, Morales-Martínez, Mezquita-Hoyos et al. (2018) reported that engineering students with a fractured schema at the end of their course managed
  • 13. 7 http://ijlter.org/index.php/ijlter to integrate information from the computational usability schema after attending a corrective course on the subject. In general, NSN provides information on how the student’s mind organizes and structures knowledge schemata according to the learning experiences during academic courses. Few studies exist which have used the C3-LEM approach to explore the knowledge domain in psychology. Specifically, the topics covered to date using C3-LEM relate to the Piagetian Theory schema and the Computational Theory of Mind (e.g., Morales-Martínez, López-Pérez et al., 2020). The results of these studies suggested that students start the courses with vague but pre-organized ideas about the knowledge that they will review throughout the course. At the end of the course, students with passing grades had acquired new information nodes in the cognitive structures related to their knowledge. Additionally, they had established new relationships between concepts and reconstructed or reorganized their schemata based on their learning experiences. However, more investigations offering empirical evidence on the learning properties of knowledge schemata in psychology are necessary to build a solid theory about the behavior of schemata in this field of knowledge. The present study contributes new information on the organization and schematic behavior of the knowledge structures acquired in one of the most relevant fields of psychology science, human cognition. 2. Methodology 2.1. Study Overview This research measured the state of knowledge on the human cognition schema amongst students enrolled in the second year of a psychology degree at the beginning and end of a course. The state of knowledge refers to the set of cognitive properties (organizational, structural, temporal, and dynamic) that characterizes students’ knowledge schemata in any academic course. For example, at the beginning of a course, students present less semantic richness than at the end of the course. In addition, throughout the course, students judge the semantic relevance of concepts in different ways. Moreover, the recognition pattern for schematic words is different at the beginning, during, and at the end of the course. Thus, this study explored the changes in the organization and structure of the human cognition schema experienced by students as a result of the learning acquired during a cognition course. The authors designed an NSN study that included a conceptual definition task related to the human cognition schema. 2.2. Participants The participants were 48 second-year psychology students enrolled in a course on human cognition. Their ages ranged from 19 to 34 years old (M = 20.3, SD = 2.58). Overall, 79% (38) were women and 21% (10) were men. The authors selected participants using a convenience sampling technique. Potential participants were included in the study only if they took part voluntarily and signed the informed consent. Participants who did not finish the two application phases or did not follow the instructions were excluded from the study.
  • 14. 8 http://ijlter.org/index.php/ijlter 2.3. Study Design The study design was based on the EVCOG sequence proposed in the C3-LEM. The researchers designed a mental representation study based on the modified NSN from Lopez and Theios (1992) and Lopez (1996). The objective was to measure the cognitive properties of the content, organization, and structure of the human cognition schema. 2.4. Instruments and Materials To build the NSN instrument, the researchers selected ten target concepts from the Protocol for the Collection of Target Concepts and Central and Deferred Definers (Morales-Martinez, 2015). This protocol guides the teacher or knowledge domain expert in terms of identifying the most relevant conceptual targets for the course. The resulting ten concepts were considered to be the evaluated schema concepts. The ten conceptual targets selected by the teacher were: cognition, cognitive psychology, perception, attention, consciousness, memory, representation of knowledge, reasoning, problem-solving, and decision-making. The researchers used EVCOG software to design and apply the cognitive studies of mental representation. Additionally, this software allowed the capture and analysis of data based on the C3-LEM (Morales-Martínez, López-Pérez et al., 2020). 2.5. Procedure In this study, the constructive cognitive evaluation of learning comprised the application of a task based on the NSN technique at the beginning and end of the course. First, the researchers invited students who were enrolled in a course on human cognition to participate in the research. Subsequently, the students who agreed to participate received information about the study and their rights as participants, and gave their informed consent. After this, they performed an exercise to familiarize themselves with the task. Finally, the NSN study was applied. During the NSN study, each participant observed the target concepts one by one on a computer screen. The task was to define the targets using verbs, nouns, adjectives, and pronouns as definers. The production criterion for definers was that they had to be directly related to their course content on human cognition. Phrases, articles, and prepositions were not allowed to be used for the definitional task. The participants had 60 seconds to define each target. Subsequently, they rated each definer using a scale from 1 to 10; 1 meant that the evaluated definer chosen was not very related to the target concept, and 10 indicated that the definer was significantly related to the target concept. The time to complete the entire task varied from 15 to 20 minutes, depending on each participant. 3. Data Analysis In this study, the authors undertook three analyses of the NSN data. The first analysis was a traditional mental representation analysis using the EVCOG system. This software allows several NSN values, proposed by Figueroa et al. (1976) and described by Lopez (1996) and Lopez and Theios (1992), to be
  • 15. 9 http://ijlter.org/index.php/ijlter computed. This analysis involved various elements which are described below. The indicators for the analysis included, firstly, semantic richness (J value), generated for each target concept through the total number of different definers. Secondly, semantic relevance (M value) was obtained from the score consciously given by the participants for each target definer, expressed as the sum of all the weights assigned by the participants to each definer. The ten most relevant defining concepts were also identified to build the meaning of the target concept of the network. This group of definers is known as the SAM group (Semantic analysis of M value or SAM) and is made up of the ten definers with the highest M values for each target concept. Another indicator was semantic distance (FMG value) between the given definer and the target concept that was defined. This is computed using the percentage range corresponding to the M value of each of the definers obtained for the SAM group in relation to the highest M value obtained in the group. Finally, semantic density (G value) was calculated. The second analysis was undertaken using the EVCOG system. This software allowed the extraction of the association matrix. This matrix is called the SASO connectivity matrix, which is calculated using a Bayesian formula proposed by Lopez and Theios (1992). According to these authors, this equation is a modification of that by Rumelhart et al. (1986). Lopez and Theios’s equation is given below: WIJ = -1n{[p(X = 0 & Y = 1) p(X = 1 & Y = 0)]*[p(X = 1 & Y = 1) p(X = 0 & Y = 0)]-1} [1] This equation calculates the co-occurrence probability amongst pairs of concepts (X and Y) throughout the NSN. Firstly, p(X = 0 & Y = 1) refers to the joint probability that Y appears but X does not appear in a SAM group. Similarly, p(X = 1 & Y = 0) denotes the joint probability that X appears but Y does not appear in a SAM group, and p(X = 1 & Y = 1) was computed in the same manner. The calculation of p(X = 1 & Y = 1) involved the hierarchical modulation of M values in the SAM groups. The SASO connectivity matrix was used to feed the Gephi software to obtain a graphical representation of the accommodation of schema concepts (see Figure 3). Finally, the authors used STATISTIC software (version 7) to apply a multidimensional scaling on the NSN data. To this end, the authors considered the co-occurrence of definer concepts for each target concept. 4. Findings/Results 4.1. Lopez and Theios’s Analysis of NSN Data The NSN data obtained before (Table 1) and after (Table 2) the course were analyzed based on the procedure described by Lopez and Theios (1992).
  • 16. 10 http://ijlter.org/index.php/ijlter Table 1. SAM groups for the human cognition schema obtained from the participants before the course Note: J = semantic richness, G = semantic density, F = occurrence frequency, M = semantic weight, IRT = inter-response time F Definer M IRT F Definer M IRT F Definer M IRT 9 Cognitive process 158 18 9 Cognitive process 159 22 2 Senses 97 26 3 Mind 115 23 3 Mind 147 18 2 Interpret 67 35 5 Memory 110 27 5 Memory 141 36 2 Stimuli 63 23 3 Learning 88 32 5 Thought 93 26 9 Cognitive process 54 33 5 Thought 79 20 3 Learning 75 24 1 Feel 51 24 4 Attention 64 32 4 Attention 73 35 4 Attention 43 22 4 Perception 44 38 4 Perception 72 37 3 Brain 36 54 5 Capacity 40 41 1 Study 69 16 1 Observe 34 22 1 Processing 36 33 1 Behavior 64 29 4 Information 33 34 3 Brain 35 31 1 Science 47 18 1 Reality 26 41 F Definer M IRT F Definer M IRT F Definer M IRT 9 Cognitive process 129 24 3 Mind 78 23 1 STM 127 26 1 Focus 91 27 9 Cognitive process 55 30 1 LTM 105 29 5 Capacity 73 31 1 Mind state 45 11 1 Store 94 14 2 Stimuli 66 19 5 Thought 44 38 3 Learning 91 25 1 Concentrate 58 23 4 Attention 42 27 9 Cognitive process 88 28 5 Memory 38 24 3 Brain 40 44 1 Memories 86 24 2 Senses 36 38 1 Vigil 39 29 1 Remember 67 14 1 Selective attention 33 20 1 Internal 33 42 1 WM 65 30 4 Perception 31 47 2 Cognition 31 41 4 Information 63 32 2 Cognition 30 72 3 Reasoning 30 23 1 Retrieve 59 33 J-value: 205 J-value: 174 J-value: 282 F Definer M IRT F Definer M IRT F Definer M IRT 1 Schemata 95 22 3 Thinking 117 16 3 Reasoning 64 18 1 Image 81 20 9 Cognitive process 72 20 9 Cognitive process 56 28 1 Symbols 43 15 1 Human 47 22 3 Thinking 52 18 1 Models 39 21 5 Thought 46 25 2 Reason 51 16 1 Mental 36 15 3 Analysis 45 33 5 Memory 46 27 4 Perception 33 32 1 Logic 43 19 5 Capacity 44 19 4 Information 31 31 1 Consciousness 32 33 3 Analysis 41 28 1 Object 28 27 5 Capacity/Ability 31 27 2 Choice 39 36 5 Memory 27 39 4 Information 29 30 2 Options 38 41 1 Concepts 27 37 2 Interpretation 27 46 5 Thought 38 32 J-value: 175 J-value: 200 J-value: 192 F Definer M IRT 2 Choice 104 17 9 Cognitive process 73 34 2 Options 58 30 1 Evaluation 53 39 2 To reason 42 18 1 Solutions 38 31 5 Capacity 34 33 3 Reasoning 33 26 3 Analysis 31 40 3 Thinking 27 50 Cognition Cognitive psychology Perception J-value: 218 G-value 12.30 J-value: 258 G-value: J-value: 217 G-value: 7.10 Attention Consciousness Memory G-value: 9.90 G-value: 4.80 G-value: 6.80 Representation Reasoning Problem solving G-value: 6.80 G-value: 9.00 G-value: 2.60 Decision making J-value: 212 G-value: 4.60
  • 17. 11 http://ijlter.org/index.php/ijlter Table 2. SAM groups for the human cognition schema obtained from the participants after the course Note: J = semantic richness, G = semantic density, F = occurrence frequency, M = semantic weight, IRT = inter-response time Table 1 shows that the definers (cognitive process, mind, memory, short-term memory (STM), thinking, long-term memory (LTM), choice, senses, schemata) with the highest M in each SAM group before the course were mostly general. At the end of the course, however, most of the concepts with the highest F Definer M IRT F Definer M IRT F Definer M IRT 10 Cognitive process 259 13 10 Cognitive process 230 26 1 Sensation 182 13 3 Information 98 28 1 Science 167 20 1 Interpret 152 22 2 Mind 89 22 1 Neisser 113 23 1 Threshold 125 29 7 Memory 80 33 2 Cognition 92 30 2 Stimuli 116 27 1 Cold cognition 79 31 7 Memory 79 31 10 Cognitive process 110 32 3 Attention 68 33 1 HIP 78 29 1 Direct perception 84 26 1 Psychology 63 43 3 Information 76 31 1 Illusion 69 30 1 Hot cognition 54 29 1 Representation 63 41 1 Senses 62 24 1 Human 49 42 3 Attention 62 36 4 Perception 55 24 1 Processing 49 32 4 Perception 61 32 1 Gestalt 49 44 F Definer M IRT F Definer M IRT F Definer M IRT 1 Filter 211 22 3 Attention 134 25 1 Store 286 20 10 Cognitive process 190 20 10 Cognitive process 127 24 1 Retrieve 258 30 1 Selective attention 124 23 1 Become aware 94 17 1 LTM 232 32 1 Divided attention 95 31 7 Memory 83 35 1 STM 231 27 2 Stimuli 93 39 4 Perception 67 36 1 SM 230 26 1 Attenuation model 82 25 2 Knowledge 62 11 1 Encoding 147 29 1 Sustained attention 81 23 1 Unconscious 49 43 10 Cognitive process 112 22 1 Capacity 78 28 1 Explicit 48 42 1 WM 96 34 4 Perception 71 25 1 Reflector 41 36 1 Implicit 77 25 1 Focus 68 28 2 Cognition 41 49 2 Semantics 74 41 J-value: 360 J-value: 279 J-value: 411 F Definer M IRT F Definer M IRT F Definer M IRT 1 Schemata 261 23 1 Reasoning 239 20 1 Objective 114 30 2 Mind 198 19 1 Conclusion 216 25 10 Cognitive process 103 19 1 Concepts 142 26 1 Inductive 201 20 1 Problem 100 25 10 Cognitive process 99 25 1 Syllogism 175 26 2 Reasoning 87 30 7 Memory 98 33 1 Analogical 110 27 1 Goal 81 17 1 Images 74 20 10 Cognitive process 109 23 1 Heuristics 77 37 1 Imagine 73 14 1 Premises 86 24 7 Memory 76 48 2 Knowledge 59 24 3 Information 78 25 2 Decision 70 38 2 Semantics 43 42 7 Memory 66 34 1 Strategies 56 33 1 Absence 41 18 1 Logic 60 26 1 Initial state 55 29 J-value: 332 J-value: 344 J-value: 311 F Definer M IRT 1 Choice 222 14 1 Alternative 112 19 10 Cognitive process 105 29 1 Evaluation 94 19 1 Experience 80 25 2 Reasoning 79 37 2 Decision 78 18 1 Options 62 29 7 Memory 56 32 1 Normative theories 23 35 Cognition Cognitive psychology Perception J-value: 373 G-value: 21.00 J-value: 378 G-value: 16.90 J-value: 336 G-value: 13.30 Attention Consciousness Memory G-value: 14.30 G-value: 9.30 G-value: 21.20 Representation Reasoning Problem solving G-value: 22.00 G-value: 17.90 G-value: 5.90 Decision making J-value: 331 G-value: 5.90
  • 18. 12 http://ijlter.org/index.php/ijlter M in each SAM group were specific (cognitive process, sensation, filter, attention, store, schema, reasoning, objective, choice), as shown in Table 2. Besides, when comparing Tables 1 and 2, it can be observed that the students at the end of the course included new definers or information nodes, rearranged some definers, or eliminated concepts in the definitions of some targets. For example, the following definers for cognition: thought, capacity, perception, learning and brain were removed, and definers such as information, cold cognition, psychology, hot cognition, and human were included (Figure 2). Figure 2. Conceptual changes in the target cognition In general terms, cognitive process (M value = 159) was the definer with the greatest semantic weight in the entire network before the course (Table 1), whilst after the course, it was store (M value = 286) (Table 2). Additionally, cognitive process was
  • 19. 13 http://ijlter.org/index.php/ijlter the definition with the highest appearance frequency at the beginning of the course (F = 9) and also at the end of the course (F = 10). The M value average for cognitive process at the beginning of the course was 93.77, whereas at the end of the course, it had increased to 144.4. 4.2. Gephi Analysis of NSN Data The researchers carried out a graphical analysis of the changes in the organization and structure of the NSN using the Gephi system (Bastian, Heymann & Jacomy, 2009). Gephi is open-access software which explores the properties of networks. At the beginning of the course, the participants’ knowledge schema on human cognition was made up of four large modules of concepts (Figure 3). The first (blue) included memory-related definers (memories, learning, remembering, storing, retrieval, working memory (WM), short-term memory, long-term memory, information). The second group (purple) consisted of definers related to cognitive psychology as a science (science, study, cognition, mental state, observing, wakefulness, feeling, internal, reality, interpretation, reasoning, attention, senses, mind, thought, cognitive process, stimuli, processing, brain, behavior). The third grouping (orange) embraced definers related to decision-making (solutions, reasoning, thinking, evaluation, analysis, ability, options, choice, consciousness, human, logic). The fourth group of definers (green) was made up of concepts relating to cognitive processes (memory, selective attention, concentration, symbols, perception, focus, image, schemata, models, mind). After the course, the participants rearranged the human cognition schema into seven conceptual modules (Figure 3). The first module embraced definers associated with perception (orange) (senses, interpretation, illusion, sensation, threshold, direct perception, Gestalt). The second module (light green) included definers related to consciousness and attention (sustained attention, divided attention, selective attention, capacity, attenuation model, unconscious, filter, focus, realize, reflector, explicit). Module 3 (pink) grouped concepts related to problem- solving (initial state, strategies, problem, goal, heuristics, objective). Conceptual group 4 (dark green) encompassed definers on decision-making (alternative, options, choice, evaluation, experience). Module 5 (purple) concentrated concepts related to three objectives: cognitive psychology, cognition, and mental representation (schemata, images, absence, concepts, cold cognition, imagine, mind, processing, hot cognition, human, reasoning, cognition, stimuli, decision, semantics, memory, psychology, memory, Neisser, cognitive process, mental representation, HIP, science, attention, knowledge). Module 6 (brown) included definers on reasoning (deductive reasoning, premises, conclusion, logic, inductive, analogical, information, syllogism). The last module (blue) involved definers related to memory (sensory memory, short-term memory, long-term memory, retrieve, store, encoding, working memory, implicit). Additionally, the Gephi analysis pointed out changes in the conceptual organization. The conceptual connections of definers had changed at the end of the course. To illustrate these changes, observe in Figure 3 that at the beginning of the course, cognitive process was a central definer concept in the primary schema that participants brought about human cognition, although it did not have a
  • 20. 14 http://ijlter.org/index.php/ijlter connection with all the schema modules. At the end of the course, the concept of cognitive process retained its quality as a central conceptual node yet now also fully connected with all the targets and all the conceptual modules. Figure 3. Gephi analysis of the NSN data obtained before and after the course
  • 21. 15 http://ijlter.org/index.php/ijlter 4.3. Multidimensional Scaling of NSN Data The researchers applied multidimensional scaling to the NSN data to examine the general structure of the human cognition schema. The analysis showed changes in the arrangement of target conceptual nodes due to the learning achieved during the course (Figure 4). Figure 4. Multidimensional scaling analysis of the target concepts
  • 22. 16 http://ijlter.org/index.php/ijlter The multidimensional scaling graph shows that the participants started the course without a specific structure in mind for the objective concepts, whilst at the end of the course, they had rearranged the objective concepts based on two dimensions. The first related to categorizing cognitive processes in terms of basic and higher order cognition (horizontal axis). Although the definition of the second dimension is not clear, in general, this dimension seems to be related to the use of knowledge structures (vertical axis). Note that the target concept for reasoning does not appear alongside targets such as problem-solving or decision-making, even though all of these processes involve making use of knowledge structures from memory. 5. Discussion This study has explored changes in the knowledge schema due to the learning process during a course on human cognition taken by second-year psychology students. First, the authors determined whether a human cognition schema existed before the course. The NSN and Gephi analyses indicated that the participants entered the course with a previous-knowledge schema or a knowledge pre-schema (see Table 1 and Figure 3). The existence of a knowledge pre-schema has been observed in other studies (e.g., Morales-Martinez, Lopez- Perez et al., 2020); however, the organization and structure are rudimentary. This finding suggests that students generally have a vague schema about the knowledge they will acquire in their courses, and it is based on this schema that they reorganize and reconfigure the information they will learn in class. As psychology teachers, the authors have observed that the use of general schemata and previous learning to begin a new knowledge schema is a common phenomenon observed in the classroom. Students generally comment that they have come across certain information about the topic. It was therefore not unexpected that the participants in this study commented that they were slightly familiar with the topics. They had reviewed readings on cognitive processes in other courses, although this had not been from the perspective of the field of cognitive psychology. The authors hypothesize that students use their previous learning experiences to form a general schema or make inferences about information related to the course in which they are enrolled. In this way, they have a conceptual basis from which to form a more sophisticated outline of the information they cover during the course. From a cognitive perspective, students can use or create a rudimentary cognitive structure that allows them to guide the reorganization and restructuring of their knowledge based on the new information inputs that they acquire through the course. If the cognitive structure is sufficiently broad and general, it will be flexible enough to undergo modifications due to the new learning experiences. Interestingly, although the initial schema with which the participants in this study entered the course was very general, their schema was not fractured as has been observed in other courses where students start on a topic for the first time (e.g., Urdiales-Ibarra et al., 2018). This result may be because the participants in this study had reviewed cognition materials the previous year when taking different
  • 23. 17 http://ijlter.org/index.php/ijlter courses, meaning that they had had previous information about the topic. At the start of their degree, the participating students were enrolled in a course where they reviewed some of the concepts included in the course on human cognition and obtained a passing grade on this initial course. Thus, they had general and pre-organized ideas about the meaning of some important target concepts in NSN studied in this research. Other studies have indicated that students who do not obtain a passing grade for a course have a fragmented schema at the end of the course compared to those who end the course with a passing grade (Morales- Martinez, Angeles-Castellanos et al., 2020; Morales-Martinez, Mezquita-Hoyos et al., 2018). In this study, at the end of the course, the authors explored the changes that had taken place in the participants’ pre-knowledge schema of human cognition due to the learning acquired through the course. The analysis of the organization of the schematic knowledge indicated that the participants had established new relationships between the concepts. This result is consistent with Bower’s (1975) idea that the acquisition of declarative schemata necessarily involves incorporating new information nodes and new connections between these nodes. The reader can compare the definers included in Tables 1 and 2 and observe that at the beginning of the course, for some target concepts, some of the definers were global concepts on the topic of human cognition. Meanwhile, at the end of the course, the definers were more specific and theoretically closer to the target evaluated. For example, for the initial conceptual definition of human cognition (Figure 2), half of the concepts were categorical (memory, thought, attention, perception, learning), and the other half were schematic (cognitive process, capacity, mind, brain, processing). At the end of the course, however, the participants included a greater number of schematic-type definers (cognitive process, information, mind, cold cognition, psychology, hot cognition, human, processing). The change in predominance from categorical to schematic relationships in knowledge structures suggests that the participants had developed more sophisticated schemata. That is, instead of using as many exemplification schemata, their perception had changed and they were using more probabilistic schemata. It is possible that, when students start learning a knowledge domain, learning by exemplification dominates most of their knowledge acquisition process. As participants in this study acquired new knowledge and refined it, they began to use or establish other semantic relationships amongst the concepts. It would be useful to carry out further research to explore this phenomenon since there has been no discussion of this issue in previous research with C3-LEM to date (e.g., Morales-Martinez, Angeles-Castellanos et al., 2020; Morales-Martinez et al., 2020; Morales-Martínez, Mezquita-Hoyos et al., 2018; Urdiales-Ibarra et al., 2018). Another modification in the knowledge organization, which is of note, was the change in the degree of generality with regard to the human cognition schema. At the beginning of the course, the participants formed some groups that included general definers and even incorporated information from other knowledge
  • 24. 18 http://ijlter.org/index.php/ijlter schemata. For instance, module 2 of the Gephi analysis shows that before the course, participants included definitions of various target concepts (cognitive psychology, cognition, mental representation, perception) in the same group of concepts and included definitions of other knowledge schemata learned for other topics. For example, participants recovered conceptual nodes from the behaviorism field as stimuli instead of inputs or behavior instead of cognitive patterns (Figure 3). The previous results indicate that at the end of the course, the participants were able to extend and refine their knowledge about human cognition, thus placing them at level three of Marzano’s Dimensions of Learning Model (Marzano & Pickering, 1997). On the other hand, according to Messick (1984), the participants in the present study would be in an intermediate stage of academic development in terms of the development of the knowledge schema on human cognition because indicators observed included not just the retrieval of information but a restructuring of their schema. In congruence with this idea, the analysis of the structural changes in the knowledge schema indicated a reconfiguration of the schema structure by the end of the course. In this regard, Figure 3 shows how the initial schema’s definers were arranged into four large modules, whilst the definers for the final schema were restructured into seven conceptual modules. Changes in the configuration of the schematic structure have been observed in other studies that have used the C3-LEM (Morales-Martinez, Lopez-Perez et al., 2020; Morales-Martínez, Mezquita-Hoyos et al., 2018; Urdiales-Ibarra et al., 2018). From the point of view of cognitive psychology, changes in schematic configuration patterns are an indication of learning. In this study, the changes to the schema’s configurational arrangement suggest that participants had rebuilt their structures based on the new meanings that they had acquired during the course. The multidimensional analysis (Figure 4) on the target concepts showed that at the beginning of the course, the participants did not have a clear idea of how the course’s target concepts could form a wholly organized knowledge schema. At the end of the course, the participants organized the ten target concepts into two dimensions, the first one relating to the cognitive nature of the processes (basic vs. higher order cognition) and the second associated with the use of knowledge structures. Although some concepts such as reasoning were not correctly located in this second dimension, in general terms, this result suggests that the participants had understood the structure of knowledge underlying the course’s thematic organization, using the information implicit in the same target concepts. Since this is a seminal intent of introducing a new way to analyze the results from C3-LEM, more evidence about this phenomenon is needed to explore and explain this kind of implicit cognitive change in the knowledge schema as a learning product. In summary, the study results indicated that there were changes in the organization and structure of the human cognition knowledge schema of the participants. They had reconfigured their old four-module schema on human
  • 25. 19 http://ijlter.org/index.php/ijlter cognition into a new one which included seven modules. The participants included new information nodes, eliminated conceptual nodes that belonged to other disciplines, and established new relationships between the old and new concepts. 6. Conclusions In conclusion, the results of the present investigation have implications in three areas. At a theoretical level, the study generated empirical evidence that supports the idea that students enter courses with prior knowledge of the subject they are going to study. For example, the study data indicated that the participants possessed a macro-schema of human cognition at the beginning of the course. This finding is relevant because it suggests that cognitive techniques such as those contemplated in the C3-LEM can help diagnose preconceived ideas. It opens up the possibility of correcting inaccurate information held by students when starting a course. The measurement of this type of pre-schema would empower the teacher to decide whether it is necessary to demystify some information or whether modifications are required in the application of the established work program to provide continuity or correct the knowledge structures held by students when starting the course. In addition, the results demonstrated that the learning process involves the assimilation of new information and the elimination of specific conceptual nodes, as well as the restructuring of schematic information. Furthermore, evidence from the NSN study indicated that this type of technique can provide information on students’ academic development level in a course. This finding has important implications at the applied level. For example, how a student configures their knowledge can also be taken as an indicator of mastery of the course knowledge. Consequently, the C3-LEM could be a valuable tool in the formative assessment of students. However, since the sample in this study was small and only addressed one domain of knowledge, new explorations must be carried out in other fields, such as the area of exact sciences (e.g., mathematics, chemistry, physics), to calibrate the scope and implications of this evaluation model in the design of new forms of educational evaluation and intervention. Finally, at a methodological level, the study’s data supported the idea that mental representation studies from the C3-LEM perspective may help assess cognitive changes in the organization and structure of knowledge schemata. 7. References Arieli-Attali, M. (2013, October 20−25). Formative assessment with cognition in mind: The cognitively based assessment of, for and as learning [Paper presentation]. 39th Annual Conference: Educational Assessment 2.0: Technology in Educational Assessment. Tel Aviv, Israel. https://www.iaea.info/conference-proceedings/ Banister, P. (2004). Assessment as a tool for fostering key skills. Psychology Learning & Teaching, 3(2), 109–113. https://doi.org/10.2304/plat.2003.3.2.109 Bastian, M., Heymann, S., & Jacomy, M. (2009, May 17−20). Gephi: An open source software for exploring and manipulating networks [Paper presentation]. Association for the Advancement of Artificial Intelligence, Third International AAAI Conference on
  • 26. 20 http://ijlter.org/index.php/ijlter Weblogs and Social Media. San Jose California, United States. https://gephi.org/publications/gephi-bastian-feb09.pdf Bower, G. H. (1975). Cognitive psychology: An introduction. In W. K. Estes (Ed.), Handbook of learning and cognitive processes: Vol. 1. Introduction to concepts and issues (pp. 25– 80). Lawrence Erlbaum Associates. El-Yassin., H. D. (2015). Integrated assessment in medical education. Journal of Contemporary Medical Sciences, 1(4), 36–38. http://www.jocms.org/index.php/jcms/article/view/51 Embretson, S. E. (1999). Cognitive psychology applied to testing. In F. T. Durso (Ed.), Handbook of applied cognition (pp. 629−660). John Wiley & Sons. Figueroa-Nazuno, J. G. (2007, October 24−25). El significado de las Redes Semánticas Naturales: Y la tradición oral, 20 años después [The meaning of natural semantic networks: And oral tradition, 20 years later] [Paper presentation]. Primer Simposium Internacional: Cognición y Representación del Conocimiento. Monterrey, Nuevo Leon, Mexico. Figueroa, J. G., Gonzalez, E. G., & Solis, V. M. (1976). An approach to the problem of meaning: Semantic networks. Journal of Psycholinguistic Research, 5(2), 107−115. https://doi.org/10.1007/BF01067252 Kyllonen, P. C., & Christal, R. E. (1988). Cognitive modeling of learning abilities: A status report of LAMP (Learning Abilities Measurement Program). In R. Dillon & J. W. Pellegrino (Eds.), Testing: Theoretical and applied issues. Freeman. Lopez, R. E. O. (1989, December 1−6). Sistema predictor de indice reprobatorio (SPIR) [Failure rate predictor system (SPIR)] [Paper presentation]. IX Coloquio de Investigación. ENEP Iztacala, Mexico city, Mexico. Lopez, E. O. (1996). Schematically related word recognition (Publication No. 9613356) (Doctoral dissertation). University of Wisconsin-Madison, Madison, Wisconsin. ProQuest Dissertations & Theses Global. Lopez, E. O, & Theios, J. (1992). Semantic analyzer of schemata organization (SASO). Behavior Research Methods, Instruments, & Computers, 24(2), 277−285. https://link.springer.com/content/pdf/10.3758/BF03203508.pdf Lopez, E. O., Morales, G. E., Hedlefs, I., & Gonzalez, C. J. (2014). New empirical directions to evaluate online learning. International Journal of Advances in Psychology, 3(2), 40−47. https://doi.org/10.14355&ijap.2014.0302.03 Marzano, R. J., & Pickering, D. J. (1997). Dimensions of learning: Teacher’s manual (2nd ed.). ASCD. Messick, S. (1984). The psychology of educational measurement. Journal of Educational Measurement, 21(3), 215–237. https://doi.org/10.1111/j.1745-3984.1984.tb01030.x Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational assessments. Measurement: Interdisciplinary research and perspectives, 1(1), 3−62. Morales-Martinez, G. E. (2015). Protocolo para la recolección de conceptos objetivo y definidores centrales y diferidos (PRECODECD): Un sistema de codificación de conceptos extraídos de las redes semánticas naturales [Protocol for the collection of objective concepts and central and deferred definers (PRECODECD): A coding system for concepts extracted from natural semantic networks] (Unpublished manuscript). Institute of Research on the University and Education, National Autonomous University of Mexico, Mexico. Morales-Martínez G. E. (2020). Sistema de evaluación cognitiva constructiva cronométrica del aprendizaje en línea y presencial [Online and face-to-face learning’s constructive- chronometric cognitive assessment system] (Manuscript submitted for publication). Institute of Research on the University and Education, National Autonomous University of Mexico, Mexico.
  • 27. 21 http://ijlter.org/index.php/ijlter Morales-Martinez, G., & Lopez-Ramirez, E. (2016). Cognitive responsive e-assessment of constructive e-learning. Journal of e-Learning and Knowledge Society (Je-LKS), 12(4), 39−49. http://www.je-lks.org/ojs/index.php/Je-LKS_EN/article/view/1187 Morales-Martinez, G. E., Lopez-Ramirez, E. O., Castro-Campos, C., Villarreal-Treviño, M. G., & Gonzales-Trujillo, C. J. (2017). Cognitive analysis of meaning and acquired mental representations as an alternative measurement method technique to innovate e-assessment. European Journal of Educational Research, 6(4), 455−464. https://www.eu-jer.com/EU-JER_6_4_455_Morales-Martinez_etal.pdf Morales-Martínez, G. E., Lopez-Perez, R. M., Garcia-Collantes, A., & López-Ramírez, E. O. (2020). Evaluación constructiva cronométrica para evaluar el aprendizaje en línea y presencial [Chronometric constructive assessment to assess online and face-to-face learning]. Tecnología, Ciencia y Educación, 15(1), 105−124. https://www.tecnologia-ciencia educacion.com/index.php/TCE/article/view/371 Morales-Martinez, G. E., Lopez-Ramirez, E. O., & Lopez-Gonzalez, A. E. (2015). New approaches to e-cognitive assessment of e-learning. International Journal for e- Learning Security (IjeLS), 5(2), 449−453. https://doi.org/10.20533/ijels.2046.4568.2015.0057 Morales-Martinez, G. E., Ángeles-Castellanos, A. M., Ibarra-Ramírez, V. H., & Mancera- Rangel, M. I. (2020). Cognitive e-tools for diagnosing the state of medical knowledge in students enrolled for a second time in an anatomy course. International Journal of Learning, Teaching and Educational Research, 19(9), 341−362. https://doi.org/10.26803/ijlter.19.9.18 Morales-Martinez, G. E., Mezquita-Hoyos, Y. N., Gonzalez-Trujillo, C. J., Lopez-Ramirez, E. O., & Garcia-Duran, P. J. (2018). Formative e-assessment of schema acquisition in the human lexicon as a tool in adaptive online instruction. In R. Lopez-Ruiz (Ed.), From natural to artificial intelligence: Algorithms and application (pp. 69−88). IntechOpen. http://doi.org/10.5772/intechopen.81623 Muskin, J. A. (2015). Student learning assessment and the curriculum: Issues and implications for policy, design and implementation (Current and critical issues in the curriculum and learning). UNESCO International Bureau of Education. http://www.ibe.unesco.org/sites/default/files/resources/ipr1-muskin- assessmentcurriculum_eng.pdf Rumelhart, D., Smolensky, P., McClelland, J., & Hinton, G. (1986). Schemata and sequential thought processes in PDP models. In J. McClelland, D. Rumelhart, & the PDP Research Group (Eds.), Parallel distributed processing: Explorations in the microstructure of cognition: Vol. 2. Psychological and biological models (pp. 7−57). MIT Press. Sadeghi, K., & Rahmati. T. (2017). Integrating assessment as, for, and of learning in a large- scale exam preparation course. Assessing Writing, 34, 50−61. https://doi.org/10.1016/j.asw.2017.09.003 Urdiales-Ibarra, M. E., Lopez-Ramirez, E. O., Castro-Campos, C., Villarreal-Treviño, M. G., & Carrillo-Colon, J. E. (2018). Biology schemata knowledge organization and meaning formation due to learning: A constructive-chronometric approach to concept mapping usability. Creative Education, 9(16), 2693−2706. https://doi.org/10.4236/ce.2018.916203 Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3–14. https://doi.org/doi:10.1016/j.stueduc.2011.03.001
  • 28. 22 ©Authors This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License (CC BY-NC-ND 4.0). International Journal of Learning, Teaching and Educational Research Vol. 20, No. 2, pp. 22-35, February 2021 https://doi.org/10.26803/ijlter.20.2.2 Portfolio as a Tool for Self- Reflection and Professional Development for Pre-Service Teachers Arlinda Beka Faculty of Education, University of Prishtina ‘Hasan Prishtina’, Kosovo https://orcid.org/0000-0002-9871-783X Ganimete Kulinxha Faculty of Education, University of Prishtina ‘Hasan Prishtina’, Kosovo https://orcid.org/0000-0003-4461-8841 Abstract. Self-reflections, especially those done through the usage of portfolios, are an essential component of teaching. Portfolio reflections allow pre-service teachers to set professional goals. This research was conducted using qualitative methodology, and all semi-structured interviews were organised with students and graduates from the Early Childhood Department-Faculty of Education, University of Prishtina. The research results show that a portfolio provides future teachers with the opportunity to organise their work better while enabling cooperation between pre-service teachers, which ergo, directly affects their professional competencies. Owing to its benefits, portfolio should be integrated by the Faculty of Education of the University of Prishtina as an integral part of its work with students. Keywords: portfolio; pre-service preschool teachers; teacher professional development 1. Introduction Self-reflection is one of the most important competencies that a teacher should possess. Reflection directly enables the identification of things that not only are a strong part of professional work but that also need to be advanced and to allow a higher level of professionalism. In terms of taking individual responsibility for thought and action processes, self-reflection is a required condition for assessment and regulation (Dilger, 2007). After completing a given task, one tries to reflect on the process and the work accomplished; however, sometimes one’s attention is drawn to the sections that have been the most successful or most challenging, leaving out many other aspects that may be equally important for enhancing the quality of one’s work and for continuing professional development. Portfolios are a proper way of recording our accomplished work and the challenges and successes encountered during the process. The record of such data
  • 29. 23 http://ijlter.org/index.php/ijlter allows one to go back in time to see what one went through, how one did one’s activities, how one conceived things or what challenges were faced during a given period. The portfolio technique is viewed as particularly huge with regard to self- intelligent learning. The technique forms a significant part in the advancement of polished skill through reflection and self-reflection (Riebenbauer, Dreisiebner & Stock, 2017). Through the use of portfolios, pre-service teachers can succeed in developing a professional working plan, reflect on their work, develop additional skills in the use of technology for professional development and have a clear review of their work (Beka & Gllareva, 2016). Portfolios for self-reflection in pre-service teacher training provide vital input for trainers since they represent both the strengths and weaknesses of students (Cimermanová, 2019). Portfolios play a significant role in not only the process of preparing teachers for employment but also in their licensing as teachers and coaches and their state- level performance appraisals. Being reflective is a key aspect of any educational experience. It involves the continuous educational growth of both professional educators and students (Webster & Whelen, 2019). However, portfolio is often regarded as a collection of facts and documents that testify teachers’ professional development and achievement. In some cases, it is considered as an important part where the teacher must present the work planned for certain periods, including the reflection on commitments which he/she has accomplished. Therefore, the teachers make the portfolio in terms of the purpose that it will serve them. Similarly, during pre-service studies, future teachers are sometimes required to have more reflections on their portfolio. In some cases, their portfolio is also required to include facts, arguments and goals that students aim to achieve during their studies (Stock & Winkelbauer, 2012). 2. Development of portfolios and the arguments The use of portfolios in pre-service teachers’ professional development and professional assessment has become widespread around the world. As a result, several contemporary studies and research projects are being conducted by various scholars and educational scientists regarding the importance of portfolios, their meaning and their relevance to teachers’ professional development. It is often thought that portfolio has begun to be affirmed in the last two decades as a contemporary approach to professional development. Portfolio, as a concept in education, has been developed in the last decades of XX century in the United States of America and the German-speaking countries (Slepcevic-Zach & Stock, 2018). For a clear understanding of the concept of portfolio usage among teachers whether in pre-service or in-service, we need to determine the purpose of the portfolio and the way in which it acts as a professional aid to the teacher. According to various authors, there is a discrepancy between the portfolio and the way it should be filled in, i.e., whether the portfolio will focus more on reflection or on the documentation of professional work.
  • 30. 24 http://ijlter.org/index.php/ijlter In his paper, ‘Do Portfolios Have a Future?’, Driessen (2017) divides portfolio into two categories, namely reflective portfolios and comprehensive portfolios. Reflective portfolio is primarily intended to help students improve their reflective abilities. It requires students to write a reflection on an aspect of their learning or career that is significant to them. Reflections can be found in comprehensive portfolios, but their material is much more varied than that of reflective portfolios (Driessen, 2017). Baumgartner (2009) classifies portfolio into the three following types depending on their purpose: reflection portfolio, development portfolio and presentation portfolio.The e-portfolio is a flexible tool that can be used for several purposes, including learning, evaluation/assessment and employability/career growth (Ciesielkiewicz, 2019). Trávníčková and Puhrová (2020) state that portfolios can be encountered in the following terms: ‘professional teaching portfolio’, ‘professional development portfolio’, ‘professional practice portfolio’ and even the relatively new term of ‘eportfolio’. All the above types of classifications from different authors can be considered as being similar due to their common elements. The reflective portfolio, which is considered a crucial part of teachers’ professional development, is common in the classifications. The aim of comprehensive portfolios is to assist students in their learning and to evaluate their progress. Reflections can be found in comprehensive portfolios, but their material is much more varied than that of reflective portfolios (Driessen, 2017). Since education methods, such as reporting, contemplation and representation of one’s learning activity, were major issues in the 1980s, the portfolio idea might have been a pillar in the German debate about educational reform (Häcker, 2009). Like in Germany, in almost all European countries and beyond, the portfolio is beginning to be considered a significant part of pre-service and pre-service teacher work. Nowadays, portfolios are seen as “rich, contextual, highly personalised documentaries of one’s learning journey.” (Jones & Shelton, 2011, p.21). There are numerous reasons why a portfolio is considered an important tool for teacher development and professional development. Portfolios are considered beneficial for pre-service and in-service teachers “as they promote autonomous learning and autonomous learners are often motivated and reflective learners.” (Fernández, 2014). In almost all professions, especially the teaching profession, reflection is a crucial part. According to Lynch (2000), reflection is considered to be an important part of a person’s skill set. Through reflection, each teacher undergoes a long and complicated process to reach his/her desired stage of professional development. Within the scope of the portfolio-work, not only the result (item) is critical but mostly the method that led to this improvement (Kerr, 2007). Therefore, the documentation of professional experiences through the use of a portfolio is an asset for every pre-service teacher. Usually, the portfolio’s documentation is structured and deliberately designed as “they contain purposefully organised documentation that demonstrates specific knowledge, skills, dispositions, and accomplishments achieved
  • 31. 25 http://ijlter.org/index.php/ijlter over time. Portfolios represent connections made between actions and beliefs, thinking and doing, and evidence and criteria. They are a medium for reflection through which the builder constructs meaning, makes the learning process transparent and learning visible, crystallises insights, and anticipates future direction.” (Jones & Shelton, 2011, p.22) Despite the different definitions of portfolio which depend on their purpose, objective and nature, certain theories consider portfolio as a personal ownership, personal record structured collection of material and so on. According to Paris and Ayres (1994), “The overarching purpose of portfolios is to create a sense of personal ownership over one’s accomplishments because ownerships engender feelings of pride responsibility and dedication (p.10).” McKimm (2014) defined teaching portfolios in her book ‘A Handbook for Teaching & Learning in Higher Education’ as “A teaching portfolio (sometimes referred to in this chapter as a ‘portfolio’) is a personal record of achievement and professional development as a university teacher. It is a carefully selected and structured collection of material that may demonstrate a level of attainment, a range of skills and activity, and/or progression.(p.473)” Kayler (2004) states that: “Teacher education institutions need to make the often-invisible professional growth of classroom teachers more visible to other teachers and programs as a way to foster and expand the teacher knowledge base (p.267).” According to Fernández (2014), portfolios play a significant role in stimulating and inspiring teachers as well as students. Learning becomes more effective and motivating with a portfolio since the students take ownership of their learning process and develop strategies to plan, monitor and evaluate their learning. Teachers benefit from portfolios in their professional development since portfolios promote autonomous learning, and autonomous learners are often motivated (Fernández, 2014). Portfolios can also be considered as a source of data, whether qualitative or statistical, and are an important tool for evaluation. In terms of assessment, Belgrad, Burke and Fogarty (2008) consider that portfolios (and e-portfolios) can become excellent tools for both, formative and summative assessments. Martin-Kniep (1999) considers portfolios as an instrument that recounts the account of the educator whose objective is to make guidance and evaluation rehearses something very similar and focused on critical learning results. The portfolio uncovers the educator’s extraordinary capacity to apply educational plan and appraisal-related ideas and abilities to the advancement of exercises and evaluations that target obviously characterised results. The instructor has provided all drafts of appraisals, enabling the user to notice the changes and upgrades made to apply plan standards and better adjust educational program, guidance and evaluation. The teachers can utilise their portfolios to report their expert history, the inquiries and worries that drive their
  • 32. 26 http://ijlter.org/index.php/ijlter work, their endeavours to plan or carry out better or diverse learning encounters for understudies, their investigation of basic issues with at least one understudy or the settings that encompass them, their expert work with partners, local area individuals or bosses and their examination of and reflection on proficient assets or critical writings. Notwithstanding the specialised parts of portfolios, it should be remembered that the “essential crowd for an educator portfolio is the instructor who readies the portfolio. Simultaneously, the portfolio can help instructors screen the change from the exercise as-wanted to the exercises educated, to the exercise as-learned. At last, portfolios can help screen educators’ endeavours to fuse new information and abilities into their expert collections. Albeit quite a bit of this work is a private undertaking, educators, as different experts, need to take part in a discussion with their partners to really comprehend and approve their work. Portfolios can fill in as a springboard for such conversations.” (Martin-Kniep, 1999, p.15) This is very well explained in ‘The Portfolio Connection: Student Work Linked to Standards’, in which Belgrad et al. (2008) states that: “Portfolios as the outcome of a continuous, student-centred process help students learn in a variety of ways. One remarkable by-product of the portfolio process in the classroom is increased student awareness of and responsibility for learning objective (p.15).” The portfolio presents pre-service teacher’s information that is directly related to the personality, mindset, worldviews, mission and vision approaches that pre- service and in-service teachers create for themselves, whether on a personal or professional level. Portfolio creation represents the nature of the individual’s personality, his/her level of knowledge and the goals that he/she wishes to achieve in his/her professional field. This approach will help teachers in their awareness of professional development. They will manage to present the goals for their work and the path that they will follow. Moreover, portfolios will act as an open statement that they will follow to be more successful. In this way, they manage to develop critical thinking about themselves and their profession and begin to be more creative regarding their plan to achieve their professional goals. They also become aware of a sustainable professional investment that will serve their work in the long term. The portfolio also presents the skills that pre-service teachers have built up during their studies and becomes a bridge between them and the work that they will aspire to have, which enables teachers to have personal and professional competence. Through reflective learning, pre-service teachers can learn from their previous experiences either in cognitive or practical terms, enabling them to make decisions for change and professional development. Pre-service teachers can also share information with their peers and obtain feedback from their colleagues, thus having an excellent opportunity to be open to others, share ideas and collaborate in the professional field.
  • 33. 27 http://ijlter.org/index.php/ijlter Creating a portfolio during the study period enables pre-service teachers to establish a culture of presenting ideas, approaches and understandings in the personal and professional field. By establishing strategies for presenting their work in a broader professional environment, they continue to present their work in the same way when they look for a job or even after they are hired. For teachers in the portfolio, a service is a great tool that can serve them for the evaluation performed either by their institution or even at a wider level. Therefore, in terms of teacher professional development, regardless of in-service or pre-service teacher training, portfolio is a considerable tool for documentation, assessment and evaluation of instructors’ work. They can be pre-designed and pre-structured to collect and document information as required for a specific task or profession, i.e., pre-service or in-service teaching. Portfolios enable teachers to document the evolution of different areas of their work, such as curriculum and assessment development, professional development, inquiry/research and teaching and learning in general. When used by students, portfolios are developed to display materials that represent the abilities and accomplishments during a specified period. According to Henderson, Davis & Day (2015), students’ presentation of materials over an extended time helps to chronicle their improvement and competence that cannot be measured by paper and pencil tests. One of the primary purposes of a portfolio is to provide a more reliable and comprehensive picture of student growth and achievement. By using pre-service teachers’ portfolios, instructors will have better opportunities for their personal and professional development and will create a professional culture for their work with children. In Kosovo, as in several other nations, one of the vital competencies of instructors is to reflect on the work they do, to archive their completed exercises and to show the goals they wish to attain in a given period. Typically, the employment of a portfolio plays a critical role in teacher instruction in Kosovo or any other nations because it makes a difference in creating the key competencies for educator teachers and pre-service instructors (Beka, 2015). According to the implemented laws and educational policies, all the pre-service and in-service teacher trainings in Kosovo are organised by public universities, particularly by their faculties of education (Ministry of Education, Science and Technology [MEST], 2011). The Faculty of Education of the University of Prishtina, being the oldest one, carries out the most significant role in teacher training. It provides pre-service and in-service teacher programs from undergraduate to doctoral level (University of Prishtina, 2004). As part of the Strategic Development Plan of the Faculty of Education, improvement of quality teaching programs and teacher professional development are considered crucial factors for enhancing our education system (Faculty of Education, 2013). Professional portfolios of pre-service teacher are becoming now an integrated part of the teaching curriculum in all of its study programs. The Faculty of Education has had in the past few years several in-service training
  • 34. 28 http://ijlter.org/index.php/ijlter programs for teachers’ professional development besides the regular study programs. Those programs have been supported by the MEST and different donors-education partners aiming to advance teachers’ professional development. Examples of these programs are: Advancement of Qualifications of Teachers Project, Capacity Building for Teacher Mentors, Re-Qualification of Pedagogical Student Graduates, and so on. So far, in- service training programs have not been part of the systematic policy of Teacher Professional Development (TPD). Rather, they were a need for the re-qualification of teachers in their teaching degree or in the new teaching strategies (MEST, 2015). With the new strategic documents in place, in-service training is expected to play a vital role in TPD, leading to teachers’ career advancement (MEST, 2016). A professional portfolio is a required tool from the pre-service studies as it will present ideas, professional goals and the ways of achieving these goals in a period of time. It will help teachers to self-reflect on what they have learned and how to improve their professional practice. Reflecting on their accomplished work is significant for the role and the well-being of the activities that they should perform on campus and in internships. However, not all students find the portfolio relevant to their professional development. Junior students treat it more as a task that they should complete during their studies, whereas senior students understand the importance of the portfolio and view it as a tool that will help them to enhance the quality of their work and to promote continuous professional development. At the same time, senior students consider the portfolio as a way to achieve the goals that they set for professional development. 3. Research methodology This research paper presents the current state of usage of portfolio as a tool for teachers’ professional development in pre-service teacher training at the Faculty of Education of the University of Prishtina. The research was conducted using qualitative methodology. Interviews were organised with students and graduates of the Early Childhood Department-Faculty of Education, University of Prishtina. Students were randomly selected; every fifth student was selected for each year, while alumni were purposely selected due to their easier access and the possibility of their inclusion in this research. Five students were selected for each year of studies, starting with the second year of studies, third and fourth, as well as five graduates. A total of 20 respondents were included in this survey. The interviews were organised with each student and graduate separately. Each of the interviews lasted approximately 45-50 min. Students and graduates have voluntarily participated in the interviews. The difference between sophomores and third- and fourth-year students was that sophomores felt more limited in giving answers due to their beliefs about the portfolio and its relevance – reflection, while third- and fourth-year students felt more confident in giving answers since they already had a clear portfolio effect on their professional development as it was their reflection on the quality of the work they did. The graduates were clear about the importance of reflection through the portfolio. With the permission of students and graduates, all interviews were recorded and transcribed. The final transcription report was provided to the participants for verification and signature. By signing these reports, they have confirmed that
  • 35. 29 http://ijlter.org/index.php/ijlter their answers were correct and the meaning they wanted to give has not changed through these statements. The research was conducted to answer the following research questions: 1. ‘How will a portfolio help pre-service teachers in their professional development’? 2. ‘How do pre-service teachers understand the importance of a portfolio at different stages of studies’? 3. ‘What impact does pre-service teacher alumni portfolio have on instructors’ employment’? 4. ‘In what way does self-reflection orient pre-service teachers towards motivation for professional development’? 5. ‘How does self-reflection affect in-service teachers for continuous professional development’? 6. ‘How appropriate is the portfolio as a tool to develop self-reflection in pre- service teachers’? The questions asked in the interview were semi-structured and intended to obtain the students’ opinions on the following: • The importance of portfolio preparation and reflection for pre-service preschool teachers; • The benefits of portfolio preparation; • The portfolio’s significance for professional development; and • The portfolio’s importance in employment. 4. Research outcomes 4.1 The importance of portfolio preparation and reflection for pre-service preschool teachers Based on interviews regarding students’ opinions on the preparation of portfolios, it is evident that learners had very different perceptions. Since the concept of portfolio creation is perceived differently by students, varying in various levels of study, second-year students are found to be more inclined to viewing it as a chore that should be checked off of their list of obligations; “I spend a lot of time preparing my portfolio…it takes hard work.” Nevertheless, third-year students seem to understand its importance in their professional upgrading. They begin to consider portfolios as a useful tool which they will continue to use at their jobs even after graduation. A third-year student stated that: “In the beginning, I thought it was not important but now it seems to be very helpful.” By contrast, fourth-year students or alumni do not view portfolios as a task but instead as a tool for summarising and evidencing works and professional ideas. Similar to the previous statement, a fourth-year student declared that: “By using the portfolio, I know what I’m doing…selection of materials is important for ongoing development.”
  • 36. 30 http://ijlter.org/index.php/ijlter Meanwhile, the alumni expressed that: “The portfolio is a necessary tool as it helps to reflect on professional development.” Based on these answers, it can be concluded that the portfolio is seen as an exceedingly important tool, necessary for the ongoing professional development of graduates who have entered the teaching profession. 4.2 The benefits of portfolio preparation There is a discrepancy between second-year students and other years students’ opinions regarding the benefits of preparing a portfolio. A second-year student stated that: “Portfolio helps me to get a good grade.” Meanwhile, third-year students stated that: “The portfolio helps me recognise my weaknesses…” This attitude is directly related to the reflection of the pre-service teacher related to his/her work. Through portfolios, students manage to self-evaluate their work and professional development. On the other hand, a fourth-year student notes the portfolio’s importance as an opportunity for the development of critical and creative thinking. In this regard, she declares that: “The portfolio helped me to be a critical thinker in my work and professional development. By using the portfolio, I can easily identify my strengths and my weaknesses.” Moreover, an alumni student considers portfolios as a tool that helps her to focus on completing her objectives. In this regard, she stated: “The portfolio has helped me become more efficient, improve my quality of work in planning and my cooperation with colleagues.” Based on the statements provided by the students, it has been concluded that second-year students consider the portfolio as a means to achieve the highest grades, while third-year, fourth-year and alumni students view it as an essential implement for professional development. Their opinions have developed precisely from their experiences with portfolio preparation. Self-reflection on the professional development of pre-service teachers is more witnessed among the fourth-year students than the students of previous years. They have already realised the importance of reflecting on their work, and through this, they begin to make ongoing plans for their professional development. The reflection itself is even more emphasized among the graduates. Through statements, they have emphasized that through this form, they become more efficient and focus to a greater extent on the quality of their work. Reflections has allowed them to improve their professional practices and knowledge, thus making them better prepared teachers. 4.3 The portfolio’s significance for professional development Students have expressed the significant contribution of portfolios to their professional development as well as their reasoning behind it. Even in the case of
  • 37. 31 http://ijlter.org/index.php/ijlter second-year students, it is evident that their lack of experience makes them more inclined to consider portfolio as a task through which they can learn. At this level, they have not yet fully established the importance of portfolios in their personal and professional development. Therefore, by continuing to think that the employment of portfolios enables them to learn new things and compare their work with others, the portfolio becomes more appealing. A second-year student declared: “I do research so that I can have a good portfolio, which in return will help me improve my work. The portfolio also helps me to compare my work with others.” Meanwhile, the opinions of third-year students begin to present a higher level of advancement regarding the importance of portfolios in their professional development. Hence, third-year students are more inclined to consider it as an encouraging tool for cooperation with others and as something that helps them feel secure. In this regard, a third-year student stated: “The portfolio has helped me to cooperate with others. Cooperation is a job-requested skill…” In contrast to the second and third-year students, the opinions of the fourth-year students hold the portfolio as a source of new ideas, an instrument that provides them with security, and more importantly, an opportunity for self-evaluation. A fourth-year student declared: “The preparation of portfolios has given me more confidence and I do better quality work. I am more confident in what I do” This allows for a better understanding of the fourth-year students’ view because through their experience, whether they have been taught on campus or internships, they have found the portfolio to have a significant impact on their preparation for the teaching profession. Therefore, their opinions at this stage are also more constructive and based on the achieved results. Meanwhile, graduates see the importance of the portfolio as an evidence of their qualitative work with which they can secure their job more easily, which encourages them to apply this kind of assessment with the children involved in their work. In this regard, an alumni declared: “The portfolio is the summary of my work. Additionally, the portfolio has enabled me to get a job at a private preschool.” Based on the students and alumni’s answers, we can conclude that portfolios are a critical tool in the professional development of pre-service teachers. By using portfolios, pre-service teachers understand goals and ways to achieve those goals. The portfolio is a testimony of the acquired knowledge and skills of pre-service teachers, which consequently enables them to focus on their professional development. 4.4 The portfolio’s importance in employment Due to the portfolio being one of the most important tools for their professional development, students have shared their opinions on the importance of the portfolio in their employment phase. This question has been answered by third-
  • 38. 32 http://ijlter.org/index.php/ijlter year students, fourth-year students and alumni since their chances of obtaining a job are higher than those of second-year students. The students have described their opinions about the professional preparation that they have achieved through the development of portfolios. They explained how much the preparation of portfolios has helped or will help them to find employment in their professional field. Third-year students have said that the portfolio has helped them to love their future profession and at the same time has helped them to develop presentational skills. As a result, they can better present their work, and according to them, these are the benefits that will give them advantage over others. One of the opinions of the third-year student in this regard was: “The portfolio has helped me to further develop my presentational skills as I had to present my work. It has helped me to love my future profession more…” A fourth-year student stated that throughout the portfolio preparation process, they are taught to organise and document their work, which is of great advantage to all those who work. The student also said that developing portfolios requires creativity and the use of new ideas and different materials, especially those that are recycled so that the portfolio is more meaningfully presented to others. “The portfolio helped me understand that it is necessary to be organised at work… Portfolio preparation pushed me to be more creative and to use recycled and reusable materials.” Given the above-mentioned characteristics, students are convinced that they will find jobs more easily since those aspects are required in the first interview. Graduates have stated that during their studies, in certain cases, they did not consider portfolio as useful work. However, when they applied for work, they realised that a portfolio is an extraordinary tool used to present their professional preparation: “In the beginning, I thought I am doing work that nobody appreciates, but now I see that the portfolio is an amazing testimony. Each time I look at my portfolio, I feel like a teacher, and I will create a portfolio for my pre- school students.” Based on the interviewers’ opinions, we can conclude that the portfolio also plays an important role in pre-service teacher employment. Through portfolios, pre- service teachers can easily prove their professional development and present their work to children. It is of utmost importance that they apply the same kind of assessment to pre-school children; the application of formative assessments and the documenting of the level of childhood achievement. Furthermore, they are able to document the ideas, approaches and goals related to their teaching profession. 5. Conclusions According to the research data, portfolio is a siginificant factor in educators’ professional development. This instrument provides future teachers with the opportunity to learn to self-reflect and to simultaneously organise their work better. To create skills in view of collaborating with others directly and to reflect