IJLTER.ORG Vol 20 No 2 February 2021

International Journal
of
Learning, Teaching
And
Educational Research
p-ISSN:
1694-2493
e-ISSN:
1694-2116
IJLTER.ORG
Vol.20 No.2

International Journal of Learning, Teaching and Educational Research
(IJLTER)
Vol. 20, No. 2 (February 2021)
Print version: 1694-2493
Online version: 1694-2116
IJLTER
International Journal of Learning, Teaching and Educational Research (IJLTER)
Vol. 20, No. 2
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically those of translation, reprinting, re-use of illustrations,
broadcasting, reproduction by photocopying machines or similar means, and storage in data banks.
Society for Research and Knowledge Management

The International Journal of Learning, Teaching and Educational
Research is a peer-reviewed open-access journal which has been
established for the dissemination of state-of-the-art knowledge in the
fields of learning, teaching and educational research.
Aims and Objectives
The main objective of this journal is to provide a platform for educators,
teachers, trainers, academicians, scientists and researchers from over the
world to present the results of their research activities in the following
fields: innovative methodologies in learning, teaching and assessment;
multimedia in digital learning; e-learning; m-learning; e-education;
knowledge management; infrastructure support for online learning;
virtual learning environments; open education; ICT and education;
digital classrooms; blended learning; social networks and education; e-
tutoring: learning management systems; educational portals, classroom
management issues, educational case studies, etc.
Indexing and Abstracting
Research is indexed in Scopus since 2018. The Journal is also indexed in
Google Scholar and CNKI. All articles published in IJLTER are assigned
a unique DOI number.

Foreword
We are very happy to publish this issue of the International Journal of
Learning, Teaching and Educational Research.
Research is a peer-reviewed open-access journal committed to
publishing high-quality articles in the field of education. Submissions
may include full-length articles, case studies and innovative solutions to
problems faced by students, educators and directors of educational
organisations. To learn more about this journal, please visit the website
http://www.ijlter.org.
We are grateful to the editor-in-chief, members of the Editorial Board
and the reviewers for accepting only high quality articles in this issue.
We seize this opportunity to thank them for their great collaboration.
The Editorial Board is composed of renowned people from across the
world. Each paper is reviewed by at least two blind reviewers.
We will endeavour to ensure the reputation and quality of this journal
with this issue.
Editors of the February 2021 Issue

VOLUME 20 NUMBER 2 February 2021
Table of Contents
Chronometric Constructive Cognitive Learning Evaluation Model: Measuring the Construction of the Human
Cognition Schema of Psychology Students .........................................................................................................................1
Guadalupe Elizabeth Morales-Martinez, Janneth Trejo-Quintana, David Jose Charles-Cavazos, Yanko Norberto Mezquita-
Hoyos, Miriam Sanchez-Monroy
Portfolio as a Tool for Self- Reflection and Professional Development for Pre-Service Teachers.............................. 22
Arlinda Beka, Ganimete Kulinxha
The Efficacy of Habits of Mind in the Inculcation of Self-Directed Learning Skills in Pre-Service Teachers ........... 36
Gideon Petrus Van Tonder, Byron John Bunt, Alvine Petzer, Hermann Davin Bosch, Nicolaas Van Deventer, Amanda
Gerber, Lizelle Van Schadewijk
The Impact of Professional Development, Modern Technologies on Lecturers’ Self-Efficacy: Implication for
Sustainable Science Education in Developing Nations....................................................................................................61
David Agwu Udu, Benson Ikechukwu Igboanugo, John Nmadu, Chidebe Chijioke Uwaleke, Benjamin Chukwunonso
Okechineke, Adaora Phina Anudu, Precious Chisom Attamah, David Onyemaechi Ekeh, Mercy Ifunanya Ani
A Competence Model to Assess and Develop Designing Competence Assessment Tool .......................................... 81
Do Tra Huong, Nguyen Thi Dieu Linh
Cultural Language Learning Approach (CLLA): An Approach for Learners’ Integrated Language Use
Acculturation....................................................................................................................................................................... 104
Hermayawati .
The Impact of Using VAKT Strategy on Oral Reading and Reading Comprehension Skills of Elementary Students
with Dyslexia....................................................................................................................................................................... 121
Ayed H. Ziadat
The Influence of Architecture Students’ Learning Approaches on their Academic Performance in Two Nigeria
Universities.......................................................................................................................................................................... 137
Gabriel Sen, Albert Adeboye, Oluwoye Alagbe
Blended Learning in Rural Primary ESL Classroom: Do or Don’t............................................................................... 152
Melanie Jerry, Melor Md Yunus
Developing Interaction in ESL Classes: An Investigation of Teacher-Student Interaction of Teacher Trainees in a
Sri Lankan University......................................................................................................................................................... 174
Abdul Majeed Mohamed Navaz
Teaching with Relevance: Saudi Students’ Perceptions of a Foundation Course in Communication Skills .......... 197
Philline M. Deraney
Students’ Soft Skills, Hard Skills, and Competitiveness (SHC): A Suggested Model for Indonesian Higher
Education Curriculum........................................................................................................................................................ 218

Hadiyanto ., Noferdiman ., Syamsurizal ., Muhaimin ., Ina Krisantia
Decision-making by Heads of Academic Department using Student Evaluation of Instruction (SEI) ................... 235
Mohammed Saleh Alkathiri
Fostering Media Literacy Skills in the EFL Virtual Classroom: A Case Study in the COVID-19 Lockdown Period
............................................................................................................................................................................................... 251
Marina Bilotserkovets, Tatiana Fomenko, Oksana Gubina, Tetiana Klochkova, Oksana Lytvynko, Maryna Boichenko, Olena
Lazareva
Analysis of Future Teachers’ Perceptions on the Evaluation of Learning: A Case Study of Chile .......................... 270
José Hernández Sepúlveda, Rodrigo Panes Chavarría, Karla Rosalia Morales Mendoza
Career Readiness of Graduating Office Technology and Management Students in Polytechnics in Southeast
Nigeria.................................................................................................................................................................................. 290
Lynda Chineze Ezechukwu, Oliver Okechukwu Okanazu, Muhammadu Madu-Saba Babalulu, Ayoola Abimbola Arowolo,
Chijioke Jonathan Olelewe
Information Technology Integration Perception on Ghanaian Distance Higher Education: A Comparative
Analysis................................................................................................................................................................................ 304
Albert Arthur Qua-Enoo, Brandford Bervell, Paul Nyagorm, Valentina Arkorful, John K. E. Edumadze
ESL Teachers’ Challenges in Implementing E-learning during COVID-19 ................................................................ 330
Brenda Anak Lukas, Melor Md Yunus

1
©Authors
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
International License (CC BY-NC-ND 4.0).
Vol. 20, No. 2, pp. 1-21, February 2021
https://doi.org/10.26803/ijlter.20.2.1
Chronometric Constructive Cognitive Learning
Evaluation Model: Measuring the Construction of the
Human Cognition Schema of Psychology Students
Guadalupe Elizabeth Morales-Martinez and Janneth Trejo-Quintana
Cognitive Science Laboratory, IISUE,
National Autonomous University of Mexico, Mexico City, Mexico
https://orcid.org/0000-0002-4662-229X
https://orcid.org/0000-0002-7701-6938
David Jose Charles-Cavazos
TecMilenio University, Mexico City, Mexico
https://orcid.org/0000-0002-3445-9026
Yanko Norberto Mezquita-Hoyos
Autonomous University of Yucatán, Yucatan, Mexico
https://orcid.org/0000-0001-6305-7440
Miriam Sanchez-Monroy
Tecnologico Nacional de Mexico-Instituto Tecnologico de Merida, Yucatan, Mexico
https://orcid.org/0000-0001-5263-1216
Abstract. This study measured the structural and organizational changes
in the knowledge schema of human cognition in response to the learning
achieved by 48 students enrolled in the second year of a psychology
degree. Two studies were carried out based on the Chronometric
Constructive Cognitive Learning Evaluation Model. This article deals
only with the first one, which consisted of a conceptual definition task
designed in line with the Natural Semantic Network technique.
Participants defined ten target concepts with verbs, nouns, or adjectives
(definers), and then weighed the grade of the semantic relationship
between the definers and the target concepts. The data indicate that the
initial knowledge structures had been modified towards the end of the
course. The participants’ human cognition schema presented changes in
terms of content, organization, and structure. This evidence supports the
idea that the acquisition and transformation of the schemata learned in
academic environments may be observed through cognitive science
indicators.
Keywords: cognitive evaluation; knowledge schema; learning; NSN;
psychology students

2
http://ijlter.org/index.php/ijlter
1. Introduction
Assessing academic learning is one of the most significant challenges for
educators in the twenty-first century. This is supported by William (2011), who
asserted that assessing learning is a central activity in the instruction-learning
process. There is a great diversity of learning measurement tools, especially given
the development of new technology, which has opened up new possibilities in
this field. However, there is still no consensus on the most convenient way to
assess student learning. This problem means that although there is a diversity of
tools with which to measure academic learning, there is no agreement about the
best way to determine what and how much content a student has learned during
a course. El-Yassin (2015) remarked that there is no right or wrong way to evaluate
student learning since each instrument can inspect a specific learning aspect. In
addition, William (2011) pointed out that although the sequence of presentation,
quality, and even teaching in a class is the same for all students, they understand
what they learn in the classroom differently and may even learn different things
to what they are taught.
This variability in students’ academic learning has long been considered a barrier
to teaching rather than a source of enrichment within the classroom. In this regard,
William (2011) discussed how for many years, those involved in the educational
field assumed that the quality of instruction alone would be enough for students
to learn, and failure to learn in spite of effective instruction was attributed to the
students’ cognitive characteristics. Currently, the educational community is
beginning to raise awareness about the role played by an individual student’s
needs and cognitive characteristics in the design of teaching-learning sequences.
Regarding the above, in the 1980s, Messick (1984) stressed that the interpretation
of achievement measures should be carried out in the context of the style of
instruction and learning to reduce errors in the interpretation of academic
performance and students’ functioning within specific learning environments.
Although this proposal sounds obvious, Messick explained that measuring
learning in such an all-encompassing way is rarely feasible due to the complexity
of the information that needs to be extracted at different levels of student life.
In general, learning assessment can be very complex due to the broad spectrum
of factors involved. According to Muskin (2015), the evaluation of learning
implies using a means to determine what a person knows in conceptual or
procedural terms. In this regard, Messick (1984) pointed out that school learning
not only involves the content that a student can store in their memory, but also
how the student structures or restructures their knowledge and cognitive skills
according to their level of academic development (beginner, intermediate, or
advanced).
Messick (1984) suggested that any learning measurement should take account of
the state of academic development of each student to establish the cognitive
functioning level at which the learning assessment will be carried out. For
example, Messick proposed that with students in an initial learning phase, the
objective should be to acquire information. At this level, information-retrieval

3
recognition assessments could be used. In contrast, at a more advanced level,
student learning should manifest itself in the restructuring of schemata and the
flexible use of schemata to solve problems. However, Messick saw the application
of such a proposal as very forward-looking rather than being based on the reality
of developing performance tests.
Currently, most evaluation instruments are focused on performance
measurement. In this regard, Banister (2004) pointed out that in psychology, the
most commonly used instruments to measure learning are exams, practical tests,
and empirical dissertations. These kinds of tools are used as summative
assessments of student performance. The tests provide valuable information
about aspects of students’ knowledge of the information evaluated in the test.
However, they are not planned to have implications for the design of instruction
techniques (Arieli-Attali, 2013). Exams have been criticized for being indirect
measures that do not take into account context and that are more oriented towards
obtaining a product rather than understanding the learning process (Sadeghi &
Rahmati, 2017).
Summative assessments are useful in this sense as they are used for what they
were designed. However, when the main objective is to provide information on
the processing of the information inputs that students receive in the classroom,
rather than on the performance (the output from the process), then the necessary
use of alternative tools to measure the cognitive processes of assimilation and
accommodation of information as a result of learning becomes evident.
Nevertheless, scientific exploration of the use and impact of evaluation tools to
assess cognitive changes and provide useful indicators to correct or promote the
restructuring of a learned schema is still an underexplored field.
One way to approximate this learning-evaluation challenge is to include cognitive
psychology tools to measure the human mind. This scientific discipline has high
potential to evaluate skills (Embretson, 1999) and the formation of knowledge
structures, and can thus be applied to different aspects of the learning process. For
example, Marzano’s Learning Dimensions Model identifies five kinds of thinking
involved in the learning process: a) attitudes and perceptions, b) acquisition and
integration of knowledge, c) extending and refining knowledge, d) the
meaningful use of knowledge, and e) mental habits (Marzano & Pickering, 1997).
The measurement of these dimensions can be approximated with the paradigms
and research techniques involved in human cognition science. For example, the
research techniques used to explore human memory can be extrapolated to
studying the cognitive mechanisms involved in dimensions b, c, and d of the
Learning Dimensions Model.
Arieli-Attali (2013) stated that the idea of including advances in cognitive science
to develop new forms of measurement or complement psychometric means of
evaluation is not new. Initiatives have been emerging since the last century to link
advances in cognitive psychology to the measurement of abilities. For example,
the Air Force Human Resources Lab carried out the Learning Abilities
Measurement Project (LAMP) (Kyllonen & Christal, 1988), which sought to

4
identify indicators of student learning and achievement, taking into account
measures for processing capacity, speed of processing, knowledge, and skills. The
results of this seminal effort demonstrated that cognitive measures could
successfully predict performance in learning tasks and even do so with greater
precision than some instruments already available. Later initiatives such as the
Cognitive Design System (CDS) (Embreston, 1999) or Evidence-centered Design
(ECD) (Mislevy, Steinberg & Almond, 2003) have continued to promote the
concept of using cognitive tools within the assessment of learning.
The Chronometric Constructive Cognitive Learning Evaluation Model (C3-LEM)
by Lopez and Morales (Lopez et al., 2014; Morales-Martinez & Lopez-Ramirez,
2016; also see Morales-Martínez, 2020; Morales-Martinez et al., 2017; Morales-
Martinez, Lopez-Ramirez & Lopez-Gonzalez, 2015) is a recent initiative to
promote the use of cognitive measurement tools to evaluate academic learning.
This evaluation model is based on applying the laws and principles for how the
human mind selects, stores, and retrieves information.
From cognitive psychology, the human mind is seen as a producer of cognitive
structures called schemata. These mental structures are formed with the
knowledge that people store in their memories. Schemata possess properties
relating to their flexibility and stability. In the educational field, the students form
schemata from materials learned on a course or in a career. These schemata can
remain or be modified over time, depending on how students store, organize and
structure their learning.
Keeping the above idea in mind, Lopez (1989) proposed an academic-failure-rate
predictor system based on evaluation techniques derived from the Theory of
Human Information Processing (HIP) and the Theory of Parallel Distributed
Processing (PDP). Lopez attempted to show that the study techniques from these
areas allow the properties of learned-knowledge schemata to be observed and
measured in the same way that general knowledge schemata can be observed. He
tested this idea in his doctoral thesis, by designing and applying the Semantic
Analyzer of Schemata Organization (SASO). This system allowed him to explore
knowledge schemata in human memory (Lopez, 1996; Lopez & Theios, 1992).
Later, Lopez et al. (2014) used this model to create a new system by which to
evaluate learning. This learning-evaluation system was the origin of the Cognitive
Evaluator (known in Spanish as EVCOG), which is a computerized system that
assesses academic learning, and which gave rise to the C3-LEM developed by
Morales-Martinez & Lopez-Ramirez (2016; also see Morales-Martinez et al. 2017,
Morales-Martinez, Ángeles-Castellanos et al. 2020).
The C3-LEM (Figure 1) offers an alternative way to measure various aspects of
mental representation of the knowledge students learn in academic courses. For
example, this model allows indicators on the schematic organization of
knowledge to be obtained. Arieli-Attali (2013) pointed out that measuring the
conceptual understanding advances of students during a course can provide
useful information to support the design of teaching and learning strategies that
help students learn the knowledge and skills necessary to adapt to an

5
environment whose economy is based precisely on information and knowledge
management.
Figure 1. Phases and components of the C3-LEM
Note: From “Cognitive e-tools for diagnosing the state of medical knowledge in students
enrolled for a second time in an anatomy course,” by Morales-Martinez, Ángeles-
Castellanos et al., 2020, International Journal of Learning, Teaching and Educational Research,
19(9), p. 346 (https://doi.org/10.26803/ijlter.19.9.18). Copyright 2020 by the authors and
IJLTER.ORG.
Figure 1. illustrates the phases and components that make up the C3-LEM. In
general, this evaluation model promotes the combined and intertwined use of
mental representation techniques, computational simulation tools, and
chronometric cognitive measurement techniques to assess the modifications in the
organization and mental structure of knowledge, as well as the dynamics and
temporal changes in the learned schemata (Morales-Martinez, Ángeles-
Castellanos et al., 2020; Morales-Martinez, Lopez-Perez et al., 2020).
C3-LEM studies are based on the EVCOG procedure, which consists of two
phases: constructive cognitive evaluation and chronometric cognitive evaluation
(Figure 1). Together, these two approaches provide indicators of students’
cognitive mechanisms in terms of their ability to select, elaborate on, and build
knowledge from the information obtained from an academic course. This article
focuses on using the constructive cognitive evaluation of knowledge since it
illustrates the first step for evaluating learning with C3-LEM. The objective is to
contribute empirical evidence on the usefulness of cognitive techniques for
measuring organization and structural changes in students’ knowledge schemata
due to the learning process in a human cognition course.
1.1. Constructive Cognitive Evaluation of Knowledge Schemata Learned
during an Academic Course
The constructive cognitive evaluation of learning involves measuring the
knowledge schema’s properties through a mental representation technique and
computer simulations. The central idea is to observe the conceptual changes that
occur in the student’s memory due to the learning process.

6
Typically, the first step consists of applying the Natural Semantic Network (NSN)
technique at the beginning and the end of the academic year (see the Methodology
section), although any other technique that allows organization indicators and
conceptual structure to be extracted can be used. Figueroa, Gonzalez & Solis
(1976) proposed the NSN as a mental representation technique to explore
meaning formation. According to Figueroa-Nazuno (2007), the construction of
meaning depends entirely on the person who constructs it. The person elaborates
and interprets knowledge through a constructive and reconstructive process of
memory. So, from this conceptualization of cognitive functioning, the formation
of meaning goes beyond free association.
Mental representation studies based on the C3-LEM have provided evidence that
students construct or reconstruct their declarative knowledge schemata as a result
of the learning obtained during a course. For example, Morales-Martinez, Lopez-
Perez et al. (2020) applied the NSN technique to measure the knowledge schema
arising from a course on the Computational Theory of Mind. They observed that
students enter the course with a pre-schema. However, no conceptual
organization could be identified between the pre-schema nodes. After the course,
the students had assimilated new concepts, eliminated some information nodes,
and established an organization amongst the conceptual nodes they had learned
during the course. These results agree with Bower’s (1975) seminal idea that the
acquisition of declarative schemata embraces the incorporation of new
information nodes.
Moreover, the studies using NSN have been able to identify limitations in the
knowledge structures of students, relating to each individual’s level of academic
development in terms of the subject they are learning. Morales-Martínez,
Mezquita-Hoyos et al. (2018) noted that students who did not achieve passing
grades on the computational usability course had fractured knowledge schemata
at the end of the course. Morales-Martinez, Angeles-Castellanos et al. (2020)
reported similar data in their cognitive diagnostic study on the structure and
organization of the human anatomy knowledge schema amongst first-year
medical students. The data from this study pointed to fractured cognitive
structure in the schema and difficulties with conceptual organization.
Some reasons for schematic fragmentation include the relevance weight given to
the different topics within a course or a lack of emphasis on establishing the
relationships or connections between the topics reviewed during the academic
course (Morales-Martinez, Ángeles-Castellanos et al., 2020). Fragmented
knowledge structures are also observed in students starting a course to review a
new topic (Morales-Martínez, López-Pérez et al., 2020; Urdiales-Ibarra et al.,
2018).
Information integration strategies influence the formation or correction of
integration limitations in knowledge structures such as those mentioned above.
In this regard, Morales-Martínez, Mezquita-Hoyos et al. (2018) reported that
engineering students with a fractured schema at the end of their course managed

7
to integrate information from the computational usability schema after attending
a corrective course on the subject.
In general, NSN provides information on how the student’s mind organizes and
structures knowledge schemata according to the learning experiences during
academic courses. Few studies exist which have used the C3-LEM approach to
explore the knowledge domain in psychology. Specifically, the topics covered to
date using C3-LEM relate to the Piagetian Theory schema and the Computational
Theory of Mind (e.g., Morales-Martínez, López-Pérez et al., 2020). The results of
these studies suggested that students start the courses with vague but
pre-organized ideas about the knowledge that they will review throughout the
course. At the end of the course, students with passing grades had acquired new
information nodes in the cognitive structures related to their knowledge.
Additionally, they had established new relationships between concepts and
reconstructed or reorganized their schemata based on their learning experiences.
However, more investigations offering empirical evidence on the learning
properties of knowledge schemata in psychology are necessary to build a solid
theory about the behavior of schemata in this field of knowledge. The present
study contributes new information on the organization and schematic behavior of
the knowledge structures acquired in one of the most relevant fields of
psychology science, human cognition.
2. Methodology
2.1. Study Overview
This research measured the state of knowledge on the human cognition schema
amongst students enrolled in the second year of a psychology degree at the
beginning and end of a course. The state of knowledge refers to the set of cognitive
properties (organizational, structural, temporal, and dynamic) that characterizes
students’ knowledge schemata in any academic course. For example, at the
beginning of a course, students present less semantic richness than at the end of
the course. In addition, throughout the course, students judge the semantic
relevance of concepts in different ways. Moreover, the recognition pattern for
schematic words is different at the beginning, during, and at the end of the course.
Thus, this study explored the changes in the organization and structure of the
human cognition schema experienced by students as a result of the learning
acquired during a cognition course. The authors designed an NSN study that
included a conceptual definition task related to the human cognition schema.
2.2. Participants
The participants were 48 second-year psychology students enrolled in a course on
human cognition. Their ages ranged from 19 to 34 years old (M = 20.3, SD = 2.58).
Overall, 79% (38) were women and 21% (10) were men. The authors selected
participants using a convenience sampling technique. Potential participants were
included in the study only if they took part voluntarily and signed the informed
consent. Participants who did not finish the two application phases or did not
follow the instructions were excluded from the study.

8
2.3. Study Design
The study design was based on the EVCOG sequence proposed in the C3-LEM.
The researchers designed a mental representation study based on the modified
NSN from Lopez and Theios (1992) and Lopez (1996). The objective was to
measure the cognitive properties of the content, organization, and structure of the
human cognition schema.
2.4. Instruments and Materials
To build the NSN instrument, the researchers selected ten target concepts from
the Protocol for the Collection of Target Concepts and Central and Deferred
Definers (Morales-Martinez, 2015). This protocol guides the teacher or knowledge
domain expert in terms of identifying the most relevant conceptual targets for the
course. The resulting ten concepts were considered to be the evaluated schema
concepts. The ten conceptual targets selected by the teacher were: cognition,
cognitive psychology, perception, attention, consciousness, memory,
representation of knowledge, reasoning, problem-solving, and decision-making.
The researchers used EVCOG software to design and apply the cognitive studies
of mental representation. Additionally, this software allowed the capture and
analysis of data based on the C3-LEM (Morales-Martínez, López-Pérez et al.,
2020).
2.5. Procedure
In this study, the constructive cognitive evaluation of learning comprised the
application of a task based on the NSN technique at the beginning and end of the
course. First, the researchers invited students who were enrolled in a course on
human cognition to participate in the research. Subsequently, the students who
agreed to participate received information about the study and their rights as
participants, and gave their informed consent. After this, they performed an
exercise to familiarize themselves with the task. Finally, the NSN study was
applied.
During the NSN study, each participant observed the target concepts one by one
on a computer screen. The task was to define the targets using verbs, nouns,
adjectives, and pronouns as definers. The production criterion for definers was
that they had to be directly related to their course content on human cognition.
Phrases, articles, and prepositions were not allowed to be used for the definitional
task. The participants had 60 seconds to define each target. Subsequently, they
rated each definer using a scale from 1 to 10; 1 meant that the evaluated definer
chosen was not very related to the target concept, and 10 indicated that the definer
was significantly related to the target concept. The time to complete the entire task
varied from 15 to 20 minutes, depending on each participant.
3. Data Analysis
In this study, the authors undertook three analyses of the NSN data. The first
analysis was a traditional mental representation analysis using the EVCOG
system. This software allows several NSN values, proposed by Figueroa et al.
(1976) and described by Lopez (1996) and Lopez and Theios (1992), to be

9
computed. This analysis involved various elements which are described below.
The indicators for the analysis included, firstly, semantic richness (J value),
generated for each target concept through the total number of different definers.
Secondly, semantic relevance (M value) was obtained from the score consciously
given by the participants for each target definer, expressed as the sum of all the
weights assigned by the participants to each definer. The ten most relevant
defining concepts were also identified to build the meaning of the target concept
of the network. This group of definers is known as the SAM group (Semantic
analysis of M value or SAM) and is made up of the ten definers with the highest
M values for each target concept. Another indicator was semantic distance (FMG
value) between the given definer and the target concept that was defined. This is
computed using the percentage range corresponding to the M value of each of the
definers obtained for the SAM group in relation to the highest M value obtained
in the group. Finally, semantic density (G value) was calculated.
The second analysis was undertaken using the EVCOG system. This software
allowed the extraction of the association matrix. This matrix is called the SASO
connectivity matrix, which is calculated using a Bayesian formula proposed by
Lopez and Theios (1992). According to these authors, this equation is a
modification of that by Rumelhart et al. (1986). Lopez and Theios’s equation is
given below:
WIJ = -1n{[p(X = 0 & Y = 1) p(X = 1 & Y = 0)]*[p(X = 1 & Y = 1) p(X = 0 & Y = 0)]-1}
[1]
This equation calculates the co-occurrence probability amongst pairs of concepts
(X and Y) throughout the NSN. Firstly, p(X = 0 & Y = 1) refers to the joint
probability that Y appears but X does not appear in a SAM group. Similarly,
p(X = 1 & Y = 0) denotes the joint probability that X appears but Y does not appear
in a SAM group, and p(X = 1 & Y = 1) was computed in the same manner. The
calculation of p(X = 1 & Y = 1) involved the hierarchical modulation of M values
in the SAM groups.
The SASO connectivity matrix was used to feed the Gephi software to obtain a
graphical representation of the accommodation of schema concepts (see Figure 3).
Finally, the authors used STATISTIC software (version 7) to apply a
multidimensional scaling on the NSN data. To this end, the authors considered
the co-occurrence of definer concepts for each target concept.
4. Findings/Results
4.1. Lopez and Theios’s Analysis of NSN Data
The NSN data obtained before (Table 1) and after (Table 2) the course were
analyzed based on the procedure described by Lopez and Theios (1992).

10
Table 1. SAM groups for the human cognition schema obtained from the participants
before the course
Note: J = semantic richness, G = semantic density, F = occurrence frequency,
M = semantic weight, IRT = inter-response time
F Definer M IRT F Definer M IRT F Definer M IRT
9 Cognitive process 158 18 9 Cognitive process 159 22 2 Senses 97 26
3 Mind 115 23 3 Mind 147 18 2 Interpret 67 35
5 Memory 110 27 5 Memory 141 36 2 Stimuli 63 23
3 Learning 88 32 5 Thought 93 26 9 Cognitive process 54 33
5 Thought 79 20 3 Learning 75 24 1 Feel 51 24
4 Attention 64 32 4 Attention 73 35 4 Attention 43 22
4 Perception 44 38 4 Perception 72 37 3 Brain 36 54
5 Capacity 40 41 1 Study 69 16 1 Observe 34 22
1 Processing 36 33 1 Behavior 64 29 4 Information 33 34
3 Brain 35 31 1 Science 47 18 1 Reality 26 41
9 Cognitive process 129 24 3 Mind 78 23 1 STM 127 26
1 Focus 91 27 9 Cognitive process 55 30 1 LTM 105 29
5 Capacity 73 31 1 Mind state 45 11 1 Store 94 14
2 Stimuli 66 19 5 Thought 44 38 3 Learning 91 25
1 Concentrate 58 23 4 Attention 42 27 9 Cognitive process 88 28
5 Memory 38 24 3 Brain 40 44 1 Memories 86 24
2 Senses 36 38 1 Vigil 39 29 1 Remember 67 14
1 Selective attention 33 20 1 Internal 33 42 1 WM 65 30
4 Perception 31 47 2 Cognition 31 41 4 Information 63 32
2 Cognition 30 72 3 Reasoning 30 23 1 Retrieve 59 33
J-value: 205 J-value: 174 J-value: 282
1 Schemata 95 22 3 Thinking 117 16 3 Reasoning 64 18
1 Image 81 20 9 Cognitive process 72 20 9 Cognitive process 56 28
1 Symbols 43 15 1 Human 47 22 3 Thinking 52 18
1 Models 39 21 5 Thought 46 25 2 Reason 51 16
1 Mental 36 15 3 Analysis 45 33 5 Memory 46 27
4 Perception 33 32 1 Logic 43 19 5 Capacity 44 19
4 Information 31 31 1 Consciousness 32 33 3 Analysis 41 28
1 Object 28 27 5 Capacity/Ability 31 27 2 Choice 39 36
5 Memory 27 39 4 Information 29 30 2 Options 38 41
1 Concepts 27 37 2 Interpretation 27 46 5 Thought 38 32
F Definer M IRT
2 Choice 104 17
9 Cognitive process 73 34
2 Options 58 30
1 Evaluation 53 39
2 To reason 42 18
1 Solutions 38 31
5 Capacity 34 33
3 Reasoning 33 26
3 Analysis 31 40
3 Thinking 27 50
Cognition Cognitive psychology Perception
J-value: 218 G-value 12.30 J-value: 258 G-value: J-value: 217 G-value: 7.10
Attention Consciousness Memory
G-value: 9.90 G-value: 4.80 G-value: 6.80
Representation Reasoning Problem solving
Decision making
J-value: 212 G-value: 4.60

11
Table 2. SAM groups for the human cognition schema obtained from the participants
after the course
Note: J = semantic richness, G = semantic density, F = occurrence frequency,
M = semantic weight, IRT = inter-response time
Table 1 shows that the definers (cognitive process, mind, memory,
short-term memory (STM), thinking, long-term memory (LTM), choice, senses,
schemata) with the highest M in each SAM group before the course were mostly
general. At the end of the course, however, most of the concepts with the highest
10 Cognitive process 259 13 10 Cognitive process 230 26 1 Sensation 182 13
3 Information 98 28 1 Science 167 20 1 Interpret 152 22
2 Mind 89 22 1 Neisser 113 23 1 Threshold 125 29
7 Memory 80 33 2 Cognition 92 30 2 Stimuli 116 27
1 Cold cognition 79 31 7 Memory 79 31 10 Cognitive process 110 32
3 Attention 68 33 1 HIP 78 29 1 Direct perception 84 26
1 Psychology 63 43 3 Information 76 31 1 Illusion 69 30
1 Hot cognition 54 29 1 Representation 63 41 1 Senses 62 24
1 Human 49 42 3 Attention 62 36 4 Perception 55 24
1 Processing 49 32 4 Perception 61 32 1 Gestalt 49 44
1 Filter 211 22 3 Attention 134 25 1 Store 286 20
10 Cognitive process 190 20 10 Cognitive process 127 24 1 Retrieve 258 30
1 Selective attention 124 23 1 Become aware 94 17 1 LTM 232 32
1 Divided attention 95 31 7 Memory 83 35 1 STM 231 27
2 Stimuli 93 39 4 Perception 67 36 1 SM 230 26
1 Attenuation model 82 25 2 Knowledge 62 11 1 Encoding 147 29
1 Sustained attention 81 23 1 Unconscious 49 43 10 Cognitive process 112 22
1 Capacity 78 28 1 Explicit 48 42 1 WM 96 34
4 Perception 71 25 1 Reflector 41 36 1 Implicit 77 25
1 Focus 68 28 2 Cognition 41 49 2 Semantics 74 41
1 Schemata 261 23 1 Reasoning 239 20 1 Objective 114 30
2 Mind 198 19 1 Conclusion 216 25 10 Cognitive process 103 19
1 Concepts 142 26 1 Inductive 201 20 1 Problem 100 25
10 Cognitive process 99 25 1 Syllogism 175 26 2 Reasoning 87 30
7 Memory 98 33 1 Analogical 110 27 1 Goal 81 17
1 Images 74 20 10 Cognitive process 109 23 1 Heuristics 77 37
1 Imagine 73 14 1 Premises 86 24 7 Memory 76 48
2 Knowledge 59 24 3 Information 78 25 2 Decision 70 38
2 Semantics 43 42 7 Memory 66 34 1 Strategies 56 33
1 Absence 41 18 1 Logic 60 26 1 Initial state 55 29
F Definer M IRT
1 Choice 222 14
1 Alternative 112 19
10 Cognitive process 105 29
1 Evaluation 94 19
1 Experience 80 25
2 Reasoning 79 37
2 Decision 78 18
1 Options 62 29
7 Memory 56 32
1 Normative theories 23 35
Cognition Cognitive psychology Perception
J-value: 373 G-value: 21.00 J-value: 378 G-value: 16.90 J-value: 336 G-value: 13.30
Attention Consciousness Memory
Representation Reasoning Problem solving
Decision making
J-value: 331 G-value: 5.90

12
M in each SAM group were specific (cognitive process, sensation, filter, attention,
store, schema, reasoning, objective, choice), as shown in Table 2. Besides, when
comparing Tables 1 and 2, it can be observed that the students at the end of the
course included new definers or information nodes, rearranged some definers, or
eliminated concepts in the definitions of some targets. For example, the following
definers for cognition: thought, capacity, perception, learning and brain were
removed, and definers such as information, cold cognition, psychology, hot cognition,
and human were included (Figure 2).
Figure 2. Conceptual changes in the target cognition
In general terms, cognitive process (M value = 159) was the definer with the greatest
semantic weight in the entire network before the course (Table 1), whilst after the
course, it was store (M value = 286) (Table 2). Additionally, cognitive process was

13
the definition with the highest appearance frequency at the beginning of the
course (F = 9) and also at the end of the course (F = 10). The M value average for
cognitive process at the beginning of the course was 93.77, whereas at the end of the
course, it had increased to 144.4.
4.2. Gephi Analysis of NSN Data
The researchers carried out a graphical analysis of the changes in the organization
and structure of the NSN using the Gephi system (Bastian, Heymann & Jacomy,
2009). Gephi is open-access software which explores the properties of networks.
At the beginning of the course, the participants’ knowledge schema on human
cognition was made up of four large modules of concepts (Figure 3). The first
(blue) included memory-related definers (memories, learning, remembering, storing,
retrieval, working memory (WM), short-term memory, long-term memory, information).
The second group (purple) consisted of definers related to cognitive psychology
as a science (science, study, cognition, mental state, observing, wakefulness, feeling,
internal, reality, interpretation, reasoning, attention, senses, mind, thought, cognitive
process, stimuli, processing, brain, behavior). The third grouping (orange) embraced
definers related to decision-making (solutions, reasoning, thinking, evaluation,
analysis, ability, options, choice, consciousness, human, logic). The fourth group of
definers (green) was made up of concepts relating to cognitive processes (memory,
selective attention, concentration, symbols, perception, focus, image, schemata, models,
mind).
After the course, the participants rearranged the human cognition schema into
seven conceptual modules (Figure 3). The first module embraced definers
associated with perception (orange) (senses, interpretation, illusion, sensation,
threshold, direct perception, Gestalt). The second module (light green) included
definers related to consciousness and attention (sustained attention, divided
attention, selective attention, capacity, attenuation model, unconscious, filter, focus,
realize, reflector, explicit). Module 3 (pink) grouped concepts related to problem-
solving (initial state, strategies, problem, goal, heuristics, objective). Conceptual
group 4 (dark green) encompassed definers on decision-making (alternative,
options, choice, evaluation, experience). Module 5 (purple) concentrated concepts
related to three objectives: cognitive psychology, cognition, and mental
representation (schemata, images, absence, concepts, cold cognition, imagine, mind,
processing, hot cognition, human, reasoning, cognition, stimuli, decision, semantics,
memory, psychology, memory, Neisser, cognitive process, mental representation, HIP,
science, attention, knowledge). Module 6 (brown) included definers on reasoning
(deductive reasoning, premises, conclusion, logic, inductive, analogical, information,
syllogism). The last module (blue) involved definers related to memory (sensory
memory, short-term memory, long-term memory, retrieve, store, encoding, working
memory, implicit).
Additionally, the Gephi analysis pointed out changes in the conceptual
organization. The conceptual connections of definers had changed at the end of
the course. To illustrate these changes, observe in Figure 3 that at the beginning
of the course, cognitive process was a central definer concept in the primary schema
that participants brought about human cognition, although it did not have a

14
connection with all the schema modules. At the end of the course, the concept of
cognitive process retained its quality as a central conceptual node yet now also fully
connected with all the targets and all the conceptual modules.
Figure 3. Gephi analysis of the NSN data obtained before and after the course

15
4.3. Multidimensional Scaling of NSN Data
The researchers applied multidimensional scaling to the NSN data to examine the
general structure of the human cognition schema. The analysis showed changes
in the arrangement of target conceptual nodes due to the learning achieved during
the course (Figure 4).
Figure 4. Multidimensional scaling analysis of the target concepts

16
The multidimensional scaling graph shows that the participants started the course
without a specific structure in mind for the objective concepts, whilst at the end
of the course, they had rearranged the objective concepts based on two
dimensions. The first related to categorizing cognitive processes in terms of basic
and higher order cognition (horizontal axis). Although the definition of the second
dimension is not clear, in general, this dimension seems to be related to the use of
knowledge structures (vertical axis). Note that the target concept for reasoning
does not appear alongside targets such as problem-solving or decision-making, even
though all of these processes involve making use of knowledge structures from
memory.
5. Discussion
This study has explored changes in the knowledge schema due to the learning
process during a course on human cognition taken by second-year psychology
students. First, the authors determined whether a human cognition schema
existed before the course. The NSN and Gephi analyses indicated that the
participants entered the course with a previous-knowledge schema or a
knowledge pre-schema (see Table 1 and Figure 3). The existence of a knowledge
pre-schema has been observed in other studies (e.g., Morales-Martinez, Lopez-
Perez et al., 2020); however, the organization and structure are rudimentary. This
finding suggests that students generally have a vague schema about the
knowledge they will acquire in their courses, and it is based on this schema that
they reorganize and reconfigure the information they will learn in class.
As psychology teachers, the authors have observed that the use of general
schemata and previous learning to begin a new knowledge schema is a common
phenomenon observed in the classroom. Students generally comment that they
have come across certain information about the topic. It was therefore not
unexpected that the participants in this study commented that they were slightly
familiar with the topics. They had reviewed readings on cognitive processes in
other courses, although this had not been from the perspective of the field of
cognitive psychology.
The authors hypothesize that students use their previous learning experiences to
form a general schema or make inferences about information related to the course
in which they are enrolled. In this way, they have a conceptual basis from which
to form a more sophisticated outline of the information they cover during the
course. From a cognitive perspective, students can use or create a rudimentary
cognitive structure that allows them to guide the reorganization and restructuring
of their knowledge based on the new information inputs that they acquire through
the course. If the cognitive structure is sufficiently broad and general, it will be
flexible enough to undergo modifications due to the new learning experiences.
Interestingly, although the initial schema with which the participants in this study
entered the course was very general, their schema was not fractured as has been
observed in other courses where students start on a topic for the first time (e.g.,
Urdiales-Ibarra et al., 2018). This result may be because the participants in this
study had reviewed cognition materials the previous year when taking different

17
courses, meaning that they had had previous information about the topic. At the
start of their degree, the participating students were enrolled in a course where
they reviewed some of the concepts included in the course on human cognition
and obtained a passing grade on this initial course. Thus, they had general and
pre-organized ideas about the meaning of some important target concepts in NSN
studied in this research. Other studies have indicated that students who do not
obtain a passing grade for a course have a fragmented schema at the end of the
course compared to those who end the course with a passing grade (Morales-
Martinez, Angeles-Castellanos et al., 2020; Morales-Martinez, Mezquita-Hoyos et
al., 2018).
In this study, at the end of the course, the authors explored the changes that had
taken place in the participants’ pre-knowledge schema of human cognition due to
the learning acquired through the course. The analysis of the organization of the
schematic knowledge indicated that the participants had established new
relationships between the concepts. This result is consistent with Bower’s (1975)
idea that the acquisition of declarative schemata necessarily involves
incorporating new information nodes and new connections between these nodes.
The reader can compare the definers included in Tables 1 and 2 and observe that
at the beginning of the course, for some target concepts, some of the definers were
global concepts on the topic of human cognition. Meanwhile, at the end of the
course, the definers were more specific and theoretically closer to the target
evaluated. For example, for the initial conceptual definition of human cognition
(Figure 2), half of the concepts were categorical (memory, thought, attention,
perception, learning), and the other half were schematic (cognitive process, capacity,
mind, brain, processing). At the end of the course, however, the participants
included a greater number of schematic-type definers (cognitive process,
information, mind, cold cognition, psychology, hot cognition, human, processing).
The change in predominance from categorical to schematic relationships in
knowledge structures suggests that the participants had developed more
sophisticated schemata. That is, instead of using as many exemplification
schemata, their perception had changed and they were using more probabilistic
schemata. It is possible that, when students start learning a knowledge domain,
learning by exemplification dominates most of their knowledge acquisition
process. As participants in this study acquired new knowledge and refined it, they
began to use or establish other semantic relationships amongst the concepts. It
would be useful to carry out further research to explore this phenomenon since
there has been no discussion of this issue in previous research with C3-LEM to
date (e.g., Morales-Martinez, Angeles-Castellanos et al., 2020; Morales-Martinez
et al., 2020; Morales-Martínez, Mezquita-Hoyos et al., 2018; Urdiales-Ibarra et al.,
2018).
Another modification in the knowledge organization, which is of note, was the
change in the degree of generality with regard to the human cognition schema. At
the beginning of the course, the participants formed some groups that included
general definers and even incorporated information from other knowledge

18
schemata. For instance, module 2 of the Gephi analysis shows that before the
course, participants included definitions of various target concepts (cognitive
psychology, cognition, mental representation, perception) in the same group of
concepts and included definitions of other knowledge schemata learned for other
topics. For example, participants recovered conceptual nodes from the
behaviorism field as stimuli instead of inputs or behavior instead of cognitive patterns
(Figure 3).
The previous results indicate that at the end of the course, the participants were
able to extend and refine their knowledge about human cognition, thus placing
them at level three of Marzano’s Dimensions of Learning Model (Marzano &
Pickering, 1997). On the other hand, according to Messick (1984), the participants
in the present study would be in an intermediate stage of academic development
in terms of the development of the knowledge schema on human cognition
because indicators observed included not just the retrieval of information but a
restructuring of their schema. In congruence with this idea, the analysis of the
structural changes in the knowledge schema indicated a reconfiguration of the
schema structure by the end of the course. In this regard, Figure 3 shows how the
initial schema’s definers were arranged into four large modules, whilst the
definers for the final schema were restructured into seven conceptual modules.
Changes in the configuration of the schematic structure have been observed in
other studies that have used the C3-LEM (Morales-Martinez, Lopez-Perez et al.,
2020; Morales-Martínez, Mezquita-Hoyos et al., 2018; Urdiales-Ibarra et al., 2018).
From the point of view of cognitive psychology, changes in schematic
configuration patterns are an indication of learning. In this study, the changes to
the schema’s configurational arrangement suggest that participants had rebuilt
their structures based on the new meanings that they had acquired during the
course.
The multidimensional analysis (Figure 4) on the target concepts showed that at
the beginning of the course, the participants did not have a clear idea of how the
course’s target concepts could form a wholly organized knowledge schema. At
the end of the course, the participants organized the ten target concepts into two
dimensions, the first one relating to the cognitive nature of the processes (basic vs.
higher order cognition) and the second associated with the use of knowledge
structures. Although some concepts such as reasoning were not correctly located
in this second dimension, in general terms, this result suggests that the
participants had understood the structure of knowledge underlying the course’s
thematic organization, using the information implicit in the same target concepts.
Since this is a seminal intent of introducing a new way to analyze the results from
C3-LEM, more evidence about this phenomenon is needed to explore and explain
this kind of implicit cognitive change in the knowledge schema as a learning
product.
In summary, the study results indicated that there were changes in the
organization and structure of the human cognition knowledge schema of the
participants. They had reconfigured their old four-module schema on human

19
cognition into a new one which included seven modules. The participants
included new information nodes, eliminated conceptual nodes that belonged to
other disciplines, and established new relationships between the old and new
concepts.
6. Conclusions
In conclusion, the results of the present investigation have implications in three
areas. At a theoretical level, the study generated empirical evidence that supports
the idea that students enter courses with prior knowledge of the subject they are
going to study. For example, the study data indicated that the participants
possessed a macro-schema of human cognition at the beginning of the course. This
finding is relevant because it suggests that cognitive techniques such as those
contemplated in the C3-LEM can help diagnose preconceived ideas. It opens up
the possibility of correcting inaccurate information held by students when starting
a course. The measurement of this type of pre-schema would empower the teacher
to decide whether it is necessary to demystify some information or whether
modifications are required in the application of the established work program to
provide continuity or correct the knowledge structures held by students when
starting the course.
In addition, the results demonstrated that the learning process involves the
assimilation of new information and the elimination of specific conceptual nodes,
as well as the restructuring of schematic information. Furthermore, evidence from
the NSN study indicated that this type of technique can provide information on
students’ academic development level in a course. This finding has important
implications at the applied level. For example, how a student configures their
knowledge can also be taken as an indicator of mastery of the course knowledge.
Consequently, the C3-LEM could be a valuable tool in the formative assessment
of students. However, since the sample in this study was small and only
addressed one domain of knowledge, new explorations must be carried out in
other fields, such as the area of exact sciences (e.g., mathematics, chemistry,
physics), to calibrate the scope and implications of this evaluation model in the
design of new forms of educational evaluation and intervention.
Finally, at a methodological level, the study’s data supported the idea that mental
representation studies from the C3-LEM perspective may help assess cognitive
changes in the organization and structure of knowledge schemata.
7. References
Arieli-Attali, M. (2013, October 20−25). Formative assessment with cognition in mind: The
cognitively based assessment of, for and as learning [Paper presentation]. 39th Annual
Conference: Educational Assessment 2.0: Technology in Educational Assessment.
Tel Aviv, Israel. https://www.iaea.info/conference-proceedings/
Banister, P. (2004). Assessment as a tool for fostering key skills. Psychology Learning &
Teaching, 3(2), 109–113. https://doi.org/10.2304/plat.2003.3.2.109
Bastian, M., Heymann, S., & Jacomy, M. (2009, May 17−20). Gephi: An open source software
for exploring and manipulating networks [Paper presentation]. Association for the
Advancement of Artificial Intelligence, Third International AAAI Conference on

20
Weblogs and Social Media. San Jose California, United States.
https://gephi.org/publications/gephi-bastian-feb09.pdf
Bower, G. H. (1975). Cognitive psychology: An introduction. In W. K. Estes (Ed.), Handbook
of learning and cognitive processes: Vol. 1. Introduction to concepts and issues (pp. 25–
80). Lawrence Erlbaum Associates.
El-Yassin., H. D. (2015). Integrated assessment in medical education. Journal of
Contemporary Medical Sciences, 1(4), 36–38.
http://www.jocms.org/index.php/jcms/article/view/51
Embretson, S. E. (1999). Cognitive psychology applied to testing. In F. T. Durso (Ed.),
Handbook of applied cognition (pp. 629−660). John Wiley & Sons.
Figueroa-Nazuno, J. G. (2007, October 24−25). El significado de las Redes Semánticas
Naturales: Y la tradición oral, 20 años después [The meaning of natural semantic
networks: And oral tradition, 20 years later] [Paper presentation]. Primer
Simposium Internacional: Cognición y Representación del Conocimiento.
Monterrey, Nuevo Leon, Mexico.
Figueroa, J. G., Gonzalez, E. G., & Solis, V. M. (1976). An approach to the problem of
meaning: Semantic networks. Journal of Psycholinguistic Research, 5(2), 107−115.
https://doi.org/10.1007/BF01067252
Kyllonen, P. C., & Christal, R. E. (1988). Cognitive modeling of learning abilities: A status
report of LAMP (Learning Abilities Measurement Program). In R. Dillon & J. W.
Pellegrino (Eds.), Testing: Theoretical and applied issues. Freeman.
Lopez, R. E. O. (1989, December 1−6). Sistema predictor de indice reprobatorio (SPIR) [Failure
rate predictor system (SPIR)] [Paper presentation]. IX Coloquio de Investigación.
ENEP Iztacala, Mexico city, Mexico.
Lopez, E. O. (1996). Schematically related word recognition (Publication No. 9613356)
(Doctoral dissertation). University of Wisconsin-Madison, Madison, Wisconsin.
ProQuest Dissertations & Theses Global.
Lopez, E. O, & Theios, J. (1992). Semantic analyzer of schemata organization (SASO).
Behavior Research Methods, Instruments, & Computers, 24(2), 277−285.
https://link.springer.com/content/pdf/10.3758/BF03203508.pdf
Lopez, E. O., Morales, G. E., Hedlefs, I., & Gonzalez, C. J. (2014). New empirical directions
to evaluate online learning. International Journal of Advances in Psychology, 3(2),
40−47. https://doi.org/10.14355&ijap.2014.0302.03
Marzano, R. J., & Pickering, D. J. (1997). Dimensions of learning: Teacher’s manual (2nd ed.).
ASCD.
Messick, S. (1984). The psychology of educational measurement. Journal of Educational
Measurement, 21(3), 215–237. https://doi.org/10.1111/j.1745-3984.1984.tb01030.x
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On the structure of educational
assessments. Measurement: Interdisciplinary research and perspectives, 1(1), 3−62.
Morales-Martinez, G. E. (2015). Protocolo para la recolección de conceptos objetivo y definidores
centrales y diferidos (PRECODECD): Un sistema de codificación de conceptos extraídos
de las redes semánticas naturales [Protocol for the collection of objective concepts
and central and deferred definers (PRECODECD): A coding system for concepts
extracted from natural semantic networks] (Unpublished manuscript). Institute of
Research on the University and Education, National Autonomous University of
Mexico, Mexico.
Morales-Martínez G. E. (2020). Sistema de evaluación cognitiva constructiva cronométrica del
aprendizaje en línea y presencial [Online and face-to-face learning’s constructive-
chronometric cognitive assessment system] (Manuscript submitted for
publication). Institute of Research on the University and Education, National
Autonomous University of Mexico, Mexico.

21
Morales-Martinez, G., & Lopez-Ramirez, E. (2016). Cognitive responsive e-assessment of
constructive e-learning. Journal of e-Learning and Knowledge Society (Je-LKS), 12(4),
39−49. http://www.je-lks.org/ojs/index.php/Je-LKS_EN/article/view/1187
Morales-Martinez, G. E., Lopez-Ramirez, E. O., Castro-Campos, C., Villarreal-Treviño, M.
G., & Gonzales-Trujillo, C. J. (2017). Cognitive analysis of meaning and acquired
mental representations as an alternative measurement method technique to
innovate e-assessment. European Journal of Educational Research, 6(4), 455−464.
https://www.eu-jer.com/EU-JER_6_4_455_Morales-Martinez_etal.pdf
Morales-Martínez, G. E., Lopez-Perez, R. M., Garcia-Collantes, A., & López-Ramírez, E. O.
(2020). Evaluación constructiva cronométrica para evaluar el aprendizaje en línea y
presencial [Chronometric constructive assessment to assess online and face-to-face
learning]. Tecnología, Ciencia y Educación, 15(1), 105−124.
https://www.tecnologia-ciencia
educacion.com/index.php/TCE/article/view/371
Morales-Martinez, G. E., Lopez-Ramirez, E. O., & Lopez-Gonzalez, A. E. (2015). New
approaches to e-cognitive assessment of e-learning. International Journal for e-
Learning Security (IjeLS), 5(2), 449−453.
https://doi.org/10.20533/ijels.2046.4568.2015.0057
Morales-Martinez, G. E., Ángeles-Castellanos, A. M., Ibarra-Ramírez, V. H., & Mancera-
Rangel, M. I. (2020). Cognitive e-tools for diagnosing the state of medical knowledge in
students enrolled for a second time in an anatomy course. International Journal of
Learning, Teaching and Educational Research, 19(9), 341−362.
Morales-Martinez, G. E., Mezquita-Hoyos, Y. N., Gonzalez-Trujillo, C. J., Lopez-Ramirez,
E. O., & Garcia-Duran, P. J. (2018). Formative e-assessment of schema acquisition in
the human lexicon as a tool in adaptive online instruction. In R. Lopez-Ruiz (Ed.),
From natural to artificial intelligence: Algorithms and application (pp. 69−88).
IntechOpen. http://doi.org/10.5772/intechopen.81623
Muskin, J. A. (2015). Student learning assessment and the curriculum: Issues and
implications for policy, design and implementation (Current and critical issues in
the curriculum and learning). UNESCO International Bureau of Education.
http://www.ibe.unesco.org/sites/default/files/resources/ipr1-muskin-
assessmentcurriculum_eng.pdf
Rumelhart, D., Smolensky, P., McClelland, J., & Hinton, G. (1986). Schemata and
sequential thought processes in PDP models. In J. McClelland, D. Rumelhart, &
the PDP Research Group (Eds.), Parallel distributed processing: Explorations in the
microstructure of cognition: Vol. 2. Psychological and biological models (pp. 7−57). MIT
Press.
Sadeghi, K., & Rahmati. T. (2017). Integrating assessment as, for, and of learning in a large-
scale exam preparation course. Assessing Writing, 34, 50−61.
https://doi.org/10.1016/j.asw.2017.09.003
Urdiales-Ibarra, M. E., Lopez-Ramirez, E. O., Castro-Campos, C., Villarreal-Treviño, M.
G., & Carrillo-Colon, J. E. (2018). Biology schemata knowledge organization and
meaning formation due to learning: A constructive-chronometric approach to
concept mapping usability. Creative Education, 9(16), 2693−2706.
https://doi.org/10.4236/ce.2018.916203
Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1),
3–14. https://doi.org/doi:10.1016/j.stueduc.2011.03.001

22
©Authors
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0
International License (CC BY-NC-ND 4.0).
Vol. 20, No. 2, pp. 22-35, February 2021
Portfolio as a Tool for Self- Reflection and
Professional Development for Pre-Service Teachers
Arlinda Beka
Faculty of Education, University of Prishtina ‘Hasan Prishtina’, Kosovo
https://orcid.org/0000-0002-9871-783X
Ganimete Kulinxha
Faculty of Education, University of Prishtina ‘Hasan Prishtina’, Kosovo
https://orcid.org/0000-0003-4461-8841
Abstract. Self-reflections, especially those done through the usage of
portfolios, are an essential component of teaching. Portfolio reflections
allow pre-service teachers to set professional goals. This research was
conducted using qualitative methodology, and all semi-structured
interviews were organised with students and graduates from the Early
Childhood Department-Faculty of Education, University of Prishtina.
The research results show that a portfolio provides future teachers with
the opportunity to organise their work better while enabling cooperation
between pre-service teachers, which ergo, directly affects their
professional competencies. Owing to its benefits, portfolio should be
integrated by the Faculty of Education of the University of Prishtina as
an integral part of its work with students.
Keywords: portfolio; pre-service preschool teachers; teacher professional
development
1. Introduction
Self-reflection is one of the most important competencies that a teacher should
possess. Reflection directly enables the identification of things that not only are a
strong part of professional work but that also need to be advanced and to allow a
higher level of professionalism. In terms of taking individual responsibility for
thought and action processes, self-reflection is a required condition for assessment
and regulation (Dilger, 2007). After completing a given task, one tries to reflect on
the process and the work accomplished; however, sometimes one’s attention is
drawn to the sections that have been the most successful or most challenging,
leaving out many other aspects that may be equally important for enhancing the
quality of one’s work and for continuing professional development.
Portfolios are a proper way of recording our accomplished work and the
challenges and successes encountered during the process. The record of such data

23
allows one to go back in time to see what one went through, how one did one’s
activities, how one conceived things or what challenges were faced during a given
period. The portfolio technique is viewed as particularly huge with regard to self-
intelligent learning. The technique forms a significant part in the advancement of
polished skill through reflection and self-reflection (Riebenbauer, Dreisiebner &
Stock, 2017).
Through the use of portfolios, pre-service teachers can succeed in developing a
professional working plan, reflect on their work, develop additional skills in the
use of technology for professional development and have a clear review of their
work (Beka & Gllareva, 2016). Portfolios for self-reflection in pre-service teacher
training provide vital input for trainers since they represent both the strengths
and weaknesses of students (Cimermanová, 2019).
Portfolios play a significant role in not only the process of preparing teachers for
employment but also in their licensing as teachers and coaches and their state-
level performance appraisals. Being reflective is a key aspect of any educational
experience. It involves the continuous educational growth of both professional
educators and students (Webster & Whelen, 2019).
However, portfolio is often regarded as a collection of facts and documents that
testify teachers’ professional development and achievement. In some cases, it is
considered as an important part where the teacher must present the work planned
for certain periods, including the reflection on commitments which he/she has
accomplished. Therefore, the teachers make the portfolio in terms of the purpose
that it will serve them. Similarly, during pre-service studies, future teachers are
sometimes required to have more reflections on their portfolio. In some cases,
their portfolio is also required to include facts, arguments and goals that students
aim to achieve during their studies (Stock & Winkelbauer, 2012).
2. Development of portfolios and the arguments
The use of portfolios in pre-service teachers’ professional development and
professional assessment has become widespread around the world. As a result,
several contemporary studies and research projects are being conducted by
various scholars and educational scientists regarding the importance of portfolios,
their meaning and their relevance to teachers’ professional development. It is
often thought that portfolio has begun to be affirmed in the last two decades as a
contemporary approach to professional development. Portfolio, as a concept in
education, has been developed in the last decades of XX century in the United
States of America and the German-speaking countries (Slepcevic-Zach & Stock,
2018).
For a clear understanding of the concept of portfolio usage among teachers
whether in pre-service or in-service, we need to determine the purpose of the
portfolio and the way in which it acts as a professional aid to the teacher.
According to various authors, there is a discrepancy between the portfolio and the
way it should be filled in, i.e., whether the portfolio will focus more on reflection
or on the documentation of professional work.

24
In his paper, ‘Do Portfolios Have a Future?’, Driessen (2017) divides portfolio into
two categories, namely reflective portfolios and comprehensive portfolios.
Reflective portfolio is primarily intended to help students improve their reflective
abilities. It requires students to write a reflection on an aspect of their learning or
career that is significant to them. Reflections can be found in comprehensive
portfolios, but their material is much more varied than that of reflective portfolios
(Driessen, 2017). Baumgartner (2009) classifies portfolio into the three following
types depending on their purpose: reflection portfolio, development portfolio and
presentation portfolio.The e-portfolio is a flexible tool that can be used for several
purposes, including learning, evaluation/assessment and employability/career
growth (Ciesielkiewicz, 2019). Trávníčková and Puhrová (2020) state that portfolios
can be encountered in the following terms: ‘professional teaching portfolio’,
‘professional development portfolio’, ‘professional practice portfolio’ and even
the relatively new term of ‘eportfolio’.
All the above types of classifications from different authors can be considered as
being similar due to their common elements. The reflective portfolio, which is
considered a crucial part of teachers’ professional development, is common in the
classifications. The aim of comprehensive portfolios is to assist students in their
learning and to evaluate their progress. Reflections can be found in
comprehensive portfolios, but their material is much more varied than that of
reflective portfolios (Driessen, 2017).
Since education methods, such as reporting, contemplation and representation of
one’s learning activity, were major issues in the 1980s, the portfolio idea might
have been a pillar in the German debate about educational reform (Häcker, 2009).
Like in Germany, in almost all European countries and beyond, the portfolio is
beginning to be considered a significant part of pre-service and pre-service teacher
work. Nowadays, portfolios are seen as
“rich, contextual, highly personalised documentaries of one’s learning
journey.” (Jones & Shelton, 2011, p.21).
There are numerous reasons why a portfolio is considered an important tool for
teacher development and professional development. Portfolios are considered
beneficial for pre-service and in-service teachers
“as they promote autonomous learning and autonomous learners are often
motivated and reflective learners.” (Fernández, 2014).
In almost all professions, especially the teaching profession, reflection is a crucial
part. According to Lynch (2000), reflection is considered to be an important part
of a person’s skill set. Through reflection, each teacher undergoes a long and
complicated process to reach his/her desired stage of professional development.
Within the scope of the portfolio-work, not only the result (item) is critical but
mostly the method that led to this improvement (Kerr, 2007). Therefore, the
documentation of professional experiences through the use of a portfolio is an
asset for every pre-service teacher. Usually, the portfolio’s documentation is
structured and deliberately designed as
“they contain purposefully organised documentation that demonstrates
specific knowledge, skills, dispositions, and accomplishments achieved

25
over time. Portfolios represent connections made between actions and
beliefs, thinking and doing, and evidence and criteria. They are a medium
for reflection through which the builder constructs meaning, makes the
learning process transparent and learning visible, crystallises insights,
and anticipates future direction.” (Jones & Shelton, 2011, p.22)
Despite the different definitions of portfolio which depend on their purpose,
objective and nature, certain theories consider portfolio as a personal ownership,
personal record structured collection of material and so on. According to Paris
and Ayres (1994),
“The overarching purpose of portfolios is to create a sense of personal
ownership over one’s accomplishments because ownerships engender
feelings of pride responsibility and dedication (p.10).”
McKimm (2014) defined teaching portfolios in her book ‘A Handbook for
Teaching & Learning in Higher Education’ as
“A teaching portfolio (sometimes referred to in this chapter as a
‘portfolio’) is a personal record of achievement and professional
development as a university teacher. It is a carefully selected and
structured collection of material that may demonstrate a level of
attainment, a range of skills and activity, and/or progression.(p.473)”
Kayler (2004) states that:
“Teacher education institutions need to make the often-invisible
professional growth of classroom teachers more visible to other teachers
and programs as a way to foster and expand the teacher knowledge base
(p.267).”
According to Fernández (2014), portfolios play a significant role in stimulating
and inspiring teachers as well as students. Learning becomes more effective and
motivating with a portfolio since the students take ownership of their learning
process and develop strategies to plan, monitor and evaluate their learning.
Teachers benefit from portfolios in their professional development since
portfolios promote autonomous learning, and autonomous learners are often
motivated (Fernández, 2014). Portfolios can also be considered as a source of data,
whether qualitative or statistical, and are an important tool for evaluation.
In terms of assessment, Belgrad, Burke and Fogarty (2008) consider that portfolios
(and e-portfolios) can become excellent tools for both, formative and summative
assessments. Martin-Kniep (1999) considers portfolios as an instrument that
recounts the account of the educator whose objective is to make guidance and
evaluation rehearses something very similar and focused on critical learning
results. The portfolio uncovers the educator’s extraordinary capacity to apply
educational plan and appraisal-related ideas and abilities to the advancement of
exercises and evaluations that target obviously characterised results. The
instructor has provided all drafts of appraisals, enabling the user to notice the
changes and upgrades made to apply plan standards and better adjust
educational program, guidance and evaluation. The teachers can utilise their
portfolios to report their expert history, the inquiries and worries that drive their

26
work, their endeavours to plan or carry out better or diverse learning encounters
for understudies, their investigation of basic issues with at least one understudy
or the settings that encompass them, their expert work with partners, local area
individuals or bosses and their examination of and reflection on proficient assets
or critical writings. Notwithstanding the specialised parts of portfolios, it should
be remembered that the
“essential crowd for an educator portfolio is the instructor who readies the
portfolio. Simultaneously, the portfolio can help instructors screen the
change from the exercise as-wanted to the exercises educated, to the
exercise as-learned. At last, portfolios can help screen educators’
endeavours to fuse new information and abilities into their expert
collections. Albeit quite a bit of this work is a private undertaking,
educators, as different experts, need to take part in a discussion with their
partners to really comprehend and approve their work. Portfolios can fill
in as a springboard for such conversations.” (Martin-Kniep, 1999, p.15)
This is very well explained in ‘The Portfolio Connection: Student Work Linked to
Standards’, in which Belgrad et al. (2008) states that:
“Portfolios as the outcome of a continuous, student-centred process help
students learn in a variety of ways. One remarkable by-product of the
portfolio process in the classroom is increased student awareness of and
responsibility for learning objective (p.15).”
The portfolio presents pre-service teacher’s information that is directly related to
the personality, mindset, worldviews, mission and vision approaches that pre-
service and in-service teachers create for themselves, whether on a personal or
professional level. Portfolio creation represents the nature of the individual’s
personality, his/her level of knowledge and the goals that he/she wishes to
achieve in his/her professional field.
This approach will help teachers in their awareness of professional development.
They will manage to present the goals for their work and the path that they will
follow. Moreover, portfolios will act as an open statement that they will follow to
be more successful. In this way, they manage to develop critical thinking about
themselves and their profession and begin to be more creative regarding their
plan to achieve their professional goals. They also become aware of a sustainable
professional investment that will serve their work in the long term. The portfolio
also presents the skills that pre-service teachers have built up during their studies
and becomes a bridge between them and the work that they will aspire to have,
which enables teachers to have personal and professional competence.
Through reflective learning, pre-service teachers can learn from their previous
experiences either in cognitive or practical terms, enabling them to make decisions
for change and professional development. Pre-service teachers can also share
information with their peers and obtain feedback from their colleagues, thus
having an excellent opportunity to be open to others, share ideas and collaborate
in the professional field.

27
Creating a portfolio during the study period enables pre-service teachers to
establish a culture of presenting ideas, approaches and understandings in the
personal and professional field. By establishing strategies for presenting their
work in a broader professional environment, they continue to present their work
in the same way when they look for a job or even after they are hired. For teachers
in the portfolio, a service is a great tool that can serve them for the evaluation
performed either by their institution or even at a wider level.
Therefore, in terms of teacher professional development, regardless of in-service
or pre-service teacher training, portfolio is a considerable tool for documentation,
assessment and evaluation of instructors’ work. They can be pre-designed and
pre-structured to collect and document information as required for a specific task
or profession, i.e., pre-service or in-service teaching. Portfolios enable teachers to
document the evolution of different areas of their work, such as curriculum and
assessment development, professional development, inquiry/research and
teaching and learning in general.
When used by students, portfolios are developed to display materials that
represent the abilities and accomplishments during a specified period. According
to Henderson, Davis & Day (2015), students’ presentation of materials over an
extended time helps to chronicle their improvement and competence that cannot
be measured by paper and pencil tests. One of the primary purposes of a portfolio
is to provide a more reliable and comprehensive picture of student growth and
achievement.
By using pre-service teachers’ portfolios, instructors will have better
opportunities for their personal and professional development and will create a
professional culture for their work with children. In Kosovo, as in several other
nations, one of the vital competencies of instructors is to reflect on the work they
do, to archive their completed exercises and to show the goals they wish to attain
in a given period. Typically, the employment of a portfolio plays a critical role in
teacher instruction in Kosovo or any other nations because it makes a difference
in creating the key competencies for educator teachers and pre-service instructors
(Beka, 2015).
According to the implemented laws and educational policies, all the pre-service
and in-service teacher trainings in Kosovo are organised by public universities,
particularly by their faculties of education (Ministry of Education, Science and
Technology [MEST], 2011). The Faculty of Education of the University of
Prishtina, being the oldest one, carries out the most significant role in teacher
training. It provides pre-service and in-service teacher programs from
undergraduate to doctoral level (University of Prishtina, 2004).
As part of the Strategic Development Plan of the Faculty of Education,
improvement of quality teaching programs and teacher professional development
are considered crucial factors for enhancing our education system (Faculty of
Education, 2013). Professional portfolios of pre-service teacher are becoming now
an integrated part of the teaching curriculum in all of its study programs. The
Faculty of Education has had in the past few years several in-service training

28
programs for teachers’ professional development besides the regular study
programs. Those programs have been supported by the MEST and different
donors-education partners aiming to advance teachers’ professional
development. Examples of these programs are: Advancement of Qualifications of
Teachers Project, Capacity Building for Teacher Mentors, Re-Qualification of
Pedagogical Student Graduates, and so on. So far, in- service training programs
have not been part of the systematic policy of Teacher Professional Development
(TPD). Rather, they were a need for the re-qualification of teachers in their
teaching degree or in the new teaching strategies (MEST, 2015). With the new
strategic documents in place, in-service training is expected to play a vital role in
TPD, leading to teachers’ career advancement (MEST, 2016). A professional
portfolio is a required tool from the pre-service studies as it will present ideas,
professional goals and the ways of achieving these goals in a period of time. It will
help teachers to self-reflect on what they have learned and how to improve their
professional practice. Reflecting on their accomplished work is significant for the
role and the well-being of the activities that they should perform on campus and
in internships. However, not all students find the portfolio relevant to their
professional development. Junior students treat it more as a task that they should
complete during their studies, whereas senior students understand the
importance of the portfolio and view it as a tool that will help them to enhance
the quality of their work and to promote continuous professional development.
At the same time, senior students consider the portfolio as a way to achieve the
goals that they set for professional development.
3. Research methodology
This research paper presents the current state of usage of portfolio as a tool for
teachers’ professional development in pre-service teacher training at the Faculty
of Education of the University of Prishtina. The research was conducted using
qualitative methodology. Interviews were organised with students and graduates
of the Early Childhood Department-Faculty of Education, University of Prishtina.
Students were randomly selected; every fifth student was selected for each year,
while alumni were purposely selected due to their easier access and the possibility
of their inclusion in this research. Five students were selected for each year of
studies, starting with the second year of studies, third and fourth, as well as five
graduates. A total of 20 respondents were included in this survey.
The interviews were organised with each student and graduate separately. Each
of the interviews lasted approximately 45-50 min. Students and graduates have
voluntarily participated in the interviews. The difference between sophomores
and third- and fourth-year students was that sophomores felt more limited in
giving answers due to their beliefs about the portfolio and its relevance –
reflection, while third- and fourth-year students felt more confident in giving
answers since they already had a clear portfolio effect on their professional
development as it was their reflection on the quality of the work they did. The
graduates were clear about the importance of reflection through the portfolio.
With the permission of students and graduates, all interviews were recorded and
transcribed. The final transcription report was provided to the participants for
verification and signature. By signing these reports, they have confirmed that

29
their answers were correct and the meaning they wanted to give has not changed
through these statements.
The research was conducted to answer the following research questions:
1. ‘How will a portfolio help pre-service teachers in their professional
development’?
2. ‘How do pre-service teachers understand the importance of a portfolio at
different stages of studies’?
3. ‘What impact does pre-service teacher alumni portfolio have on
instructors’ employment’?
4. ‘In what way does self-reflection orient pre-service teachers towards
motivation for professional development’?
5. ‘How does self-reflection affect in-service teachers for continuous
professional development’?
6. ‘How appropriate is the portfolio as a tool to develop self-reflection in pre-
service teachers’?
The questions asked in the interview were semi-structured and intended to
obtain the students’ opinions on the following:
• The importance of portfolio preparation and reflection for pre-service
preschool teachers;
• The benefits of portfolio preparation;
• The portfolio’s significance for professional development; and
• The portfolio’s importance in employment.
4. Research outcomes
4.1 The importance of portfolio preparation and reflection for pre-service
preschool teachers
Based on interviews regarding students’ opinions on the preparation of portfolios,
it is evident that learners had very different perceptions. Since the concept of
portfolio creation is perceived differently by students, varying in various levels of
study, second-year students are found to be more inclined to viewing it as a chore
that should be checked off of their list of obligations;
“I spend a lot of time preparing my portfolio…it takes hard work.”
Nevertheless, third-year students seem to understand its importance in their
professional upgrading. They begin to consider portfolios as a useful tool which
they will continue to use at their jobs even after graduation. A third-year student
stated that:
“In the beginning, I thought it was not important but now it seems to be
very helpful.”
By contrast, fourth-year students or alumni do not view portfolios as a task but
instead as a tool for summarising and evidencing works and professional ideas.
Similar to the previous statement, a fourth-year student declared that:
“By using the portfolio, I know what I’m doing…selection of materials is
important for ongoing development.”

30
Meanwhile, the alumni expressed that:
“The portfolio is a necessary tool as it helps to reflect on professional
development.”
Based on these answers, it can be concluded that the portfolio is seen as an
exceedingly important tool, necessary for the ongoing professional development
of graduates who have entered the teaching profession.
4.2 The benefits of portfolio preparation
There is a discrepancy between second-year students and other years students’
opinions regarding the benefits of preparing a portfolio. A second-year student
stated that:
“Portfolio helps me to get a good grade.”
Meanwhile, third-year students stated that:
“The portfolio helps me recognise my weaknesses…”
This attitude is directly related to the reflection of the pre-service teacher related
to his/her work. Through portfolios, students manage to self-evaluate their work
and professional development. On the other hand, a fourth-year student notes the
portfolio’s importance as an opportunity for the development of critical and
creative thinking. In this regard, she declares that:
“The portfolio helped me to be a critical thinker in my work and
professional development. By using the portfolio, I can easily identify my
strengths and my weaknesses.”
Moreover, an alumni student considers portfolios as a tool that helps her to focus
on completing her objectives. In this regard, she stated:
“The portfolio has helped me become more efficient, improve my quality
of work in planning and my cooperation with colleagues.”
Based on the statements provided by the students, it has been concluded that
second-year students consider the portfolio as a means to achieve the highest
grades, while third-year, fourth-year and alumni students view it as an essential
implement for professional development. Their opinions have developed
precisely from their experiences with portfolio preparation. Self-reflection on the
professional development of pre-service teachers is more witnessed among the
fourth-year students than the students of previous years. They have already
realised the importance of reflecting on their work, and through this, they begin
to make ongoing plans for their professional development. The reflection itself is
even more emphasized among the graduates. Through statements, they have
emphasized that through this form, they become more efficient and focus to a
greater extent on the quality of their work. Reflections has allowed them to
improve their professional practices and knowledge, thus making them better
prepared teachers.
4.3 The portfolio’s significance for professional development
Students have expressed the significant contribution of portfolios to their
professional development as well as their reasoning behind it. Even in the case of

31
second-year students, it is evident that their lack of experience makes them more
inclined to consider portfolio as a task through which they can learn. At this level,
they have not yet fully established the importance of portfolios in their personal
and professional development. Therefore, by continuing to think that the
employment of portfolios enables them to learn new things and compare their
work with others, the portfolio becomes more appealing. A second-year student
declared:
“I do research so that I can have a good portfolio, which in return will help
me improve my work. The portfolio also helps me to compare my work
with others.”
Meanwhile, the opinions of third-year students begin to present a higher level of
advancement regarding the importance of portfolios in their professional
development. Hence, third-year students are more inclined to consider it as an
encouraging tool for cooperation with others and as something that helps them
feel secure. In this regard, a third-year student stated:
“The portfolio has helped me to cooperate with others. Cooperation is a
job-requested skill…”
In contrast to the second and third-year students, the opinions of the fourth-year
students hold the portfolio as a source of new ideas, an instrument that provides
them with security, and more importantly, an opportunity for self-evaluation. A
fourth-year student declared:
“The preparation of portfolios has given me more confidence and I do
better quality work. I am more confident in what I do”
This allows for a better understanding of the fourth-year students’ view because
through their experience, whether they have been taught on campus or
internships, they have found the portfolio to have a significant impact on their
preparation for the teaching profession. Therefore, their opinions at this stage are
also more constructive and based on the achieved results. Meanwhile, graduates
see the importance of the portfolio as an evidence of their qualitative work with
which they can secure their job more easily, which encourages them to apply this
kind of assessment with the children involved in their work. In this regard, an
alumni declared:
“The portfolio is the summary of my work. Additionally, the portfolio has
enabled me to get a job at a private preschool.”
Based on the students and alumni’s answers, we can conclude that portfolios are
a critical tool in the professional development of pre-service teachers. By using
portfolios, pre-service teachers understand goals and ways to achieve those goals.
The portfolio is a testimony of the acquired knowledge and skills of pre-service
teachers, which consequently enables them to focus on their professional
development.
4.4 The portfolio’s importance in employment
Due to the portfolio being one of the most important tools for their professional
development, students have shared their opinions on the importance of the
portfolio in their employment phase. This question has been answered by third-

32
year students, fourth-year students and alumni since their chances of obtaining a
job are higher than those of second-year students. The students have described
their opinions about the professional preparation that they have achieved through
the development of portfolios. They explained how much the preparation of
portfolios has helped or will help them to find employment in their professional
field. Third-year students have said that the portfolio has helped them to love
their future profession and at the same time has helped them to develop
presentational skills. As a result, they can better present their work, and according
to them, these are the benefits that will give them advantage over others. One of
the opinions of the third-year student in this regard was:
“The portfolio has helped me to further develop my presentational skills
as I had to present my work. It has helped me to love my future profession
more…”
A fourth-year student stated that throughout the portfolio preparation process,
they are taught to organise and document their work, which is of great advantage
to all those who work. The student also said that developing portfolios requires
creativity and the use of new ideas and different materials, especially those that
are recycled so that the portfolio is more meaningfully presented to others.
“The portfolio helped me understand that it is necessary to be organised
at work… Portfolio preparation pushed me to be more creative and to use
recycled and reusable materials.”
Given the above-mentioned characteristics, students are convinced that they will
find jobs more easily since those aspects are required in the first interview.
Graduates have stated that during their studies, in certain cases, they did not
consider portfolio as useful work. However, when they applied for work, they
realised that a portfolio is an extraordinary tool used to present their professional
preparation:
“In the beginning, I thought I am doing work that nobody appreciates,
but now I see that the portfolio is an amazing testimony. Each time I look
at my portfolio, I feel like a teacher, and I will create a portfolio for my pre-
school students.”
Based on the interviewers’ opinions, we can conclude that the portfolio also plays
an important role in pre-service teacher employment. Through portfolios, pre-
service teachers can easily prove their professional development and present their
work to children. It is of utmost importance that they apply the same kind of
assessment to pre-school children; the application of formative assessments and
the documenting of the level of childhood achievement. Furthermore, they are
able to document the ideas, approaches and goals related to their teaching
profession.
5. Conclusions
According to the research data, portfolio is a siginificant factor in educators’
professional development. This instrument provides future teachers with the
opportunity to learn to self-reflect and to simultaneously organise their work
better. To create skills in view of collaborating with others directly and to reflect

IJLTER.ORG Vol 20 No 2 February 2021

IJLTER.ORG Vol 20 No 2 February 2021

IJLTER.ORG Vol 20 No 2 February 2021

IJLTER.ORG Vol 20 No 2 February 2021

Recomendados

Más contenido relacionado

Similar a IJLTER.ORG Vol 20 No 2 February 2021

Similar a IJLTER.ORG Vol 20 No 2 February 2021(20)

Más de ijlterorg

Más de ijlterorg(10)

Último

Último(20)

IJLTER.ORG Vol 20 No 2 February 2021