1. September 28, 2012
Villach, Austria
An Adaptive Assessment System
with Knowledge Representation and Visual Feedback
Speaker: Joaquim Fernando Silva joaquim.silva@fe.up.pt
Francisco Restivo fjr@fe.up.pt
1

2. Agenda
• Goal Research
• Research Problem
• Research Hypothesis & Question
• Adaptive Assessment Systems
• General Architecture
• Solution Approach
• Conclusions and Ongoing Work
September, 28 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 2

3. Goal Research
• Regulate the education process.
• Design a technological artefact for formative
adaptive assessment.
• Effectively monitor student’s learning
achievements.
• Get student’s involvement through visual
feedback.
September, 28 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 3

4. Goal Research
• Use assessment as individual/class awareness.
• Deliver an effective feedback assessment tool
for a teaching-learning system.
September, 28 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 4

5. Research Problem
• Assessment missed opportunity in education.
• Students are not motivated or engaged in
learning activities.
• Teachers aspire to find the average student.
• Students with difficulties :
– make small gains in school performance.
– academic achievement gap widens year after
year.
September 28, 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 5

6. Adaptive Assessment Systems
• Different approaches:
– Presentation (color schemes, layouts,
functionalities);
– Curriculum sequencing; My Focus
– Problem-solving;
– Evidence based (knowledge, skills, attitudes).
• Several processes:
– Tests (item based);
– Questions (curriculum and rule based). My Focus
September, 28 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 6

7. Research Hypothesis
Students desire immediate results during the
learning process and quantitative measures to
differentiate themselves.
Research Question
How to use technology, for formative
assessment, in formal learning contexts to
individually infer which knowledge each student
is acquiring and identify the learning strengths
and weaknesses.
September 28, 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 7

8. General Architecture
Modules:
• Domain
Supports the domain/concepts structure. Teacher
• Activities
Defines, classifies and stores activities
for assessment tests. Activities Module Domain Module
Pedagogical
• Pedagogical Module
Defines the instructional method and the
multiple concepts/ activities sequences.
• Student Environment
Module
Feedback
Module
Student Module
Stores student’s profile and log
information when doing the tests.
• Feedback
Transmits, using text and graphics, the Student
test results and gives hints to students.
• Environment
Web/desktop/game/mobile based.
September 28, 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 8

9. Solution Approach
Adaptive assessment
– Static dimension based on a structured domain
knowledge;
– Dynamic dimension based on student’s performance
and curriculum sequence;
– Modelling student behaviour (age, sex, learning style,
reaction towards failure, future commitment) within
assessment patterns.
– Centred in the first three lower Bloom’s Taxonomies
(remembering, understanding and applying).
September 28, 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 9

10. Solution Approach
Visual Feedback
•Static for the knowledge
domain, using a Treemap
graphic;
•Dynamic for the student
performance, using a time-
oriented graphic
(PeopleGarden).
September 28, 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 10

11. Solution Approach
Use Case *
Adaptive Assessment System
«extends» «uses» *
Create Class Class Choose Class
Teacher: -End37
«extends»
-End40
Creates class;
*
* -End33 *
*
* -End36
-End28
-End34 -End35 -End32
-End30
-End38 Choose Domain -End39
-End22
*
Choose domain;
-End24
-End26 *
*
* * * *
* -End27
Teacher n
Authors, classifies activities;
Register Student n
-End21 *
* -End29
Structure concepts; -End23
Profile
*
Defines rules and abilities.
* -End31
Global Reports
-End25 *
+End2
Author Activities
Student: -End4
*
«uses» -End8
Register and fill profile;
Classify Activities * -End10
«uses»
-End3
+End1
-End5
-End5 *
-End5 -End6 -End7
-End9 *
Choose class and domain;
«uses» -End11
** On Line Assessment Student 1 -End12
* Structure Concepts
* Test *
* * *
*
«uses» Student 2
*
Defines global reports;
-End6
«uses»
Teacher n Define Rules
*
Do online assessment tests;
«uses» Student n
-End6
Define Abilities *
*
Visualizes feedback results.
-End18
-End15
* * * -End17
-End19 -End20
*
Feedback Results
-End16 Student 1
-End14 -End13 ** *
Student 2
Teacher n Student n
September 28, 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 11

12. Solution Approach
Proof of Concept
 Experimental work in a high school context;
 Assessment artefact in Web and mobile based
environments;
 A test pilot school experiment with multiple domains
(disciplines) and school levels (years);
 Online survey for qualitative measuring the tool
impact in terms of interest, changing attitudes towards
learning and improving knowledge acquisition.
 September 28, 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 12

13. Some Conclusions
Teacher
Provides knowledge domain structure as
individual basis, difficulty in sharing;
Slow changes in teaching patterns;
Student
 Appealing interest in visual representations;
 Emotional and intuitive thinkers;
 On-demand for personalised feedback;
 Media interaction provides deeper insight;
 Social dimension reaches individual learning
awareness.
September 28, 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 13

14. Ongoing Work
• Complete the evaluation experiment in the High
School ;
• Inquiry teachers and students about the tool
usefulness;
• Design the interactive visual feedback;
• Qualitative analysis on changing teacher’s
attitude and student’s impact of visual
assessment feedback.
September 28, 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 14

15. Thank you for listening.
Any questions are welcome.
Faculty of Engineering at University of Porto
PORTUGAL
Joaquim Fernando Silva joaquim.silva@fe.up.pt
Soon more information available at:
September 28 2012 J. F. Silva & F. Restivo ICL 2012 Villach Austria 15