The final part of our evaluation was a try out with a nine year old child and one of the designers. The findings of this evalutions can be read in slideshare below. During the end presentation next week you can see a video of this try-out.

1. Try-out (Notice: a video of the try-out will be presented next Tuesday.)
In the last week before the presentation of our design, one of the designers tested the entire project
herself together with her 9 year old daughter (the pupil). This try-out was filmed and will be
presented during the end-presentation next Tuesday. The designer started with an explanation, in
the way it was written in the teacher manual. So the designer acted as teacher, and her daughter as a
pupil. Together they went through each step of the project. The designer observed the parts of the
project where pupils should work independently. The entire project took 3 hours, whereas the third
step, the actual design of the artwork, was executed a bit faster than should be because of the
limited time.
This try-out gave insight of the practicality of the design as a whole, the clarity of the text and tasks,
the time that is needed per step, and the expected educational value (will our product achieve its
goals?).
Practicality
The scenario: In general the scenario was executed as expected, but not step 1. The information
found on the Wikipanion app was limited, and the pupil was not sure whether it was about the right
location. Therefore she needed to go to the location itself to verify the information she had found,
and to find out exactly about which part of the location it was. The other three steps were executed
as expected, with the exception of sending email. The apps that were used all had email functions,
and it appeared to be far more logical to email directly from the app (e.g. the notes app and the
moodboard app). Therefore the email link in the instruction was not used. The final presentation was
skipped because of the time.
Clarity/interpretation: for the greater part the text was clear. Most difficult was the Aurasma
explanation in step 4b. What made it even more complicated, is the fact that switching to the step to
read the instruction, caused that you had to start over in Aurasma. Also not completely clear was the
way the examples in the introduction should be treated. From the teacher manual is seemed that
they were shown and explained before the pupils started the instruction. But when the pupils were
going through the first screen on the Ipad, the examples were presented again to be discussed in the
group. This was a bit confusing. It also wasn’t very clear in the teacher manual.
Another clarity problem was in the app used for step 3: 123D Sculpt. When saving the artwork, it was
saved in the app and not in the photo library. To achieve saving in the library, it was necessary to
apply the following operation in the app 123D Sculpt: first put on transparent background, then
make a photo, then go to gallery, at last click on the option send to Photo Library. The pupil did not
find this out by herself, so the designer had to help her do it.
Step 2 was executed without any problem and completely independently.
Execution of steps: step 1 was executed correctly, but the information found was superficial and
limited. It seemed not possible for the pupil to find background information like historical or
anecdotic stories on the basis of the instruction. She was also helped to verify the info of Wikipanion.
Without help she would probably tell a story that was not completely true. Also finding the exact

2. spot for the artwork on the map was not found without help. The story was written in the app
“Notes”, but the pupil did not write the name of the street above it, as was stated in the instruction.
The introduction before step 1 took 15 minutes and step 1 took 1 hour and 15 minutes.
Step 2 was executed independently. The moodboard was filled with photos from Google images and
a title. The pupil was capable of describing the mood she was trying to express. This step took 30
minutes.
Step 3 was executed with a little help of the designer. This was also because of the time. With
exclusion of the saving, she would have been able to find out the app herself. This step took 45
minutes.
Step 4 was executed with a lot of help. The text of the instruction was unclear to the pupil, and she
could not remember the actions she had to take in 4b. She was also confused about the icons and
slides when she read the instruction. The designer had to give a hint that she could resize and rotate
the artwork after placing it. The last part, scanning the image to make the artwork appear, was not a
problem. The result however, according to the designer, was disappointing. Publishing the artwork
on a A4 paper is far more dull than outside on the real location. This step took 15 minutes, but the
presentation part was not executed.
Effectiveness
Expected effectiveness: will our product reach its goals?
Our main goal is citizenship. This goal was not reached. The information about the location was very
superficial, and without help it could even have been incorrect. Engagement could have been
reached by the stories behind the location, but these stories were not revealed by the actions taken.
This also caused that the pupil did not like this step: she did a lot of effort and the outcome was
uninteresting. Probably our target group pupils (12-15) are more skilled in finding relevant
information, but the instruction does not support better or deeper investigation. The app Wikipanion
was too limited.
The pupil did develop technical skills: the apps, saving pictures from google, copying text, saving
objects to the photo library, and artistic designing on the ipad. She also learned to read a map, but
she did need help for that. This pupil is at least 3 years younger than our intended target group, but
she is skilled above average with computers, and she is good at text interpretation. It is expected that
the technical skills she learned could also be learned by first year secondary schools pupils, but this
depends on their experience with tablets. Perhaps most of them already know all this: the starting
level of the target group should be tested if this technical goal is considered to be important.
The creative/art goal will probably be reached. The classical part of the lessons give room for
explanation and group discussion about art. The process of making art by finding a story, through
making a moodboard to creating an artwork were well understood by the 9 year old pupil. Therefore
it is to be expected that the target group will understand this as well.
Design implications

3. What we will definitely change is step 4b. We decided that this should be done in pairs, so one pupil
can read the instruction and the other can execute this in Aurasma. This is logic, because testing if
others can scan your artwork will also be done in pairs. We will also remove the mail link from the
instruction. We will also change the introduction text, probably in the teacher manual, but maybe in
the pupil instruction.
Perhaps we will change the instruction of step 3, and give a hint how to save an artwork of 123D
Sculpt. It is not sure whether pupils who are older, and read some English, will be able to find it out
their selves with peers. This specific aspect should be tested with the target group.
We will not change the instruction of the first step. Although it is far more efficient to go to the real
location to combine information of the internet with the real life, this is not possible yet. Internet
connection is needed outside, and schools have Wifi only tablets. In the future, this should be
changed, because it really improves the quality of our design. Also for the end result (scanning the
artwork by other students) working outside on the real location is much more exciting.
We will discuss the possibility to put in again the interview. We had removed it before, because this
takes a lot of time to film and edit. But we could skip the filming, and just add the interview to gain
information. Another disadvantage was the fact that interviewing is scary at this age, but you could
allow interviewing someone pupils know well, or doing it in pairs.