1. Team 12: meMem
Adam Faeth
Dave Watkins
Sujoy Kumar Chowdhury
SD4: It's expensive
Summary
We set out to create a prospective memory aid that would help people over the age of 55 stay
active by scheduling appointments, and organizing lists of information that they may need to
recall. While the users vary in experience and abilities, we noticed similar needs among our users.
Among the nine people we interviewed, we found that they all used a paper-based notebook
or calendar to keep track of their schedules, appointments, notes, and passwords. Although our
users reported that they did not believe they needed a memory aid, several users reported
anecdotes of breakdowns in their paper-based solutions. Sometimes an event was recorded on the
schedule without any context ("6:00" written on the day without an event), other times
they inadvertently scheduled simultaneous meetings.
This report will present the paper prototype that we took to users and the feedback that
we received from the users. The first section details the prototype that we created based on the
interviews and analysis conducted for previous assignments. After designing the prototype, we
collected responses from three users. In the discussion section, we summarize the surprising
responses and our reactions to what we learned about user evaluation from the evaluation of this
prototype.
Prototype
We created the meMem prototype, as a low-fidelity prototype in order to focus the
users responses on the design of the interaction rather than details of the design itself such as
color choices. The prototype is paper-based, with an emphasis on hand-drawn elements. The
paper prototype presented a small challenge in distributing to team members, so we chose to scan
the drawings and clean them up in an image editor for distribution to the rest of the team. This
allowed some refinement of the drawings and duplication of similar interface elements while
maintaining the overall simplicity of a sketch.
The meMem prototype consists of two devices, the wall calendar and the portable device. The
wall calendar is the size of an A3 sheet of paper (approximately 11x17"), and is designed for use
on the wall. The portable device is the size of an A6 sheet of paper (approximately 4x6"). We
decided to use two devices because it seemed like each device could serve a separate role, and
many of the peer evaluations we received liked the alternative that consisted of two devices.

2. Figure 1: Wall calendar paper prototype
The wall calendar consists of a vertical row of nine buttons on the left side, and a large touch
screen area. The prototype also contained a microphone and speaker for speech interaction, and a
fingerprint scanner for biometric authentication.
Figure 2: Touch screen keyboard for the wall calendar prototype

3. Figure 3: Add appointment screen
The screen to schedule an appointment would appear over the entire touch screen area covering the whole
calendar of the previous view. There is also a slip containing the hours of the day that slides through slots
in the paper to simulate scrolling.
Figure 4: Today view on the wall calendar
The today view shows the user upcoming appointments from their day and items on their to-do list. This
screen would also appear over the entirety of the touch screen area on the wall calendar prototype when
the user presses the Today button.

4. Figure 5: Paper prototype of the portable (left) and extra cutouts (right)
Most of the portable device is also a touch screen. Five physical buttons on the top and bottom of the
screen allow quick access to the functions of the device. The portable also supports speech interaction
through a built-in microphone and speaker, or a Bluetooth headset. Some of the other elements that can be
cut out and used to highlight changes on the prototypes are also shown on the right. For example, when
the user presses one of the buttons at the top of the portable device, or issues the equivalent speech
command, a LED on the button would light up giving the user feedback about their action. The portable
device also combines the three plus-buttons to add appointments, notes, and to-do items into one plus
button.

5. Figure 6: Screens from the portable device
The prototype has several screens that were cut out and laid over the touch screen area. These screens
show the notes storage interface to the portable (upper left and lower right), the screen presented after the
user presses the (+) button (upper right), and the today view of the portable device.

6. Figure 7: Additional screens from the portable memory aid
The next set of screens depicts how the device would prompt the user to authenticate with the fingerprint
scanner (top left), view a two-column list (bottom left), and schedule an appointment (right). Once again,
the slip of paper with the times on it can slide through the slots in the paper prototype.

7. Evaluation Methods
Approach
To evaluate the meMem paper prototype, we selected five tasks that represented a cross-section of the
tasks that the user might perform with the memory aid. In SD2, we outlined a range of three tasks that
demonstrated the interaction of a memory aid. The tasks were of scheduling a new event, retrieving a
password, and viewing a snapshot of today's appointments and tasks. Since the prototype consists of two
devices, we decided to test these tasks on each device. However, we dropped the password retrieval task
from the Wall calendar because we thought that the user was much more likely to use the portable for this
task, and that the interaction would be largely redundant on the wall calendar. This resulted in a series of
five tasks across the two devices.
For each task, we decided to record the number of steps and the number of errors that occurred while the
user completed the task. The measurement of the number of steps was suggested in class by Dr. Gilbert as
an alternative to measuring completion time because time measurements are problematic with a paper
prototype. We defined an error as any step that did not bring the user closer to accomplishing the goal,
and recorded the total number of errors separately from the total number of steps.
In addition to the quantitative measurements, we recorded the qualitative evaluation of the prototype by
our users. We decided to use Think-Aloud, post-task discussion, and a post-evaluation interview to gather
the qualitative evaluation of our prototype. Before starting the tasks, we asked each participant to tell us
what they were thinking as they worked through the tasks. If they paused, we prompted them to keep
talking. If there were any interesting comments or interactions, we decided to ask them for clarification
after the task. Finally, we created a series of short interview questions to ask after all the tasks were
complete to solicit further comments from each participant. We decided to use these three methods
because we did not want to trust one method of qualitative data gathering. The Think-Aloud protocol can
sometimes break down when the user stops talking. In post-task or post-evaluation discussion, the user
might give an incorrect reason for why they did something, because they forgot or did not think about it.
We felt that the combination of qualitative methods, combined with the quantitative measurements would
provide a more accurate evaluation of the prototype.
Users
Our users have one thing in common, they have a minimum of 55 years of experience; it is a diverse
population that continues to grow. They face unique challenges including memory loss, vision changes,
hearing loss, and other general health concerns. Due to these health concerns they are encouraged by
health professionals to become organized, set goals, eat healthy, exercise regularly, and be active. This
group has a variety of skills: typing, organizing, speaking, leading, dancing, and teaching. They are
experienced with devices such as phones and alarm clocks, however they are appreciative of computers.
Most have basic computer experience that includes checking email, editing documents, and searching for
information.
Script
Before meeting with your participant, modify:
• The portable "today" view to reflect the date of the interview
• The wall "today" view to reflect the date of the interview
• Position the highlight cell on the wall calendar on today's date
• Change any dates on the task cards
Thank you for agreeing to help us test a prototype of a memory aid. The prototype consists of two
devices, one that would be hung like a wall calendar, and the other is a portable device that you could
take with you. We will present you with a paper prototype of each device and ask you to complete a set of
five tasks. Remember that we are testing the prototype, not you, so please point out any difficulties or

8. frustrations with the prototype. We would also like you to think-aloud while you perform the tasks, so
that we may gain a better understanding of your interaction with the device. If you have a question at any
time, please feel free to ask. You are also free to withdraw at any time.
The wall calendar (meMemCal):
• Has nine physical buttons on the left side (you can trace the outline of each button, but do not tell
them the function of each)
• The entire calendar area is a touch-screen
• The calendar also has the ability to recognize voice commands when you turn on that ability
The portable device (meMemPod):
• Has five physical buttons
• The rest of the area is a touch screen
• Has a fingerprint reader (do not indicate where)
• The portable also has the ability to recognize voice commands when you turn on that ability
For the researcher:
• Present each of the task cards one at a time
• Read the task scenario
• Give the user the task card with the goal written on it
• DO NOT tell the user how to accomplish the task
• Prompt: If the user stops talking, say "keep talking please" or something similar
• Step: Record each button press, voice command, and touch screen touch as a step
• Error: Record each step that does not bring them closer to achieving the goal as an error
• Write: How the user accomplished the task, the number of steps and errors
I would also like to ask you a few questions about the prototype.
Give interview
Thank you for taking the time to help us to evaluate this prototype today.
Tasks
Task 1
User Scenario:
You are starting to make breakfast and you are trying to decide if you want to make a pot of coffee to go
with it. You have a nagging feeling that you are supposed to meet someone for coffee later, so maybe it's
not worth making now. You want to quickly find out what time the meeting is before the toast pops up.
Goal: Determine what time you're meeting Adam today using the wall calendar
Task 2
User Scenario:
You are sitting in your favorite comfortable spot at home when you receive a call from your good friend
Dave. He asks if you would be free to go to dinner this evening at 6:00 p.m. You need to add the
appointment quickly because the call from Dave reminded that you wanted to get to the copy place on the
way to the doctor's appointment. Use the wall calendar to schedule this event on your way out the door.
Goal: Schedule Dinner with Dave at 6:00 p.m. on Friday, November 21st using the wall calendar.
Task 3

9. User Scenario:
The copy shop was just finishing your fliers when you walked in the door, and they are putting them in a
box for you. You just have a few minutes before your doctor's appointment, but luckily it is just down the
street. You want to cross off that you'd picked up your fliers.
Goal: Cross off the reminder to pick up the fliers using the portable device.
Task 4
User Scenario:
You just finish a regular doctor's appointment and they want to schedule your next appointment six
months from now on May 29th, 2009 at 10 a.m. Rather than wait for their friendly post-card reminder, use
the portable calendar to put this appointment on your schedule while you are still at the doctor's offices.
Goal: Schedule the doctor's appointment for 29 May 2009 at 10:00 AM using the portable device
Task 5
User Scenario:
You just got an intriguing email from your friend with a link to an article in the New York Times about
how the economy is bouncing back and we'll all be rich soon. This must be good and so you are
determined to take a look. However, the New York Times requires that you log in to view the article, and
you've forgotten the password to your account. Use the portable device to find your password.
Goal: Retrieve the password for your New York Times online account using the portable device
Post-evaluation Interview
1. What are your overall impressions of the prototype?
2. Are there any additional thoughts or suggestions that you would like to share?
3. How did you prefer to interact with the device, through the speech interface or the buttons?
4. Would you prefer a default reminder before each event or to customize the amount of time between
each event and its reminder?
5. How long before an appointment do you like to be reminded?
a. 15 minutes before event
b. 1 hour before event
c. 1 day before event
d. 1 week before event
6. How much are you ready to pay for this device?
Testing Environment
The evaluations were conducted in environments that were similar to the environment that we would
expect the user to need the device: in their homes or meeting friends. Two users conducted their
evaluations at their homes, where they were comfortable, and free from distractions. A third evaluation
was conducted at a bustling, noisy coffee shop, a favorite of the participant. There was an interruption of
that user during the task when his nephew phoned him during the completion of one of the tasks, but
since we were not measuring time to completion and he remembered where he left off, we did not have
him repeat the task.
Results
We asked three users to evaluate our prototype memory aid, and recorded the number of steps they
required to complete each task and the number of errors that occurred for each user. If a user's action did

10. not bring them closer to completing the goal of the task, we recorded that action as an error rather than a
step.
There was a contrast in the overall technology use of our participants. User 2 and his wife continued a
conversation after the evaluation where they discussed how they dislike the effect of computers on
society. They brought up two general points of frustration. First, they pointed out was how devices,
screen sizes, and the font of instructions that accompany those devices continue to get smaller. User 2
pulled out a piece of paper that explained the function of his remote control, which was about the size of
that same remote. He said it was difficult to read, and so he had written notes on top of it for him to
reference later. Second, he and his wife also expressed frustration with the negative effects of computers
on billing mistakes and transactions. Referencing a conversation with a friend, they could not believe it
when their friend said they could not live without their Palm Pilot. User 3 clearly held the opposite belief,
he said he was always open to new technologies. User 1 also has a more positive view of computers,
moving several group calendars of organizations he belongs to online, and maintaining seven blogs on a
variety of topics.
User 1 User 2 User 3
Task: Steps Errors Steps Errors Steps Errors
Task 1: View today (wall) 2 0 2 1 1 0
Task 2: Schedule appointment (wall) 4 0 4 3 6 4
Task 3: View Today (pod) 3 1 3 1 2 0
Task 4: Schedule Appointment (portable) 5 1 5 1 7 1
Task 5: Retrieve password 4 5 4 0 3 0
Table 1: Qualitative user evaluation of the prototype
In the description of the system, we indicated it was possible to turn on a speech recognition interface.
The first user primarily used speech recognition to interact with the prototype. The first button that User 1
pressed enabled the speech interface. After that, he only pressed a button or the touch screen of either
device once. The third user alternated between the speech and button interfaces within the tasks. The
second user did not use the speech interface for any part of the tasks, however, he commented after the
evaluation that he would love to use the speech interface because he had difficulty typing.
Most of the commands spoken by the users brought them one step closer to achieving the goal of the task.
However, they did issue some interesting commands that we reprinted in Figure 8 because they give an
idea of the kinds of speech interactions the user might like to have when viewing or adding events.

11. The most complicated tasks were Tasks 2
Viewing an event (Task 1):
and 4, which asked the user to schedule an
"Computer, show me coffee w/Adam"
appointment. One user required a hint
"Computer, show me where"
before they were able to complete Task 2.
"Computer, who is Sujoy?"
User 3 got stuck and we reminded the user
"What am I'm calling Sujoy about?"
that the speech interface was still on, and
listening to them. User 3 also scheduled
Adding an event (Task 4):
the event a half-hour early. When we
"Appointment please"
asked why he did this, User 3 responded
"Make an appointment"
he wanted to leave a buffer before the
"Do appointment"
event. We did not categorize this as an
"Remind me at 4:00"
error because we think this is a matter of
"Remind me 24 hours before the appointment"
individual preference rather than an error
"List doctor's phone number" (add the phone
in completing the task. Later, when User 3
number of the doctor to the event details)
was working on Task 4, he expected to
"Email him one week in advance to confirm"
have to authenticate before adding an
event, and after some time passed, he
Figure 8: Sample of Speech interactions that the
understood that no authentication was
prototype does not yet support
required.
In contrast, User 1 did not encounter many errors with the scheduling tasks, but struggled with Task 5.
This task asked the user to retrieve a password from the portable device. User 1 issued a sequence of
consecutive speech commands that the device would not be able to interpret that is reprinted below.
After a pause, the investigator asked him
1. "Retrieve password for New York Times."
what he was looking for. He responded that
2. "Computer, open New York Times."
the device should be smart enough to just
3. "Computer, open Passwords."
input the password when he followed the link
4. "Computer show me my <expletive> password!"
to the New York Times article from his
5. "Computer, why don't you understand me?"
email. He then said that since the device was
obviously just a dumb calendar, he would Figure 9: String of consecutive speech commands
have stored his passwords as appointments on that resulted in errors for User 1
the date of his birth. The investigator asked
him if he saw another way of storing passwords on the device. User 1 then said, "Oh, there's a notes
button." He did not press the button, but instead issued a sequence of voice commands that accomplished
the task with no further errors.
Sometimes the design did not match the conceptual model of the user, and this usually resulted in
additional or redundant button presses. After we presented User 2 with Task 1, he pressed the calendar
button on the wall calendar and expected it to do something, but since the device started with the calendar
view, the button did not do anything. Similarly, User 3 issued the voice command, "Notes," while
working on Task 3 on the portable device. The device presented him with the notes stored on the device,
and illuminated the physical Notes button at the top of the device. He then pressed the physical Notes
button, which resulted in no effect.
The button labels also confused our users when they tried to accomplish the tasks. User 2 did not
recognize the 'add appointment' button on the wall calendar prototype, so he tried several other buttons
before he found it and was able to complete Task 2. The same user also did not recognize the 'plus' button
on the prototype as what he needed to use to add appointments, and tried many of the buttons labeled with
words before discovering the plus button. User 3 also commented after the evaluation that he did not
notice the 'plus' buttons on either device and did not understand the difference between the Notes and To-

12. do buttons on the wall calendar.
Post-Evaluation Interview Results
The post-evaluation interview asked for overall impressions of the prototype, how the users would like to
interact with the device, how they like to be reminded of appointments, and how much users would be
willing to pay for the device.
All three users believe the overall concept was good. They felt the speech interface was easier than
dealing with buttons. User 3 suggested the Notes and To-Do functions should be merged together. User
2 was interested in making sure button sizes and text was large enough. In addition to these
suggestions, one major issue brought out by User 2 and 3 were the plus signs on the buttons. They both
did not equate the symbol with the function it performed.
For the alerts, User 3 wanted to make each field for the date editable while the other two users did not
give any real response. User 3 and User 1 expressed a preference to be alerted 15 minutes prior and one
week prior to an event, respectively.
In the same manner that all the users agreed this was a good idea, they all agreed the device should be low
in cost. How they determined their buying-price varied. User 1 picked $100. User 2 tried to think of a
comparable device (palm-pilot), and User 3 reasoned things out in terms of cost of the product after it's
initial introduction into the market where he decided the device should be about $25 to $75.
Discussion
The largest source of errors appeared to be a mismatch between the conceptual model of the user and the
model of the device that we created. When User 1 struggled with the sequence of commands to display
the password in Figure 9, he was either expecting the device to be too smart or too dumb. He expected it
to be smart enough to recognize that he was asking for a list of passwords, and that it would realize that it
had a list of passwords stored as a note. Or he expected it to be too dumb, that the list of passwords was
hard-wired as a specific command. In reality the device was designed to be flexible, and the designers had
created a list of passwords as a note. The device was listening for the word 'note' or a derivative before it
could recognize that one of the notes was called "Passwords." It is interesting that this hierarchy was
created to provide flexibility and resulted in a model that was difficult for the user to create because their
expectations were both that it was either smarter than that or hard-wired for specific commands.
Other mismatches occurred when the User 2 and User 3 pressed the buttons corresponding to the mode
that they were already in or did not recognize the function of a button from its label. It is interesting that
the redundant button presses also occurred on the portable device, which gave the user feedback by
illuminating a light on the 'Notes' button. While the users struggled to recognize the button labels, a real
device might have a diagram or manual explaining the different parts of the device. It was interesting to
test the device without this kind of prior information, and this gave us an idea of how easy the device
would be to pick up and use, rather than how well the user could learn to use the device. In order to better
improve the experience of first time users, we might want to include even more feedback from the
interface to help the user understand whether their action was valid or invalid.
For the users that found the speech interface, it appeared that they learned how to use it. User 3 saw the
speech button illuminate when he pressed it, and remarked that it was similar to an on-off light switch.
User 3 also changed the pattern of their speech commands; his early commands gave the impression that
he was talking to an imagined third person in the room, and the later commands were direct commands to
the device. In contrast, User 2 said that he would love to use the speech interface instead of typing; this
suggests that he did not understand that the device could respond to his speech commands. This is

13. interesting because the users were presented with the same description of the device, where possible.
One difference in the description of the device occurred when the investigator let it slip not only that the
device included the ability to recognize the users fingerprint, but also indicated the location of the
fingerprint scanner. This may have caused User 2 to attempt a fingerprint scan before starting several of
their tasks. However, User 3 also tried to find a method to authenticate before starting some of their tasks,
without this additional information. This makes it unclear whether the users had a preconception that they
had to authenticate before using the device, or that the indication of the fingerprint scanner caused them
to try to use it before using the device.
A surprising result was that the paper prototype appeared to provide structure to the speech interaction.
We did not provide the users with a list of speech commands that they could use, yet the users largely
seemed to grasp the concepts that the prototype would recognize from the tasks or the depiction of the
device itself. While it is possible that the users picked up some of the concepts from the task cards, we did
try to avoid writing anything that would give away the commands required to perform the task, like
avoiding the word "Today" on the tasks that would require the user to press the Today button. After
making a number of errors on Task 5, User 1 took another look at the device and recognized that he
needed to talk to it about 'notes' to reach the password list. This suggests that the depiction of the device
had some influence on the speech commands issued by the users.
The paper prototype method did work to focus the evaluation from our users on the interaction itself
rather than arbitrary design choices like color palette. However, the paper prototype did also have
drawbacks. It was difficult to convey feedback to the user when the device would use animation, sounds,
and vibrations to provide feedback. Sometimes we forgot to provide the feedback after an action,
especially if it was the result of a sequence of actions that we had not anticipated.
It was also a small challenge to use the Think-Aloud protocol with a prototype that contained a speech
interface. We came up with a couple of workarounds like only enabling the speech interface with a
button. Another method of resolving issues with the speech interface and Think-Aloud was to stop the
user, reach for the appropriate piece of paper, and repeat their command, "When you said November 24th,
the device changed to show you this."
Another surprise was that most users completed the tasks in fewer steps than the number of steps
indicated in the Hierarchal Task Analysis performed for SD2. While the sample size is very small, this
does point to some accuracy in both the HTA from SD2. It also suggests that the prototype did not
introduce many additional steps into the sequence required to complete the tasks. While it was not
something we intended to use to evaluate our prototype, it was interesting to be able to compare the
results to our design. It appears to be a useful method of evaluating whether a prototype introduces
unnecessary steps in the completion of tasks.
The main changes that the evaluation suggested for the design is to try another set of button labels. It
appeared that users had difficulty in recognizing the purpose of each of the buttons, except for the speech
interface button. We could also use a better method to convey the purpose of the device for list storage.
Most users seemed to regard it as a calendar, and had trouble understanding the list-storage function of
the memory aid. It also seems that none of our users would buy the device as prototyped, unless they
could obtain both devices for under $100. A redesigned prototype might try to make use of existing
devices that are approximately the size of the wall calendar and the portable, such as a television or an
iPod touch to save money. It also might be possible to use either the wall calendar or the portable, but it is
not a question that we asked our users.
The project was an interesting exercise in obtaining user evaluations. The paper prototype was successful

14. in testing the interaction rather than specific design choices, however we did have the feeling that some
users were still holding back criticism or trying to be polite. User 1 remarked, "Oh, you must have spent a
lot of time on this," when he saw the pieces of paper arrayed on the table. The evaluations were still
useful in pointing out areas that need improvement, as there were plenty of statements that we did not
expect. We learned a lot about the users reaction to the design with just the minimal investment in the
paper prototype. Paper prototyping and user evaluation seem like an effective combination for obtaining
early feedback on risky or new ventures. This is the first venture of our own education in obtaining honest
and useful feedback from user evaluations, and it is an area where there is a lot of nuance to master.