Cheap& Quick Internal user testing

Cheap & Quick Internal User Testing
- Flirtomatic’s Recent Challenge -

3 Dec 2008

Jaeseung Chang (jaeseung@flirtomatic.com)
Usability Architect @ Handmade Mobile Entertainment Ltd.

1
Mobile Design UK © 2008 Handmade Mobile. All Rights Reserved.

Quick Introduction

- BA in Education from SNU
- MSc in Information Technology from QMUL
- PgDip in HCI from UCLIC
- MSc by Research in Computer Science from QMUL

- Open Education (Korea) / Usability Designer
- Samsung Electronics (Headquarter in Korea)
/ Mobile Usability Designer + Researcher
- Handmade Mobile (Flirtomatic) / Mobile Usability Architect

2

– Flirtomatic’s Recent Challenge

1. Why Cheap & Quick?

2. Experiment Design

Good Lessons – Dos and Donts
3.

3

1.1. Evaluation Needs

- We all know the necessity of usability evaluation
- At least 2 times per 1 iteration cycle
- But the reality sometimes doesn’t help

Requirement Fast
Analysis Prototyping

Evaluation Evaluation

Specification
Implementation

4

1.2. Usability Firms

- Hiring professional research firms?
- Excellent facilities, professional experiment design,
reliable recruitment, & nice report (maybe?)
- But costs a fortune!

5

1.3. Self-Evaluation

- Then we should go DIY
- Venue: How about using our meeting room?
- Experiment design: We can do that maybe?
- Recruitment: Find some volunteers from our users?
- Observation and analysis?: 15 day trial version MORAE!

6

7




3.

8

2.1. Overview

- The 1st iteration of our ‘Redesign Project’
- Based on wireframes (workflows) produced so far
- Evaluation on our first redesign UI/IA works
- To discover good/bad points for more future works
- Included three parts: UT + GUI Preference + FGD

+ +
GUI Focus Group
User Testing
Preference Discussion

9

2.2. User Testing

- Normal user testing techniques (think-aloud)
- Went through 6 sample tasks with 8 participants
- Used a hi-fi prototype to simulate our new UI
- Controlled experiment with video recording (MORAE)
- 1 moderator and multiple observers

CONFIDENTIAL
CONFIDENTIAL

10

2.3. GUI Preference

- Asked the same participants to choose our new GUI designs
- A sort of multiple-choice survey questions
- Asked to choose the best 2 & the worst 1 design

CONFIDENTIAL

11

2.4. Focus Group Discussion

- To collect real users’ preferences and behaviours
- In terms of using the current version of Flirtomatic
- Focused on gathering additional information
- To help redesign the details of our WAP service
- Went through 5 topics with 3 groups of users (m, f, & gay)

12




3.

13

3. Good Lessons – Dos and Donts
3.1. Outcomes

- Were able to conduct evaluation ourselves – DIY!
- Captured some important findings to validate our prototype
- Less than 2 wks (Experiment design + experiment + analysis)
- Less than 500.00 GBP
- Much quicker and cheaper than hiring research firms
- Were able to share the importance of usability evaluation
with the management level and all other departments
- Self-learn about the process of usability evaluation

14

3.2. Difficulties

- Recruitment error: No show up, recruited friends, etc.
- MORAE error (due to the inexperienced operation):
No video recording, low quality audio recording, etc.
- Some mistakes of our colleagues:
intruding into the UT room to connect to wireless network,
interrupting UT sessions to ask questions to the moderator,
giving money to participants during the session, etc.

15

3.3. Dos

- Good preparation in every way:
1. Documentation on task scenario, forms, analysis
2. Train yourself to master any SW tools (like MORAE)
3. Clean up the venue in advance
- Share the importance of usability evaluation with everyone
1. With the management level
2. Especially with colleagues nearby the UT room
(should politely ask silence during each session)
- Share the formal process of evaluation with team members
- Educate yourself how to run each session professionally
16

3.4. Donts

- Do not rely on the quick list of recruited users
(importance of having some back-up participants)
- Do not expect everyone else in office will behave as you do
1. They don’t know how important it is
2. Actually they don’t care what you do
- Do not expect your team members understand everything
1. They don’t know how to formally run each session
2. Importance of pre-training for usability evaluation

17

3.5. Resources

- MORAE: http://www.techsmith.com/morae.asp

- The Usability Methods Toolbox by James Hom:
http://jthom.best.vwh.net/usability

- Usability First: http://www.usabilityfirst.com/

- Jakob Nielsen: http://www.useit.com/

18

Thank you

Jaeseung Chang
(jaeseung@flirtomatic.com)

19

Cheap& Quick Internal user testing

Recommended

Recommended

More Related Content

What's hot

What's hot (19)

Viewers also liked

Viewers also liked (18)

Similar to Cheap& Quick Internal user testing

Similar to Cheap& Quick Internal user testing (20)

Recently uploaded

Recently uploaded (20)

Cheap& Quick Internal user testing