There are two main approaches to user evaluation - qualitative and quantitative. Qualitative involves observing users and asking open-ended questions to understand their perspectives, while quantitative focuses on collecting numeric data through closed questions and analysis. Both approaches have benefits and challenges. Effective evaluation combines multiple techniques like playtesting, think-aloud protocols, and card sorting to iteratively improve the user experience based on direct feedback.

1. http://www.comu346.com [email_address] Game Design 2 Lecture 7: User Evaluation 2011

7. Can you see any difficulties in receiving data like this?

8. Open Ended Questions

13. Closed Ended Questions

14. How would you redesign the question as a closed ended one so as to avoid the problems identified earlier?

24. Think Aloud/Online Self Report “ Think aloud technique is pretty much what it sounds like. You ask someone to do a task, and to think aloud about what they are doing while they are doing it.” - Rugg, 2007

25. “ The basic concept is simple: you tell the respondent what the task is, and ask them to think aloud while doing it. If they are silent for more than a set length of time (e.g. five seconds) then you use a prearranged prompt to get them talking again” - Rugg, 2007   “ Could you tell me what you’re thinking about now?” “ Are you looking at the background of the picture?” “ Could you tell me what you’re thinking about now?” “ Are you looking at the background of the picture?”

26. “… if you have to transcribe the data, then this can be very time-consuming (in the order of ten hours of transcription per hour of tape, depending on how good your typing is and how loquacious your respondents are).” - Rugg, 2007

27. “ the convention on transcripts is to use one full stop per second of silence (so “….” shows four seconds of silence). “Um” and “er” sounds are also worth noting, for the same reason, particularly when the respondent is otherwise articulate.” - Rugg, 2007

31. Example superordinate grouping Super-ordinate grouping performed by an Independent Judge “ Your task is to interpret the criteria into super-ordinate constructs. You should try to identify where the criterion given by one respondent could be said to have meant the same as another but simply have chosen different wording.”

32. Co-occurrence matrix