Eye tracking in usability studies
•
2 likes•744 views
A brief description of why and how to use eye tracking in usability studies. Pros and cons, what to consider when designing the study, analysing the data, and writing a report
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (8)
Similar to Eye tracking in usability studies
Similar to Eye tracking in usability studies (20)
Recently uploaded
Recently uploaded (20)
Eye tracking in usability studies
- 1. EyeTracking in usability studies
- 2. Agenda • Eye tracking as research method • Designing the research question • Setting up the experiment • Conducting the interview • Cued vs. non cued RTA • Data analysis • Report
- 3. Eye tracking in usability studies • Unco nscio us cognitive processes TA is not sufficient for verbalizing⇾ thought process • Instinctive and subco nscio us behavior at first meeting with website • Eye movements are not easily interpreted without participant providing context to data
- 4. The Research Process CTA vs. RTA Atmosphere Honesty Body Language Framing Ill posed questions Qual. vs. qant. data? Sorroundings Calibration Live transmissions Deception (demand effects) Cued vs. non cued Interest vs. confusion Eye-mind hypothesis Perceptual span and AOIs Information overload Easy to dazzle
- 5. Research Questions (Yarbus: The Visitor)
- 6. Experimental Setup • Glasses/hard lenses vs. no glasses/lenses (is the sample still even?) • Calibration • Avoid demand effects ask questions the right way⇾ • Several short tasks • Location (lighting, distracting elements ) • Fixation cross (mainly in choice experiments)
- 7. Data Collection • We’re testing the website, not the skills of the user • The person conducting the interview does not have any personal interests in the client • Shy/nervous users body language⇾ • Pleasers/demand effects • Yes/no and leading questions are a no-go • Ratings after each task
- 8. CTA vs. RTA Two ways to conduct a think aloud interview: 1)the participants are asked to verbalize their thoughts while they are doing tasks = concurrent think aloud (CTA) Distractions, disruption of intuitive/natural interaction, user might look away from screen 2)the participants provide a description of their experiences doing the tasks after each or all of the tasks are completed = retrospective think aloud (RTA) Users rely on memory, risk of fabrication
- 9. Cued vs. non cued RTA No cued RTA - No pictures or video during RTA - User must rely on memory only - Few words (depending on the user) Video cued RTA -Video replaying ations - Stimulates memory - Helps identify usability problems - Reduces risk of fabrication Gaze plot cued RTA -Superimposed eye movements on pictures - Stimulates memory less than video cue (no interaction replay) - Layout and navigation comments Gaze video cued RTA -Superimposed eye movements on video - Shows all tasks and interactions - Cognitive reflections - Produces most words
- 10. Data Analysis • Heatmaps accumulate all fixations, short or long ⇾ Eye mind hypothesis ⇾ What has been seen vs. usability problem • Global viewing vs. local viewing ⇾ how engaged is the user • AOIs; when and for what? ⇾ Clear, specific questions ⇾ Perceptual span ⇾ False positives/false negatives • Data generating process ⇾ Bottom-up (emotional, saliency) vs. top-down (task driven) Opacity map
- 11. ”The design is clumsy and doesn’t match my needs.” R8 Seg. 2 Report • Information overload • Heatmaps and gaze plots give ideas and assumptions – charts show hard facts • Include quotes (positive, negative and neutral) • What they did/where they stumbled ”It seems easier than what they offer now. More logical” R13 Seg. 1 “It’s not revolutionary, but pretty mainstream and up-to-date.” R10 Seg. 1
Editor's Notes
- - Think aloud processes may not be sufficient since certain cognitive processes are unconscious and participants may not be able to adequately verbalize their thought process - people in general have good memory of what AOIs they have looked at, but they do not necessarily remember the order in which they focused on each one, nor do they remember seeing a logo even though they had looked at it. - Subconscious/instictive behavior → do they notice, what I want them to notice? - For usability research, eye tracking data should be combined with additional qualitative data because eye movements cannot always be clearly interpreted without the participant providing context to the data When doing user research one can get a lot of information from asking the user different questions while making them use a website or flip through a magazine. The answers you will get from the user, however, will always be their subjective opinion and very often they will do things out of habit which makes it hard for them to really explain what they do or why they do it. Instead they end up post-rationalizing their behavior. This is why we use eye tracking in our studies. By combining the qualitative interview method with eye tracking, you get not only the subjective opinion from the user, but also the objective behavior: you get to see what they were actually looking at often without even knowing it. When doing user research one can get a lot of information from asking the user different questions while making them use a website or flip through a magazine. The answers you will get from the user, however, will always be their subjective opinion and very often they will do things out of habit which makes it hard for them to really explain what they do or why they do it. Instead they end up post-rationalizing their behavior. This is why we use eye tracking in our studies. By combining the qualitative interview method with eye tracking, you get not only the subjective opinion from the user, but also the objective behavior: you get to see what they were actually looking at often without even knowing it.
- Setup: deception (skjuler dagsorden for deltageren) – demand effects
- Avoid ill posed problems (vage spørgsmål) man kan ikke løse problemer hvis ikke de er klart defineret. Er vores hjemmeside god/dårlig? (for hvem, på hvilken måde?) – hvordan øger vi konverteringsraten i denne del til dette segment
- Loation even though in public places it’s easy to recruit, there are too many distractions Skal data bruges kval eller kvant? Der skal flere til kvant
- People get nervous Afraid of not being able to solve the tasks Afraid to offend someone Will I be recognized (if they are customers) As someone watching ud now ” I think, what you want me to do now is...”
- Any cue will provide a higher number of words Eye movement cued RTA methods tend to stimulate participants to make more visual and cognitive comments, while video cued RTA methods produce somewhat more manipulative comments. Overall, using eye movements in combination with RTA proved to be a successful method for learning more about users’ problems with a website. The eye movement cued RTA method provided both qualitative information in the form of the interview comments, and quantitative data from the eye tracker that could be analyzed in the context of qualitative feedback provided by the participants.
- ⇾ E ye tracking data should me supplemented with additional information gained from the participants about their experiences Eye-movement patterns may indicate potential usability problems for the experienced evaluator, a remaining challenge is that the gaze data are hard to interpret by the evaluator without discussions with the user. Kval data behandles Think aloud data + kval opsumme. Af adfærd Hvor leder de efter tingene? Ser de det, de leder efter? Hvor længe kigger de (heatmaps kan være tvetydige) Heatmaps skelner ikke mellem no af fix og lngden af fix kval = længden af fix, heatmap akkumulerer bare alle lange ok korte fix Godt til at identificere hvad man ser, men ikke usability problem 500ms eller over = lang fix Under 250ms = kort fix (scanner hjemmeside) Saccader (sanpaths) fortæller fx om hvor pligtopfylende deltagerne er Global viewing = lange sacc korte fix = overblik, scanner isg gennem en sig Loal viewing= engagement i et særligt område, korte sacc lange fix (fx udfylde en formular) Eye-mind hypothesis fixation duration og working mwmory AOI kun til meget klare og specifikke spørgsmål AOI: de fleste ignorerer det faktum, at folk har en given perceptuel spænvidde (det område, med udgangspunkt i fovia) antallet af grader ud fra fovia hvori du kan genkende objekter perceptual span Læser text 4 bags til venst og 14 til højre for fix har betydning for AOIs str Call to action + andet interessant ved siden af – man risikerer false positives (for store AOI, fixeringer, der ikke tilhører objektet)/false negatives(for lille AOI, man kan have fixeret lidt uden for, men er det indenfor spænvidde kan de alligevel godt genkende dem) Antagelse om perceptuel spænvidde AOI skal gå 5grader rundt om objektet, hvilket er meget på 60cms afstand Data generating process – hvilke kognitive processer får folk til at fiksere – task drevet OG bottom up (emotionelt og salient stimuli) naturlig bias for at kigge på ansigter. Vigtigt til bagefter når man sidder med data og skal danne antagelser
- Man kan nemt drukne folk i data Folk forstår ikke eye traking, man kan nemt forblænde dem Grafer kan være det mest interessante