SlideShare ist ein Scribd-Unternehmen logo
1 von 16
Downloaden Sie, um offline zu lesen
X-Construction Lite: HCI evaluation
                                               Title
                                                             Andy Keavey, Mariana de Rojas-Morao,
                                                       Merve Yildirim, Queen Ajifa Ododo, Tom Hume

 Text




Tom Hume   Human-Computer Interaction, assignment 1
Title

 Text


                                                Screenshot of page from
                                                Android Marketplace




Tom Hume   Human-Computer Interaction, assignment 1
Overview of gameplay




           First screen of instructions (animates)       Second screen of instructions




                    Game-play screen                        Results of play screen


Tom Hume      Human-Computer Interaction, assignment 1
Evaluation approach

 1. Expert review
   1.1. Cognitive walkthrough, written up
   1.2. Evaluation vs Nielsen heuristics

 2. User testing
   2.1. Pre-test questionnaire
   2.2. User test of first level (video recorded)
   2.3. Interview with user whilst reviewing recording
   2.4. Post-test questionnaire




Tom Hume   Human-Computer Interaction, assignment 1
User evaluation




 8 test subjects recruited
  (all Android owners, familiar with UI)
Tom Hume   Human-Computer Interaction, assignment 1
Results of cognitive walkthrough

                                   Issue                                  # reports
      Zoom functions misused                                          ✓✓✓
      Errors placing girders                                          ✓✓✓
      Misunderstanding help messages                                  ✓✓✓
      Unsure how to proceed                                           ✓✓
      Game crashed on start                                           ✓
      Problem starting play                                           ✓
      Confused by pop-up adverts                                      ✓       “I successfully drew some
                                                                             girders, but I struggled a bit
                                                                             because they were going
                         “Some of my girders went out of view at             the wrong direction most
                         the top of the screen. I used the zoom out
                                                                             of the time”
                         gesture consistent with Google Maps and
                         could see a much better view of the task”


Tom Hume   Human-Computer Interaction, assignment 1
Results of heuristic evaluation




                       Agreement                                  Disagreement
       Match between system & real world              User control & freedom
       Consistency & standards                        Error prevention
       Flexibility & efficiency                        Recognition over recall
       Aesthetics & minimalism


Tom Hume   Human-Computer Interaction, assignment 1
Results of quantitative data


                                     300
    Seconds to complete first level




                                     225


                                     150


                                      75


                                        0
                                            0                   1                   2             3            4
                                                                                # attempts to complete first level




Tom Hume                             Human-Computer Interaction, assignment 1
Results of questionnaire data
                Age of test subjects                               Female         Male
   40
   30                                                             25%
   20
   10
                                                                            75%
    0
           8      7    5    4     6    1     3      2


               Estimated vs actual fun                    Android gamer     Not android gamer
    8
    6
    4                                                            38%
    2
                                                                             63%
    0
           1      2    3    4     5    6     7      8
                  Estimated                Actual

Tom Hume       Human-Computer Interaction, assignment 1
Results of user testing

                                   Issue                      # users
      Misunderstood girder placement                    ✓✓✓✓✓✓
      Didn’t understand help/instructions               ✓✓✓✓✓
      Mixed up replaying/restarting level               ✓✓✓✓
      Couldn’t get the train started                    ✓✓✓
      Panned unintentionally                            ✓✓
      Couldn’t tell they were complete                  ✓
      Couldn’t find girder removal tool                  ✓
      Confused by an advert                             ✓
      Misunderstood tool instructions                   ✓
      Couldn’t find instructions again                   ✓
Tom Hume   Human-Computer Interaction, assignment 1
Key issue: misunderstood girder placement




Tom Hume   Human-Computer Interaction, assignment 1
Key issue: mixed up replaying/restarting level




Tom Hume   Human-Computer Interaction, assignment 1
Conclusions

 Good news
 • Most players complete the first level, first time
 • Most players enjoy the game more than they thought
 • Many expert evaluators are now regular players
 Bad news
 • 75% players, 3/5 experts had issues with placing girders
 • 62% players don’t read help, 3/5 experts misunderstand it
 • 50% players mix up replaying and restarting the level
 • 37% players, 2/5 experts have trouble starting their train
 • 20% players pan unintentionally, 3/5 experts had zoom issues


Tom Hume   Human-Computer Interaction, assignment 1
Recommendations

 1. Warn players on the first level when they place a girder which
 doesn’t finish attached to an end-point;

 2. Clarify language in help messages and screens;

 3. Improve the iconography, labelling or placement of the “replay
 failure” and “restart level” icons to reduce confusion between
 them;

 4. Teach players to use the “play” button by animating it on the
 first level when they have a workable bridge;

 5. Allow pinch-and-zoom to zoom into the bridge during play.


Tom Hume   Human-Computer Interaction, assignment 1
Critique of methods




  Hard to record video in sunlight
  Users were self-selecting, to a degree
  Slight inconsistencies in questionnaire usage
Tom Hume   Human-Computer Interaction, assignment 1
Questions?




Tom Hume   Human-Computer Interaction, assignment 1

Weitere ähnliche Inhalte

Mehr von twh

Algorithmic art
Algorithmic artAlgorithmic art
Algorithmic arttwh
 
Mobile apps and live data
Mobile apps and live dataMobile apps and live data
Mobile apps and live datatwh
 
Eduserv 2010
Eduserv 2010Eduserv 2010
Eduserv 2010twh
 
MoMoLondon Demo Night: Guardian Anywhere
MoMoLondon Demo Night: Guardian AnywhereMoMoLondon Demo Night: Guardian Anywhere
MoMoLondon Demo Night: Guardian Anywheretwh
 
Capitalising on Popular Culture
Capitalising on Popular CultureCapitalising on Popular Culture
Capitalising on Popular Culturetwh
 
Mobile Mountains - OverTheAir 2009
Mobile Mountains - OverTheAir 2009Mobile Mountains - OverTheAir 2009
Mobile Mountains - OverTheAir 2009twh
 
Agile2009
Agile2009Agile2009
Agile2009twh
 
Many paths to the top of the (mobile) mountain - Skiff Warmup
Many paths to the top of the (mobile) mountain - Skiff WarmupMany paths to the top of the (mobile) mountain - Skiff Warmup
Many paths to the top of the (mobile) mountain - Skiff Warmuptwh
 
Mobile User Experience
Mobile User ExperienceMobile User Experience
Mobile User Experiencetwh
 
Knocking Down Walls
Knocking Down WallsKnocking Down Walls
Knocking Down Wallstwh
 
Revenue Opportunities in Mobile
Revenue Opportunities in MobileRevenue Opportunities in Mobile
Revenue Opportunities in Mobiletwh
 
Feel the FP-ness
Feel the FP-nessFeel the FP-ness
Feel the FP-nesstwh
 
Knitting Design & Development Together
Knitting Design & Development TogetherKnitting Design & Development Together
Knitting Design & Development Togethertwh
 
Strengths, Weakness and Trade-offs of Mobile Platforms
Strengths, Weakness and Trade-offs of Mobile PlatformsStrengths, Weakness and Trade-offs of Mobile Platforms
Strengths, Weakness and Trade-offs of Mobile Platformstwh
 
A Year of Scrum
A Year of ScrumA Year of Scrum
A Year of Scrumtwh
 
£5 Apps Sanitised
£5 Apps Sanitised£5 Apps Sanitised
£5 Apps Sanitisedtwh
 
Mo Mo Birthday
Mo Mo BirthdayMo Mo Birthday
Mo Mo Birthdaytwh
 
D.Construct 2005
D.Construct 2005D.Construct 2005
D.Construct 2005twh
 

Mehr von twh (18)

Algorithmic art
Algorithmic artAlgorithmic art
Algorithmic art
 
Mobile apps and live data
Mobile apps and live dataMobile apps and live data
Mobile apps and live data
 
Eduserv 2010
Eduserv 2010Eduserv 2010
Eduserv 2010
 
MoMoLondon Demo Night: Guardian Anywhere
MoMoLondon Demo Night: Guardian AnywhereMoMoLondon Demo Night: Guardian Anywhere
MoMoLondon Demo Night: Guardian Anywhere
 
Capitalising on Popular Culture
Capitalising on Popular CultureCapitalising on Popular Culture
Capitalising on Popular Culture
 
Mobile Mountains - OverTheAir 2009
Mobile Mountains - OverTheAir 2009Mobile Mountains - OverTheAir 2009
Mobile Mountains - OverTheAir 2009
 
Agile2009
Agile2009Agile2009
Agile2009
 
Many paths to the top of the (mobile) mountain - Skiff Warmup
Many paths to the top of the (mobile) mountain - Skiff WarmupMany paths to the top of the (mobile) mountain - Skiff Warmup
Many paths to the top of the (mobile) mountain - Skiff Warmup
 
Mobile User Experience
Mobile User ExperienceMobile User Experience
Mobile User Experience
 
Knocking Down Walls
Knocking Down WallsKnocking Down Walls
Knocking Down Walls
 
Revenue Opportunities in Mobile
Revenue Opportunities in MobileRevenue Opportunities in Mobile
Revenue Opportunities in Mobile
 
Feel the FP-ness
Feel the FP-nessFeel the FP-ness
Feel the FP-ness
 
Knitting Design & Development Together
Knitting Design & Development TogetherKnitting Design & Development Together
Knitting Design & Development Together
 
Strengths, Weakness and Trade-offs of Mobile Platforms
Strengths, Weakness and Trade-offs of Mobile PlatformsStrengths, Weakness and Trade-offs of Mobile Platforms
Strengths, Weakness and Trade-offs of Mobile Platforms
 
A Year of Scrum
A Year of ScrumA Year of Scrum
A Year of Scrum
 
£5 Apps Sanitised
£5 Apps Sanitised£5 Apps Sanitised
£5 Apps Sanitised
 
Mo Mo Birthday
Mo Mo BirthdayMo Mo Birthday
Mo Mo Birthday
 
D.Construct 2005
D.Construct 2005D.Construct 2005
D.Construct 2005
 

Kürzlich hochgeladen

Mike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtMike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtTeeFusion
 
Create Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comCreate Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comjakyjhon00
 
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Ed Orozco
 
Embroidery design from embroidery magazine
Embroidery design from embroidery magazineEmbroidery design from embroidery magazine
Embroidery design from embroidery magazineRivanEleraki
 
How to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTHow to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTThink 360 Studio
 
Construction Documents Checklist before Construction
Construction Documents Checklist before ConstructionConstruction Documents Checklist before Construction
Construction Documents Checklist before ConstructionResDraft
 
LRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfLRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfHctorFranciscoSnchez1
 
High-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillHigh-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillCre8iveskill
 
Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...khushisharma298853
 
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLMath Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLkenzukiri
 
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Ted Drake
 
Cold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxCold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxSamKuruvilla5
 
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxWCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxHasan S
 
The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024Alan Dix
 
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Amil baba
 
Designing for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsDesigning for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsBlock Party
 
UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024mikailaoh
 
Production of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxProduction of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxb2kshani34
 
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfBuilding+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfsaidbilgen
 

Kürzlich hochgeladen (19)

Mike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy ShirtMike Tyson Sign The Contract Big Boy Shirt
Mike Tyson Sign The Contract Big Boy Shirt
 
Create Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.comCreate Funeral Invites Online @ feedvu.com
Create Funeral Invites Online @ feedvu.com
 
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...Design mental models for managing large-scale dbt projects. March 21, 2024 in...
Design mental models for managing large-scale dbt projects. March 21, 2024 in...
 
Embroidery design from embroidery magazine
Embroidery design from embroidery magazineEmbroidery design from embroidery magazine
Embroidery design from embroidery magazine
 
How to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPTHow to use Ai for UX UI Design | ChatGPT
How to use Ai for UX UI Design | ChatGPT
 
Construction Documents Checklist before Construction
Construction Documents Checklist before ConstructionConstruction Documents Checklist before Construction
Construction Documents Checklist before Construction
 
LRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdfLRFD Bridge Design Specifications-AASHTO (2014).pdf
LRFD Bridge Design Specifications-AASHTO (2014).pdf
 
High-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkillHigh-Quality Faux Embroidery Services | Cre8iveSkill
High-Quality Faux Embroidery Services | Cre8iveSkill
 
Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...Khushi sharma undergraduate portfolio...
Khushi sharma undergraduate portfolio...
 
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOLMath Group 3 Presentation OLOLOLOLILOOLLOLOL
Math Group 3 Presentation OLOLOLOLILOOLLOLOL
 
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
Introduce Trauma-Informed Design to Your Organization - CSUN ATC 2024
 
Cold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptxCold War Tensions Increase - 1945-1952.pptx
Cold War Tensions Increase - 1945-1952.pptx
 
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptxWCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
WCM Branding Agency | 210519 - Portfolio Review (F&B) -s.pptx
 
The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024The future of UX design support tools - talk Paris March 2024
The future of UX design support tools - talk Paris March 2024
 
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
Best-NO1 Pakistani Amil Baba Real Amil baba In Pakistan Najoomi Baba in Pakis...
 
Designing for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teamsDesigning for privacy: 3 essential UX habits for product teams
Designing for privacy: 3 essential UX habits for product teams
 
UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024UX Conference on UX Research Trends in 2024
UX Conference on UX Research Trends in 2024
 
Production of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptxProduction of Erythromycin microbiology.pptx
Production of Erythromycin microbiology.pptx
 
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdfBuilding+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
Building+your+Data+Project+on+AWS+-+Luke+Anderson.pdf
 

X-Construction Lite: HCI Evaluation

  • 1. X-Construction Lite: HCI evaluation Title Andy Keavey, Mariana de Rojas-Morao, Merve Yildirim, Queen Ajifa Ododo, Tom Hume Text Tom Hume Human-Computer Interaction, assignment 1
  • 2. Title Text Screenshot of page from Android Marketplace Tom Hume Human-Computer Interaction, assignment 1
  • 3. Overview of gameplay First screen of instructions (animates) Second screen of instructions Game-play screen Results of play screen Tom Hume Human-Computer Interaction, assignment 1
  • 4. Evaluation approach 1. Expert review 1.1. Cognitive walkthrough, written up 1.2. Evaluation vs Nielsen heuristics 2. User testing 2.1. Pre-test questionnaire 2.2. User test of first level (video recorded) 2.3. Interview with user whilst reviewing recording 2.4. Post-test questionnaire Tom Hume Human-Computer Interaction, assignment 1
  • 5. User evaluation 8 test subjects recruited (all Android owners, familiar with UI) Tom Hume Human-Computer Interaction, assignment 1
  • 6. Results of cognitive walkthrough Issue # reports Zoom functions misused ✓✓✓ Errors placing girders ✓✓✓ Misunderstanding help messages ✓✓✓ Unsure how to proceed ✓✓ Game crashed on start ✓ Problem starting play ✓ Confused by pop-up adverts ✓ “I successfully drew some girders, but I struggled a bit because they were going “Some of my girders went out of view at the wrong direction most the top of the screen. I used the zoom out of the time” gesture consistent with Google Maps and could see a much better view of the task” Tom Hume Human-Computer Interaction, assignment 1
  • 7. Results of heuristic evaluation Agreement Disagreement Match between system & real world User control & freedom Consistency & standards Error prevention Flexibility & efficiency Recognition over recall Aesthetics & minimalism Tom Hume Human-Computer Interaction, assignment 1
  • 8. Results of quantitative data 300 Seconds to complete first level 225 150 75 0 0 1 2 3 4 # attempts to complete first level Tom Hume Human-Computer Interaction, assignment 1
  • 9. Results of questionnaire data Age of test subjects Female Male 40 30 25% 20 10 75% 0 8 7 5 4 6 1 3 2 Estimated vs actual fun Android gamer Not android gamer 8 6 4 38% 2 63% 0 1 2 3 4 5 6 7 8 Estimated Actual Tom Hume Human-Computer Interaction, assignment 1
  • 10. Results of user testing Issue # users Misunderstood girder placement ✓✓✓✓✓✓ Didn’t understand help/instructions ✓✓✓✓✓ Mixed up replaying/restarting level ✓✓✓✓ Couldn’t get the train started ✓✓✓ Panned unintentionally ✓✓ Couldn’t tell they were complete ✓ Couldn’t find girder removal tool ✓ Confused by an advert ✓ Misunderstood tool instructions ✓ Couldn’t find instructions again ✓ Tom Hume Human-Computer Interaction, assignment 1
  • 11. Key issue: misunderstood girder placement Tom Hume Human-Computer Interaction, assignment 1
  • 12. Key issue: mixed up replaying/restarting level Tom Hume Human-Computer Interaction, assignment 1
  • 13. Conclusions Good news • Most players complete the first level, first time • Most players enjoy the game more than they thought • Many expert evaluators are now regular players Bad news • 75% players, 3/5 experts had issues with placing girders • 62% players don’t read help, 3/5 experts misunderstand it • 50% players mix up replaying and restarting the level • 37% players, 2/5 experts have trouble starting their train • 20% players pan unintentionally, 3/5 experts had zoom issues Tom Hume Human-Computer Interaction, assignment 1
  • 14. Recommendations 1. Warn players on the first level when they place a girder which doesn’t finish attached to an end-point; 2. Clarify language in help messages and screens; 3. Improve the iconography, labelling or placement of the “replay failure” and “restart level” icons to reduce confusion between them; 4. Teach players to use the “play” button by animating it on the first level when they have a workable bridge; 5. Allow pinch-and-zoom to zoom into the bridge during play. Tom Hume Human-Computer Interaction, assignment 1
  • 15. Critique of methods Hard to record video in sunlight Users were self-selecting, to a degree Slight inconsistencies in questionnaire usage Tom Hume Human-Computer Interaction, assignment 1
  • 16. Questions? Tom Hume Human-Computer Interaction, assignment 1