SlideShare ist ein Scribd-Unternehmen logo
1 von 26
WTF with all these
 tools anyway?
      2010 edition
     late
12 years ago

     us e r
re s e
       a rcb
 tool
       s
Methods & Practices

• Empathy
• Curiosity
• Interviewing skills
• Analytical techniques
Since 2007
Nobody wants this
What a circus.
How can we avoid...
just

more
technology
Can tools make us
better researchers?
Weaknesses

• Most of these tools are for websites*
• None of them can ask the right questions
• Most of them are incomplete
• Many of them aren’t really for us

         *There’s actually more to life
5
CATEGORIES
Moderated



Self-Moderated

       For A Long Time




Automated Live

       Conceptual



Automated Static



Web Analytics
Talk To People



People Talk To A Machine

        For A Long Time




Tasks On A Live Site

        Tasks On Concepts



Tasks On Images



Fancy-Ass Analytics
Insight



Ammunition

          For Ad Agencies




More Ammunition

          I.A. & Navigation



Still More Ammunition



Just Better Analytics
Moderated



Self-Moderated

       For A Long Time




Automated Live

       Conceptual



Automated Static
                             Deep
                         Behavioral UX
Web Analytics               Insight
Moderated
                         Opinions

Self-Moderated

       For A Long Time




Automated Live

       Conceptual



Automated Static



Web Analytics
Moderated                Ad Agencies
                           Love It
Self-Moderated

       For A Long Time




Automated Live

       Conceptual



Automated Static



Web Analytics
Moderated                  Jared Says
                         Watch Out For
Self-Moderated           Bad Inferences
       For A Long Time




Automated Live

       Conceptual



Automated Static



Web Analytics
Most important thing in
       the box
Rdio Recipe
•   GoToMeeting: 5 users (moderated)

•   UserTesting.com: 10 users (self-moderated)

•   Usabilla: 50 users (automated)

•   Ethnio: 468 recruits
DUAL LENS
STREAMING
FIELD RESEARCH


• Livestream
• Flip Cam
• It’s the view
  finder, silly
• Actual duct
  tape
Go forth

• What’s the craziest research idea you’ve
  ever had?
• Maybe one (or three) of these tools can
  make it happen.
• Keep your duct tape handy.
Thank you. Questions?

cyd@boltpeters.com    @cydharrell
nate@boltpeters.com   @boltron

Weitere ähnliche Inhalte

Ähnlich wie Cyd Harrell - State of The Vendor Circus

On demand v1 webvisions
On demand v1 webvisionsOn demand v1 webvisions
On demand v1 webvisionsbolt peters
 
Remote Research Workshop, UX Week 2012 - Cyd Harrell
Remote Research Workshop, UX Week 2012 - Cyd HarrellRemote Research Workshop, UX Week 2012 - Cyd Harrell
Remote Research Workshop, UX Week 2012 - Cyd HarrellCyd Harrell
 
Best Practices From 10 Years of Remote Research
Best Practices From 10 Years of Remote ResearchBest Practices From 10 Years of Remote Research
Best Practices From 10 Years of Remote Researchuxpin
 
User Experience Design Fundamentals - Part 2: Talking with Users
User Experience Design Fundamentals - Part 2: Talking with UsersUser Experience Design Fundamentals - Part 2: Talking with Users
User Experience Design Fundamentals - Part 2: Talking with UsersLaura B
 
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik MarselisTesting machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik MarselisRik Marselis
 
DIY Website Usability Testing
DIY Website Usability TestingDIY Website Usability Testing
DIY Website Usability TestingSteve Martin
 
How to escape from a Google penalty
How to escape from a Google penaltyHow to escape from a Google penalty
How to escape from a Google penaltyWoptimo
 
Why Using Data And Talking To Users Is Critical
Why Using Data And Talking To Users Is CriticalWhy Using Data And Talking To Users Is Critical
Why Using Data And Talking To Users Is CriticalProduct School
 
Software Tests and Robots
Software Tests and RobotsSoftware Tests and Robots
Software Tests and RobotsLarry Cynkin
 
Reinforcement Learning In AI Powerpoint Presentation Slide Templates Complete...
Reinforcement Learning In AI Powerpoint Presentation Slide Templates Complete...Reinforcement Learning In AI Powerpoint Presentation Slide Templates Complete...
Reinforcement Learning In AI Powerpoint Presentation Slide Templates Complete...SlideTeam
 
Design Recommender systems from scratch
Design Recommender systems from scratchDesign Recommender systems from scratch
Design Recommender systems from scratchDr. Amit Sachan
 
Using Analytics for User Research
Using Analytics for User ResearchUsing Analytics for User Research
Using Analytics for User ResearchLuke Hay
 
User-Testing, Testing, 1,2,3
User-Testing, Testing, 1,2,3User-Testing, Testing, 1,2,3
User-Testing, Testing, 1,2,3BusinessOnline
 
Directions - Council for Vocational Services Society - Halifax [2010-10-07]
Directions - Council for Vocational Services Society - Halifax [2010-10-07]Directions - Council for Vocational Services Society - Halifax [2010-10-07]
Directions - Council for Vocational Services Society - Halifax [2010-10-07]CanadaHelps / MyCharityConnects
 
Remote usability testing and remote user research for usability
Remote usability testing and remote user research for usabilityRemote usability testing and remote user research for usability
Remote usability testing and remote user research for usabilityUser Vision
 
SEO disasters: the good, the bad and the taboo: Steven van Vessum
SEO disasters: the good, the bad and the taboo: Steven van VessumSEO disasters: the good, the bad and the taboo: Steven van Vessum
SEO disasters: the good, the bad and the taboo: Steven van VessumSearchNorwich
 
Surviving a Hackathon and Beyond
Surviving a Hackathon and BeyondSurviving a Hackathon and Beyond
Surviving a Hackathon and Beyondimoneytech
 

Ähnlich wie Cyd Harrell - State of The Vendor Circus (20)

On demand v1 webvisions
On demand v1 webvisionsOn demand v1 webvisions
On demand v1 webvisions
 
Remote Research Workshop, UX Week 2012 - Cyd Harrell
Remote Research Workshop, UX Week 2012 - Cyd HarrellRemote Research Workshop, UX Week 2012 - Cyd Harrell
Remote Research Workshop, UX Week 2012 - Cyd Harrell
 
Best Practices From 10 Years of Remote Research
Best Practices From 10 Years of Remote ResearchBest Practices From 10 Years of Remote Research
Best Practices From 10 Years of Remote Research
 
User Experience Design Fundamentals - Part 2: Talking with Users
User Experience Design Fundamentals - Part 2: Talking with UsersUser Experience Design Fundamentals - Part 2: Talking with Users
User Experience Design Fundamentals - Part 2: Talking with Users
 
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik MarselisTesting machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
Testing machine learning, learning machine testing. EuroSTAR 2016 Rik Marselis
 
DIY Website Usability Testing
DIY Website Usability TestingDIY Website Usability Testing
DIY Website Usability Testing
 
How to escape from a Google penalty
How to escape from a Google penaltyHow to escape from a Google penalty
How to escape from a Google penalty
 
Why Using Data And Talking To Users Is Critical
Why Using Data And Talking To Users Is CriticalWhy Using Data And Talking To Users Is Critical
Why Using Data And Talking To Users Is Critical
 
Software Tests and Robots
Software Tests and RobotsSoftware Tests and Robots
Software Tests and Robots
 
Reinforcement Learning In AI Powerpoint Presentation Slide Templates Complete...
Reinforcement Learning In AI Powerpoint Presentation Slide Templates Complete...Reinforcement Learning In AI Powerpoint Presentation Slide Templates Complete...
Reinforcement Learning In AI Powerpoint Presentation Slide Templates Complete...
 
Design Recommender systems from scratch
Design Recommender systems from scratchDesign Recommender systems from scratch
Design Recommender systems from scratch
 
Using Analytics for User Research
Using Analytics for User ResearchUsing Analytics for User Research
Using Analytics for User Research
 
Agile iOS
Agile iOSAgile iOS
Agile iOS
 
User-Testing, Testing, 1,2,3
User-Testing, Testing, 1,2,3User-Testing, Testing, 1,2,3
User-Testing, Testing, 1,2,3
 
Directions - Council for Vocational Services Society - Halifax [2010-10-07]
Directions - Council for Vocational Services Society - Halifax [2010-10-07]Directions - Council for Vocational Services Society - Halifax [2010-10-07]
Directions - Council for Vocational Services Society - Halifax [2010-10-07]
 
Remote usability testing and remote user research for usability
Remote usability testing and remote user research for usabilityRemote usability testing and remote user research for usability
Remote usability testing and remote user research for usability
 
SEO disasters: the good, the bad and the taboo: Steven van Vessum
SEO disasters: the good, the bad and the taboo: Steven van VessumSEO disasters: the good, the bad and the taboo: Steven van Vessum
SEO disasters: the good, the bad and the taboo: Steven van Vessum
 
Surviving a Hackathon and Beyond
Surviving a Hackathon and BeyondSurviving a Hackathon and Beyond
Surviving a Hackathon and Beyond
 
Doonish
DoonishDoonish
Doonish
 
Doonish
DoonishDoonish
Doonish
 

Mehr von bolt peters

Ben McAllister - The Science of Good Design: A Dangerous Idea
Ben McAllister - The Science of Good Design: A Dangerous IdeaBen McAllister - The Science of Good Design: A Dangerous Idea
Ben McAllister - The Science of Good Design: A Dangerous Ideabolt peters
 
Sara Summers - Invisible Me: Emotional Intelligence & Observational Research
Sara Summers - Invisible Me: Emotional Intelligence & Observational ResearchSara Summers - Invisible Me: Emotional Intelligence & Observational Research
Sara Summers - Invisible Me: Emotional Intelligence & Observational Researchbolt peters
 
Aaron Irizarry - Bootstrapping User Research
Aaron Irizarry - Bootstrapping User ResearchAaron Irizarry - Bootstrapping User Research
Aaron Irizarry - Bootstrapping User Researchbolt peters
 
Gini Keating - Getting Out There: Research for Mobile Augmented Reality
Gini Keating - Getting Out There: Research for Mobile Augmented RealityGini Keating - Getting Out There: Research for Mobile Augmented Reality
Gini Keating - Getting Out There: Research for Mobile Augmented Realitybolt peters
 
Stop Listening to Your Customers
Stop Listening to Your CustomersStop Listening to Your Customers
Stop Listening to Your Customersbolt peters
 
Jared Spool - Opening Plenary
Jared Spool - Opening PlenaryJared Spool - Opening Plenary
Jared Spool - Opening Plenarybolt peters
 
Remote Research, The Talk.
Remote Research, The Talk.Remote Research, The Talk.
Remote Research, The Talk.bolt peters
 
Remote Research at IxD10
Remote Research at IxD10Remote Research at IxD10
Remote Research at IxD10bolt peters
 
Escape The Lab Tri Upa Slides
Escape The Lab Tri Upa SlidesEscape The Lab Tri Upa Slides
Escape The Lab Tri Upa Slidesbolt peters
 
When Design Research is Evil
When Design Research is EvilWhen Design Research is Evil
When Design Research is Evilbolt peters
 
Future of Library User Experience
Future of Library User ExperienceFuture of Library User Experience
Future of Library User Experiencebolt peters
 
Researching Spore: A New Approach to Player Testing
Researching Spore: A New Approach to Player TestingResearching Spore: A New Approach to Player Testing
Researching Spore: A New Approach to Player Testingbolt peters
 

Mehr von bolt peters (14)

Ben McAllister - The Science of Good Design: A Dangerous Idea
Ben McAllister - The Science of Good Design: A Dangerous IdeaBen McAllister - The Science of Good Design: A Dangerous Idea
Ben McAllister - The Science of Good Design: A Dangerous Idea
 
Sara Summers - Invisible Me: Emotional Intelligence & Observational Research
Sara Summers - Invisible Me: Emotional Intelligence & Observational ResearchSara Summers - Invisible Me: Emotional Intelligence & Observational Research
Sara Summers - Invisible Me: Emotional Intelligence & Observational Research
 
Aaron Irizarry - Bootstrapping User Research
Aaron Irizarry - Bootstrapping User ResearchAaron Irizarry - Bootstrapping User Research
Aaron Irizarry - Bootstrapping User Research
 
Gini Keating - Getting Out There: Research for Mobile Augmented Reality
Gini Keating - Getting Out There: Research for Mobile Augmented RealityGini Keating - Getting Out There: Research for Mobile Augmented Reality
Gini Keating - Getting Out There: Research for Mobile Augmented Reality
 
Stop Listening to Your Customers
Stop Listening to Your CustomersStop Listening to Your Customers
Stop Listening to Your Customers
 
Jared Spool - Opening Plenary
Jared Spool - Opening PlenaryJared Spool - Opening Plenary
Jared Spool - Opening Plenary
 
Remote Research, The Talk.
Remote Research, The Talk.Remote Research, The Talk.
Remote Research, The Talk.
 
ETL May 2010
ETL May 2010ETL May 2010
ETL May 2010
 
Remote Research at IxD10
Remote Research at IxD10Remote Research at IxD10
Remote Research at IxD10
 
Escape The Lab Tri Upa Slides
Escape The Lab Tri Upa SlidesEscape The Lab Tri Upa Slides
Escape The Lab Tri Upa Slides
 
When Design Research is Evil
When Design Research is EvilWhen Design Research is Evil
When Design Research is Evil
 
Future of Library User Experience
Future of Library User ExperienceFuture of Library User Experience
Future of Library User Experience
 
Researching Spore: A New Approach to Player Testing
Researching Spore: A New Approach to Player TestingResearching Spore: A New Approach to Player Testing
Researching Spore: A New Approach to Player Testing
 
In The Moment
In The MomentIn The Moment
In The Moment
 

Cyd Harrell - State of The Vendor Circus

Hinweis der Redaktion

  1. \n
  2. back in the day, there was a lot of technology associated with research\nbut it was all for documenting it, not really for doing it\n\n
  3. our major tools as researchers were things that live inside us\nwe knew how to elicit different kinds of information\nhow to structure research interactions\nwhat to do with qualitative data\nwe’re still good at all those things\n\n
  4. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  5. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  6. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  7. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  8. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  9. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  10. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  11. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  12. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  13. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  14. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  15. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  16. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  17. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  18. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  19. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  20. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  21. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  22. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  23. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  24. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  25. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  26. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  27. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  28. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  29. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  30. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  31. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  32. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  33. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  34. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  35. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  36. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  37. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  38. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  39. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  40. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  41. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  42. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  43. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  44. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  45. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  46. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  47. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  48. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  49. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  50. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  51. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  52. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  53. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  54. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  55. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  56. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  57. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  58. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  59. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  60. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  61. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  62. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  63. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  64. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  65. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  66. So, something changed a few years ago. People started to create software to make this easier, and maybe to expand it a little bit. First it was really helping researchers do some of the traditional practices remotely. Or reducing the cost of those labs. But it started to branch out. And then more and more software, real experimentation, so now, I count 60 or so tools that are supposed to do or help do user research.\nWe’ve got a sack full of hammers and we could just saddle up and go looking for some nails. And we might have happy clients if we did.\n
  67. \n
  68. \n
  69. \n
  70. \n
  71. I read a lot of science fiction, and my ideal relationship with these kinds of tools is to think about the exoskeletons that badass characters have. (And yes, Raytheon actually built a prototype.) They amplify what the person inside can do, but they don’t act for them. So the angle I like to take on the tools is\n
  72. \n
  73. The spirit is right. I'm not sure the representation works. (I wasn't sure if the arrow was going in the direction of more hate or more love since it starts at "hates it".)\n\nAlso, Hate is probably the wrong word. I don't hate the methods. I could care less about the data collection methods. \n\nWhat I care about is the sloppy way those methods drive people to making solid, informed inferences. \n\nFor example, I don't hate eye tracking equipment. (In fact, I think it has lots of cool uses, like giving quadriplegics a power assistive device.) I hate when someone tells me they watched the blue dot on the eye tracker and the user didn't see the link. They don't know what the user saw because the eye tracker doesn't tell them anything about what users see. It only tells them what the device thinks the user gazed at, which has nothing to do with what they see.  I've proven I can gaze directly at the ketchup bottle on the top shelf of the fridge all day without seeing it.\n\nThe problem with unmoderated tools is they always have unmoderated analysis. People who have no skill in making proper inferences end up making bad inferences.\n\nWhile we shouldn't blame DVD players for bad Jim Carrey movies, we can do something to reduce the problem. Sure you can make proper inferences from these tools, but when you've expended all the energy to train the users to make great inferences (which, by the way, greatly reduce the perceived value of the results because everything has to be qualified), you probably are better off using the more traditional methods that return equivalent value for about the same cost.\n\nNow, get that on a slide. :)\n\nJared\n\n\n
  74. Here are some of the tools out here. \n
  75. There are some unexpected advantages to some of these methods. Usertesting passed along some unexpected use of their tool, which some of you said you were interested in learning more. \n
  76. Here are some of the tools out here. \n
  77. Here are some of the tools out here. \n
  78. \n
  79. \n
  80. Nate chimes in\n
  81. \n
  82. \n
  83. \n
  84. \n
  85. \n
  86. \n