SlideShare ist ein Scribd-Unternehmen logo
1 von 69
Downloaden Sie, um offline zu lesen
presents
How to get out of the building
  without getting out of the building




@UXdna
What I won’t talk about
    How to use different remote testing tools

    Advantages and pitfalls of each type (moderated, unmoderated,
    automatic, live, etc.) For this you can read Nate Bolt’s amazing book
    ‘Remote Research’ or watch his presentation here: http://
    www.slideshare.net/boltpeters/remote-research-the-book-the-talk




@UXdna
This is about


    A practical guide on how to interview and observe
    customers remotely




@UXdna
PREPARE & ANTICIPATE




@UXdna
Example


    A Spanish company wants to launch an online
    restaurant listing and booking service in Brazil.




@UXdna
The process

                                     2. Who?

                                     Who you need
                                     to talk to/test
                                     with?
               1. What?

         What do you need to learn      3. How?
         from it?

                                      How can you get
                                      there, step by step




@UXdna
PREPARE & ANTICIPATE




@UXdna
1. What?
                 What do you need to learn from it?

     Interview       Show + Interview   Beta/Online product   Prototype




@UXdna
1. What?
                 What do you need to learn from it?

     Interview       Show + Interview   Beta/Online product   Prototype




@UXdna
1. What?
                 What do you need to learn from it?

     Interview       Show + Interview   Beta/Online product    Prototype




                                                                JOURNEY
                                            JOURNEY                +
     SCRIPT OR         SCRIPT OR               +              A PROTOTYPE
     TOPIC MAP         TOPIC MAP           SCRIPT OR               +
                                           TOPIC MAP           SCRIPT OR
                                                               TOPIC MAP




@UXdna
1. What?
                 What do you need to learn from it?

     Interview       Show + Interview   Beta/Online product    Prototype




                                                                JOURNEY
                                            JOURNEY                +
     SCRIPT OR         SCRIPT OR               +              A PROTOTYPE
     TOPIC MAP         TOPIC MAP           SCRIPT OR               +
                                           TOPIC MAP           SCRIPT OR
                                                               TOPIC MAP




@UXdna
1. What?
                 What do you need to learn from it?

     Interview       Show + Interview   Beta/Online product      Prototype




                                                                 JOURNEY
                                            JOURNEY                 +
     SCRIPT OR         SCRIPT OR               +               A PROTOTYPE
     TOPIC MAP         TOPIC MAP           SCRIPT OR                +
                                           TOPIC MAP            SCRIPT OR
                                                                TOPIC MAP



                        VOICE               VOICE                 VOICE
      VOICE
                       WEBCAM              WEBCAM                WEBCAM
     WEBCAM
                    SCREEN SHARING      SCREEN SHARING        SCREEN SHARING

@UXdna
1. What?
                 What do you need to learn from it?

     Interview       Show + Interview   Beta/Online product      Prototype




                                                                 JOURNEY
                                            JOURNEY                 +
     SCRIPT OR         SCRIPT OR               +               A PROTOTYPE
     TOPIC MAP         TOPIC MAP           SCRIPT OR                +
                                           TOPIC MAP            SCRIPT OR
                                                                TOPIC MAP



                        VOICE               VOICE                 VOICE
      VOICE
                       WEBCAM              WEBCAM                WEBCAM
     WEBCAM
                    SCREEN SHARING      SCREEN SHARING        SCREEN SHARING

@UXdna
Example

         An Axure prototype of a 3 page journey
                 Will need to build and
                 test prototype

                 Will need screen
                 sharing

                 Will need voice

                 Will need to define a
                 scenario to users

                 Will need to explain to
                 users that this is a
                 prototype, not a real
                 site
@UXdna
Example

         An Axure prototype of a 3 page journey
                 Will need to build and
                 test prototype

                 Will need screen
                 sharing

                 Will need voice

                 Will need to define a
                 scenario to users

                 Will need to explain to
                 users that this is a
                 prototype, not a real
                 site
@UXdna
2. Who?
                                RECRUITMENT

     A. Location          B. Behaviours      C. Channels     D. What to say?


                         What do you need
     Where are the                                           What will you need
                        to know about them
    people you will                          Where can you   to say to get their
                        to make them good
   need to interview/                         find them?      attention and how
                          candidates for a
      test with?                                                   to do it
                           test/interview?




@UXdna
2. Who? Recruitment
   A. LOCATION                         Where are the people you will need to interview/test with?




                                                            ACCESS
       TIME ZONE                 LANGUAGES                                            VIEWERS
                                                        TO TECHNOLOGY



Are the potential           Do I speak the local     What is the country’s     Will I be able to stream
participants on my time     language?                internet connection       to viewers as well?
zone?                                                speed like?
                            Do I need to hire
What are the times of       translation?             Skype says 300 kbps
the day I will be able to                            but you need at least
talk to them/test?          Do I need to recruit     512 kbps
                            participants who speak
                            English?                 Speedtest.net




@UXdna
Example




@UXdna
Example




@UXdna
Example



           PORTUGUESE
           5 HOURS BEHIND
           OK SPEED




@UXdna
2. Who? Recruitment
 B. BEHAVIOURS           What do I need to know about them?




    TIGHTLY
    DEFINED SET OF                                        LOOSE SET OF
    BEHAVIOURS                                             BEHAVIOURS


     I need to talk to
      teenagers who                                      I need to talk to
     have an Android                                      people who go
        phone and                                         out dining and
    download at least                                    check for good
     3 new games a                                      restaurants online
    week, every week




@UXdna
Example

    TIGHTLY
    DEFINED SET OF                                                   LOOSE SET OF
    BEHAVIOURS                                                        BEHAVIOURS



    We needed to know they were people who are likely to be our current hypothesis
    customers, i.e. anyone with a certain level of income (+ 5 minimum wages), who
                 habitually looked for advice about places to eat online




@UXdna
Example

    TIGHTLY
    DEFINED SET OF                                                   LOOSE SET OF
    BEHAVIOURS                                                        BEHAVIOURS



    We needed to know they were people who are likely to be our current hypothesis
    customers, i.e. anyone with a certain level of income (+ 5 minimum wages), who
                 habitually looked for advice about places to eat online




@UXdna
2. Who? Recruitment
   C. CHANNEL                                    Where can I find them?



  Easier                                                                                Difficult




    List of emails from                          Publish an ad or        Intercept as they
                          Call centre contacts
    current customers                            post in a forum         use website




                           LIST OF EMAIL ADDRESSES
                           or PHONE NUMBERS




@UXdna
Example

  Easier                                                                                      Difficult




    List of emails from                                 Publish an ad or       Intercept as they
                            Call centre contacts
    current customers                                   post in a forum        use website




                  We published a job position for                    NOTE: It is important to
                   ‘site tester’ in free job posting                 get an idea of what sites
                   websites and ‘Craigslist’ style                   people use in the
                                                                     country you are testing
                                                lists

@UXdna
Example

  Easier                                                                                      Difficult




    List of emails from                                 Publish an ad or       Intercept as they
                            Call centre contacts
    current customers                                   post in a forum        use website




                  We published a job position for                    NOTE: It is important to
                   ‘site tester’ in free job posting                 get an idea of what sites
                   websites and ‘Craigslist’ style                   people use in the
                                                                     country you are testing
                                                lists

@UXdna
2. Who? Recruitment
 D. WHAT TO SAY?



 D1                                    D2                                 D3
      Create a screener                     Write an email                     Collect survey
      or list of questions                  through which you                  results, filter and
      with which you                        will send the       SURVEY         select
                             SURVEY LINK                        ANSWERS
      can filter out                         survey link
      unsuitable
      participants




@UXdna
2. Who? Screener questionnaire
 D. WHAT TO SAY?                                    What makes a good screener?




                                 A GOOD SURVEY MAKES IT EASY TO
                                            FILTER


  1                         2                         3                         4
      List 3                    List 3 behaviours         Based on these            Don’t ask ‘would
      characteristics           that make                 factors, write            you’ questions.
      someone should            someone unlikely          questions that            Ask ‘what was the
      display to make           to be your                have discrete             last time you...’
      them likely to be a       customer                  answers (like ‘yes’       questions
      customer                                            and ‘no’ or
                                                          multiple choice)




@UXdna
Example
 1                         2                            3                         4
     List 3                    List 3 behaviours            Based on these            Don’t ask ‘would
     characteristics           that make                    factors, write            you’ questions.
     someone should            someone unlikely             questions that            Ask ‘what was the
     display to make           to be your                   have discrete             last time you...’
     them likely to be a       customer                     answers (like ‘yes’       questions
     customer                                               and ‘no’ or
                                                            multiple choice)




     Must live in SP            Only seek               Where do you live?            What do you
                                recommendations         (a-d)                         usually do...
     Must have dined in         from friends
     a restaurant about                                 When was the last             Would you
     which they found           Always go to the        time you visited a            generally...
     reviews online in          same restaurant,        new restaurant?
     the last 7 days            don’t like trying new
                                places                  Which, of the sites
     Must have                                          below, have you
     consulted at least         Only dine out for       visited in the last 7
     1 review site              very special            days?
                                occasions
@UXdna
Example
 1                         2                            3                         4
     List 3                    List 3 behaviours            Based on these            Don’t ask ‘would
     characteristics           that make                    factors, write            you’ questions.
     someone should            someone unlikely             questions that            Ask ‘what was the
     display to make           to be your                   have discrete             last time you...’
     them likely to be a       customer                     answers (like ‘yes’       questions
     customer                                               and ‘no’ or
                                                            multiple choice)




     Must live in SP            Only seek               Where do you live?            What do you
                                recommendations         (a-d)                         usually do...
     Must have dined in         from friends
     a restaurant about                                 When was the last             Would you
     which they found           Always go to the        time you visited a            generally...
     reviews online in          same restaurant,        new restaurant?
     the last 7 days            don’t like trying new
                                places                  Which, of the sites
     Must have                                          below, have you
     consulted at least         Only dine out for       visited in the last 7
     1 review site              very special            days?
                                occasions
@UXdna
2. Who? Screener questionnaire

MY SURVEY
              Only show me answers from people who
              replied marked the option ‘last 7 days’ to the
              question

              “When was the last time you looked online for
              restaurant reviews?”


              456 respondents = 36 fit our criteria




@UXdna
2. Who? Email contact
 D. WHAT TO SAY?                       What makes a good contact email?


 Hi,

 You are receiving this email because you replied to a job post on [name of
 website you used here].

 My name is [your name here] and I am a researcher/CEO/Founder at [name of
 company here], ([website link here, if it exists]). We are working on a project
 about leisure habits in Brazil.

 We would like to count on you for a phone interview, but beforehand we would
 like to know a little more about you. Could you please answer the questions on
 the following link

 (link to survey, very visible)

@UXdna
2. Who? Email contact
 D. WHAT TO SAY?                       What makes a good contact email?


 Hi,
    This is why you are receiving this email from someone you don’t know.
 YouYou are interestedemail because yousay, keep onjob post on [name of
     are receiving this in what I have to replied to a reading.
 website you used here].

 My name is [your name here] and I am a researcher/CEO/Founder at [name of
 company here], ([website link here, if it exists]). We are working on a project
 about leisure habits in Brazil.

 We would like to count on you for a phone interview, but beforehand we would
 like to know a little more about you. Could you please answer the questions on
 the following link

 (link to survey, very visible)

@UXdna
2. Who? Email contact
 D. WHAT TO SAY?                      What makes a good contact email?


 Hi,
    This is why you are receiving this email from someone you don’t know.
 YouYou are interestedemail because yousay, keep onjob post on [name of
     are receiving this in what I have to replied to a reading.
 website you used here].

 My name is [your name here] and I am a researcher/CEO/Founder at [name of
 company here], I([website link here, am exists]). We are working on a project
    This is who am, this is what I if it working on and these are
    my credentials
 about leisure habits in Brazil.

 We would like to count on you for a phone interview, but beforehand we would
 like to know a little more about you. Could you please answer the questions on
 the following link

 (link to survey, very visible)

@UXdna
2. Who? Email contact
 D. WHAT TO SAY?                       What makes a good contact email?


 Hi,
    This is why you are receiving this email from someone you don’t know.
 YouYou are interestedemail because yousay, keep onjob post on [name of
     are receiving this in what I have to replied to a reading.
 website you used here].

 My name is [your name here] and I am a researcher/CEO/Founder at [name of
 company here], I([website link here, am exists]). We are working on a project
    This is who am, this is what I if it working on and these are
    my credentials
 about leisure habits in Brazil.

 We would like to count on you for a phone interview, but beforehand we would
 like Weknow a little more including you in the research, but I needquestionsif you
      to are interested in about you. Could you please answer the to know on
 the are the right person first
      following link

 (link to survey, very visible)

@UXdna
2. Who? Email contact
 D. WHAT TO SAY?                       What makes a good contact email?


 Hi,
    This is why you are receiving this email from someone you don’t know.
 YouYou are interestedemail because yousay, keep onjob post on [name of
     are receiving this in what I have to replied to a reading.
 website you used here].

 My name is [your name here] and I am a researcher/CEO/Founder at [name of
 company here], I([website link here, am exists]). We are working on a project
    This is who am, this is what I if it working on and these are
    my credentials
 about leisure habits in Brazil.

 We would like to count on you for a phone interview, but beforehand we would
 like Weknow a little more including you in the research, but I needquestionsif you
      to are interested in about you. Could you please answer the to know on
 the are the right person first
      following link

 (link to survey, very visible)
       VERY VISIBLE LINK TO SURVEY
@UXdna
2. Who? Email contact
 D. WHAT TO SAY?                         What makes a good contact email?


 We will be in touch if you are selected for the interview. You should be available
 for a phone call on the [day & date] or [day & date] morning.

 We will call you using Skype, and the whole interview shouldn’t last more than
 40 minutes. You should provide us with a Skype name or a landline number we
 can call you on.

 Lastly, I would like to let you know that the payment for your time will be made
 via Paypal, so you must have an active Paypal account.

 Thank you already for your interest in participating

 Best regards

 [your name here]
@UXdna
2. Who? Email contact
 D. WHAT TO SAY?                         What makes a good contact email?


 We Next steps and if you are selected for the interview. You should be available,
      will be in touch confirmation of availability (dates). If you are not available
 for a phone call on the [day further or [day & date] morning.
     don’t bother going any & date]

 We will call you using Skype, and the whole interview shouldn’t last more than
 40 minutes. You should provide us with a Skype name or a landline number we
 can call you on.

 Lastly, I would like to let you know that the payment for your time will be made
 via Paypal, so you must have an active Paypal account.

 Thank you already for your interest in participating

 Best regards

 [your name here]
@UXdna
2. Who? Email contact
 D. WHAT TO SAY?                         What makes a good contact email?


 We Next steps and if you are selected for the interview. You should be available,
      will be in touch confirmation of availability (dates). If you are not available
 for a phone call on the [day further or [day & date] morning.
     don’t bother going any & date]

 We will call you using Skype, and the whole interview shouldn’t last more than
 40 minutes. You of what provide us with a have: Skype, or a landline number we
    Explanation should you will need to Skype name phone,computer with
 canunrestricted internet connection, etc.
     call you on.

 Lastly, I would like to let you know that the payment for your time will be made
 via Paypal, so you must have an active Paypal account.

 Thank you already for your interest in participating

 Best regards

 [your name here]
@UXdna
2. Who? Email contact
 D. WHAT TO SAY?                         What makes a good contact email?


 We Next steps and if you are selected for the interview. You should be available,
      will be in touch confirmation of availability (dates). If you are not available
 for a phone call on the [day further or [day & date] morning.
     don’t bother going any & date]

 We will call you using Skype, and the whole interview shouldn’t last more than
 40 minutes. You of what provide us with a have: Skype, or a landline number we
    Explanation should you will need to Skype name phone,computer with
 canunrestricted internet connection, etc.
     call you on.

 Lastly, I would like to let you know that the payment for your time will be made
 via Paypal, so you must have an active Paypal account.
    How I will pay for your time. If you are not happy, let me know
 Thank you already for your interest in participating

 Best regards

 [your name here]
@UXdna
3. How?
                                   Show time!

         A. Set up            B. Pilot           C. Test        D. Post test


       What you are
                           Test your whole
     testing and how                                         What can I improve
                         setup. What can go     Show time!
    you are collecting                                         for next time?
                               wrong?
         feedback




@UXdna
3. How?
   A. SET UP   What do I need to prepare for the test?




@UXdna
3. How?
   A. SET UP                        What do I need to prepare for the test?



     Interview   Show + Interview        Beta/Online product           Prototype




@UXdna
3. How?
   A. SET UP                        What do I need to prepare for the test?



     Interview   Show + Interview        Beta/Online product           Prototype




@UXdna
3. How?
   A. SET UP                        What do I need to prepare for the test?



     Interview   Show + Interview        Beta/Online product           Prototype




                                                                       JOURNEY
                                              JOURNEY                     +
     SCRIPT OR     SCRIPT OR                     +                   A PROTOTYPE
     TOPIC MAP     TOPIC MAP                 SCRIPT OR                    +
                                             TOPIC MAP                SCRIPT OR
                                                                      TOPIC MAP




@UXdna
3. How?
   A. SET UP                        What do I need to prepare for the test?



     Interview   Show + Interview        Beta/Online product           Prototype




                                                                       JOURNEY
                                              JOURNEY                     +
     SCRIPT OR     SCRIPT OR                     +                   A PROTOTYPE
     TOPIC MAP     TOPIC MAP                 SCRIPT OR                    +
                                             TOPIC MAP                SCRIPT OR
                                                                      TOPIC MAP




@UXdna
3. How?
   A. SET UP                        What do I need to prepare for the test?



     Interview   Show + Interview        Beta/Online product           Prototype




                                                                       JOURNEY
                                              JOURNEY                     +
     SCRIPT OR     SCRIPT OR                     +                   A PROTOTYPE
     TOPIC MAP     TOPIC MAP                 SCRIPT OR                    +
                                             TOPIC MAP                SCRIPT OR
                                                                      TOPIC MAP



                     VOICE                    VOICE                    VOICE
      VOICE
                    WEBCAM                   WEBCAM                   WEBCAM
     WEBCAM
                 SCREEN SHARING           SCREEN SHARING           SCREEN SHARING

@UXdna
Example
         A scenario so you can situate people:
   “Imagine it is Sunday morning and you need to
  find a place to have lunch with a friend who just      Prototype
  arrived in town. You want to try something new”



 Prototype was built with wireframes. It was
    important to let participants know that they
                                                        JOURNEY
  were not going to be looking at a real website,
                                                           +
            even though buttons were clickable
                                                      A PROTOTYPE
                                                           +
                                                       SCRIPT OR
   Themes and questions you would like to              TOPIC MAP
                    discuss with participants.
    Do they understand the order of the search
   results list? Do they see and understand the
                                       ratings ?
                                                         VOICE
                                                        WEBCAM
                        Does my set up work?         SCREEN SHARING
    Test all technology works and is compatible        RECORDING
                                with each other
@UXdna
3. How?




@UXdna
3. How?

         PREPARE & ANTICIPATE




@UXdna
3. How?

         PREPARE & ANTICIPATE
              What can go wrong,
                will go wrong




@UXdna
3. How?

         PREPARE & ANTICIPATE
              What can go wrong,
                will go wrong




                PILOT, PILOT, PILOT!
              HAVE A FALL BACK PLAN
@UXdna
3. How?
    B. PILOT     What problems should I be trying to mitigate in a pilot?


 Problem       What to check?                     Fallback during test




@UXdna
3. How?
      B. PILOT                          What problems should I be trying to mitigate in a pilot?


 Problem                            What to check?                         Fallback during test

                                                                            •Turn off webcam
 Internet connection:               •Speedtest.net can give you an idea of
                                                                            •Kill other applications
 gets really slow or drops           the connection speed - or at least let
                                                                            •Connections often better at
 completely                          you know if participant has dial speed
                                                                             weekends or holidays


                                    •Test out screen sharing
 Screen sharing:                    •Send ‘how to’ step by step            •Alternative screen sharing
 drags a lot or stops completely     screenshots.                           platform (joinme, WebEx,
 or participant can’t start it      •Check Skype version (4.0 ok, 5.0       GoToMeeting, Co-pilot)
                                     better)


 Can’t hear participant
                                    •Test your mic & speakers              •Call a landline and still do
 He can hear you, but you can’t
                                    •Test their mic & speakers              screen sharing
 hear him - or the other way around


@UXdna
3. How?
      B. PILOT                          What problems should I be trying to mitigate in a pilot?


 Problem                            What to check?                         Fallback during test

                                                                            •Turn off webcam
 Internet connection:               •Speedtest.net can give you an idea of
                                                                            •Kill other applications
 gets really slow or drops           the connection speed - or at least let
                                                                            •Connections often better at
 completely                          you know if participant has dial speed
                                                                             weekends or holidays


                                    •Test out screen sharing
 Screen sharing:                    •Send ‘how to’ step by step            •Alternative screen sharing
 drags a lot or stops completely     screenshots.                           platform (joinme, WebEx,
 or participant can’t start it      •Check Skype version (4.0 ok, 5.0       GoToMeeting, Co-pilot)
                                     better)


 Can’t hear participant
                                    •Test your mic & speakers              •Call a landline and still do
 He can hear you, but you can’t
                                    •Test their mic & speakers              screen sharing
 hear him - or the other way around


@UXdna
3. How?
      B. PILOT                          What problems should I be trying to mitigate in a pilot?


 Problem                            What to check?                         Fallback during test

                                                                            •Turn off webcam
 Internet connection:               •Speedtest.net can give you an idea of
                                                                            •Kill other applications
 gets really slow or drops           the connection speed - or at least let
                                                                            •Connections often better at
 completely                          you know if participant has dial speed
                                                                             weekends or holidays


                                    •Test out screen sharing
 Screen sharing:                    •Send ‘how to’ step by step            •Alternative screen sharing
 drags a lot or stops completely     screenshots.                           platform (joinme, WebEx,
 or participant can’t start it      •Check Skype version (4.0 ok, 5.0       GoToMeeting, Co-pilot)
                                     better)


 Can’t hear participant
                                    •Test your mic & speakers              •Call a landline and still do
 He can hear you, but you can’t
                                    •Test their mic & speakers              screen sharing
 hear him - or the other way around


@UXdna
3. How?
    B. PILOT     What problems should I be trying to mitigate in a pilot?


 Problem       What to check?                     Fallback during test




@UXdna
3. How?
      B. PILOT                          What problems should I be trying to mitigate in a pilot?


 Problem                            What to check?                           Fallback during test

                                    •Recruit spare participants or ‘floaters’
                                     for each slot allocated for test/
 No show                             interview, who will be waiting in case •If both participant and floater
 participant doesn’t show up at      someone doesn’t show up                 fail you, check if other floaters
 show time                          •They only get paid if they are needed can fill the gap.
                                     (you should make this clear when
                                     recruiting)


                                    •Send them a link to the prototype/
                                     mock prototype (if you don’t want to
 Prototype/site
                                     show real thing before the test)
 Can participants see the prototype
                                    •Screen share it!                      •Don’t waste time, get your
 exactly as I planned?
                                    •Send link to a friend first             spare participant in there
 Can users access the site?
                                    •Ask them to identify something on the
                                     screen


@UXdna
3. How?
      B. PILOT                          What problems should I be trying to mitigate in a pilot?


 Problem                            What to check?                           Fallback during test

                                    •Recruit spare participants or ‘floaters’
                                     for each slot allocated for test/
 No show                             interview, who will be waiting in case •If both participant and floater
 participant doesn’t show up at      someone doesn’t show up                 fail you, check if other floaters
 show time                          •They only get paid if they are needed can fill the gap.
                                     (you should make this clear when
                                     recruiting)


                                    •Send them a link to the prototype/
                                     mock prototype (if you don’t want to
 Prototype/site
                                     show real thing before the test)
 Can participants see the prototype
                                    •Screen share it!                      •Don’t waste time, get your
 exactly as I planned?
                                    •Send link to a friend first             spare participant in there
 Can users access the site?
                                    •Ask them to identify something on the
                                     screen


@UXdna
3. How?
    B. PILOT     What problems should I be trying to mitigate in a pilot?


 Problem       What to check?                     Fallback during test




@UXdna
3. How?
      B. PILOT                              What problems should I be trying to mitigate in a pilot?


 Problem                                What to check?                           Fallback during test


                                    •Try recording the conversation you          •Turn off your webcam
 Recording
                                     have during the pilot session               •Record voice only
 Does my recording set up work
                                    •WARNING: Silverback is a little odd         •Use spare participant
 with the screen sharing and voice?
                                     when webcam + screen sharing!               •




 Language
                                                                                •You shouldn’t get to the actual
 I can’t understand what the            •Have a conversation with the
                                                                                 test with someone who
 participant is saying (mine or          participant ahead of time and you will
                                                                                 doesn’t fit.
 participant’s language skills aren’t    be able to gauge how you
                                                                                •If you did, and only realise it at
 as good as I thought, accent is         understand each other
                                                                                 the time of test, call in a spare
 really difficult to understand)



@UXdna
3. How?
      B. PILOT                              What problems should I be trying to mitigate in a pilot?


 Problem                                What to check?                           Fallback during test


                                    •Try recording the conversation you          •Turn off your webcam
 Recording
                                     have during the pilot session               •Record voice only
 Does my recording set up work
                                    •WARNING: Silverback is a little odd         •Use spare participant
 with the screen sharing and voice?
                                     when webcam + screen sharing!               •




 Language
                                                                                •You shouldn’t get to the actual
 I can’t understand what the            •Have a conversation with the
                                                                                 test with someone who
 participant is saying (mine or          participant ahead of time and you will
                                                                                 doesn’t fit.
 participant’s language skills aren’t    be able to gauge how you
                                                                                •If you did, and only realise it at
 as good as I thought, accent is         understand each other
                                                                                 the time of test, call in a spare
 really difficult to understand)



@UXdna
3. How?
    B. PILOT     What problems should I be trying to mitigate in a pilot?


 Problem       What to check?                     Fallback during test




@UXdna
3. How?
      B. PILOT                             What problems should I be trying to mitigate in a pilot?


 Problem                              What to check?                            Fallback during test

                                      •WARNING: first impressions can be
                                       deceiving, but talking to the
 Bad recruiting:
                                       participant will give you an idea. You
 the participant doesn’t sound like a                                           •Spare participant
                                       can also ask extra screening
 customer
                                       questions, or check answers in their
                                       questionnaire.



                                     •It will always be a problem
 Rapport
                                     •The pilot helps!
 Main drawback of not being in the                                       •If at the time of testing
                                     •Smile & be nice
 room with participant is: How can I                                      participants don’t feel
                                     •Webcam, so they know you are not a
 make them trust me and tell me                                           comfortable for some reason,
                                      machine
 stuff?                                                                   don’t insist, call in your spare!
                                     •Show your surroundings, if you can
                                      (don’t ask them to do the same)


@UXdna
3. How?
      B. PILOT                             What problems should I be trying to mitigate in a pilot?


 Problem                              What to check?                            Fallback during test

                                      •WARNING: first impressions can be
                                       deceiving, but talking to the
 Bad recruiting:
                                       participant will give you an idea. You
 the participant doesn’t sound like a                                           •Spare participant
                                       can also ask extra screening
 customer
                                       questions, or check answers in their
                                       questionnaire.



                                     •It will always be a problem
 Rapport
                                     •The pilot helps!
 Main drawback of not being in the                                       •If at the time of testing
                                     •Smile & be nice
 room with participant is: How can I                                      participants don’t feel
                                     •Webcam, so they know you are not a
 make them trust me and tell me                                           comfortable for some reason,
                                      machine
 stuff?                                                                   don’t insist, call in your spare!
                                     •Show your surroundings, if you can
                                      (don’t ask them to do the same)


@UXdna
3. How?
    C. TEST                                  How to set it up



              1. Establish Skype
              call with viewers
              (separate username)
                                     SKYPE
              2. Turn off their
              webcam and mic


                                     VIEWER



                  SKYPE PRO


                                                     3. Add participant to
                      YOU                            the conversation
                                       SKYPE +
                                    Screen sharing   4. Ask them to turn
                                                     on web cam
                                                     5. Explain who are
                                                     the other people in
                                    PARTICIPANT      the call
                                                     6. Begin test!



@UXdna
3. How? Checklist
      C. TEST



                     You                                       Participant

                                                 Check participant internet connection
 Check your own internet connection speed        speed

 Check Skype Pro credit ($$)                     Check Skype mic & speakers

                                                 Check screen sharing works and they
 Check Skype mic                                 know how to do it
                                                 Check they can see your site or prototype
                                                 well
 Add participant to Skype contact list, so you
 know when they are available                    Check Paypal email address (might be
                                                 different from the one participant used
 Check recording application/device              when replying to ad)
 (Highlight, Silverback, notebook) is working    Get their landline & mobile phone number

@UXdna
Thank you!

         sarah@uxdna..c o. uk




@UXdna

Weitere ähnliche Inhalte

Ähnlich wie Remote customer feedback

Story mapping introduction short version
Story mapping introduction short versionStory mapping introduction short version
Story mapping introduction short version
Craig Brown
 
Tom Chi - Rapid Prototyping at Google X - MindTheProduct 2012
Tom Chi - Rapid Prototyping at Google X - MindTheProduct 2012Tom Chi - Rapid Prototyping at Google X - MindTheProduct 2012
Tom Chi - Rapid Prototyping at Google X - MindTheProduct 2012
Mind the Product
 
Conducting User Research for Startups - part 2
Conducting User Research for Startups - part 2Conducting User Research for Startups - part 2
Conducting User Research for Startups - part 2
Diane Loviglio
 
How Tools Have Shaped the Role of the Designer
How Tools Have Shaped the Role of the DesignerHow Tools Have Shaped the Role of the Designer
How Tools Have Shaped the Role of the Designer
Jonathan Lieberman
 

Ähnlich wie Remote customer feedback (20)

Story mapping introduction short version
Story mapping introduction short versionStory mapping introduction short version
Story mapping introduction short version
 
Morris prototyping - oredev - share
Morris   prototyping - oredev - shareMorris   prototyping - oredev - share
Morris prototyping - oredev - share
 
Tom Chi - Rapid Prototyping at Google X - MindTheProduct 2012
Tom Chi - Rapid Prototyping at Google X - MindTheProduct 2012Tom Chi - Rapid Prototyping at Google X - MindTheProduct 2012
Tom Chi - Rapid Prototyping at Google X - MindTheProduct 2012
 
What is this UX thing?
What is this UX thing? What is this UX thing?
What is this UX thing?
 
Remote Research at IxD10
Remote Research at IxD10Remote Research at IxD10
Remote Research at IxD10
 
Fast ways to create better products - by UX Rocks
Fast ways to create better products - by UX RocksFast ways to create better products - by UX Rocks
Fast ways to create better products - by UX Rocks
 
App For That : OSU-COE-ISE5640
App For That : OSU-COE-ISE5640App For That : OSU-COE-ISE5640
App For That : OSU-COE-ISE5640
 
Service Jam LA 2013 // Day 2
Service Jam LA 2013 // Day 2Service Jam LA 2013 // Day 2
Service Jam LA 2013 // Day 2
 
SVC UX class - Prototyping + Testing
SVC UX class - Prototyping + TestingSVC UX class - Prototyping + Testing
SVC UX class - Prototyping + Testing
 
The Developer Experience
The Developer ExperienceThe Developer Experience
The Developer Experience
 
Wireframe and prototyping google Campus talk by Zoe Guiraudon
Wireframe and prototyping google Campus talk by Zoe GuiraudonWireframe and prototyping google Campus talk by Zoe Guiraudon
Wireframe and prototyping google Campus talk by Zoe Guiraudon
 
Conducting User Research for Startups - part 2
Conducting User Research for Startups - part 2Conducting User Research for Startups - part 2
Conducting User Research for Startups - part 2
 
Creating Online Tutorials with Screencasting
Creating Online Tutorials with ScreencastingCreating Online Tutorials with Screencasting
Creating Online Tutorials with Screencasting
 
Life on Hold - Milan
Life on Hold - MilanLife on Hold - Milan
Life on Hold - Milan
 
COMP 4026 Lecture2: Design and Prototype
COMP 4026 Lecture2: Design and PrototypeCOMP 4026 Lecture2: Design and Prototype
COMP 4026 Lecture2: Design and Prototype
 
What is this UX thing 11-24-15
What is this UX thing 11-24-15What is this UX thing 11-24-15
What is this UX thing 11-24-15
 
What is this UX thing?
What is this UX thing?What is this UX thing?
What is this UX thing?
 
How Tools Have Shaped the Role of the Designer
How Tools Have Shaped the Role of the DesignerHow Tools Have Shaped the Role of the Designer
How Tools Have Shaped the Role of the Designer
 
Digital Storytelling by Gail Lovely
Digital Storytelling by Gail LovelyDigital Storytelling by Gail Lovely
Digital Storytelling by Gail Lovely
 
Nick Fine - Scientific Design
Nick Fine - Scientific Design Nick Fine - Scientific Design
Nick Fine - Scientific Design
 

Remote customer feedback

  • 2. How to get out of the building without getting out of the building @UXdna
  • 3. What I won’t talk about How to use different remote testing tools Advantages and pitfalls of each type (moderated, unmoderated, automatic, live, etc.) For this you can read Nate Bolt’s amazing book ‘Remote Research’ or watch his presentation here: http:// www.slideshare.net/boltpeters/remote-research-the-book-the-talk @UXdna
  • 4. This is about A practical guide on how to interview and observe customers remotely @UXdna
  • 6. Example A Spanish company wants to launch an online restaurant listing and booking service in Brazil. @UXdna
  • 7. The process 2. Who? Who you need to talk to/test with? 1. What? What do you need to learn 3. How? from it? How can you get there, step by step @UXdna
  • 9. 1. What? What do you need to learn from it? Interview Show + Interview Beta/Online product Prototype @UXdna
  • 10. 1. What? What do you need to learn from it? Interview Show + Interview Beta/Online product Prototype @UXdna
  • 11. 1. What? What do you need to learn from it? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP @UXdna
  • 12. 1. What? What do you need to learn from it? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP @UXdna
  • 13. 1. What? What do you need to learn from it? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP VOICE VOICE VOICE VOICE WEBCAM WEBCAM WEBCAM WEBCAM SCREEN SHARING SCREEN SHARING SCREEN SHARING @UXdna
  • 14. 1. What? What do you need to learn from it? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP VOICE VOICE VOICE VOICE WEBCAM WEBCAM WEBCAM WEBCAM SCREEN SHARING SCREEN SHARING SCREEN SHARING @UXdna
  • 15. Example An Axure prototype of a 3 page journey Will need to build and test prototype Will need screen sharing Will need voice Will need to define a scenario to users Will need to explain to users that this is a prototype, not a real site @UXdna
  • 16. Example An Axure prototype of a 3 page journey Will need to build and test prototype Will need screen sharing Will need voice Will need to define a scenario to users Will need to explain to users that this is a prototype, not a real site @UXdna
  • 17. 2. Who? RECRUITMENT A. Location B. Behaviours C. Channels D. What to say? What do you need Where are the What will you need to know about them people you will Where can you to say to get their to make them good need to interview/ find them? attention and how candidates for a test with? to do it test/interview? @UXdna
  • 18. 2. Who? Recruitment A. LOCATION Where are the people you will need to interview/test with? ACCESS TIME ZONE LANGUAGES VIEWERS TO TECHNOLOGY Are the potential Do I speak the local What is the country’s Will I be able to stream participants on my time language? internet connection to viewers as well? zone? speed like? Do I need to hire What are the times of translation? Skype says 300 kbps the day I will be able to but you need at least talk to them/test? Do I need to recruit 512 kbps participants who speak English? Speedtest.net @UXdna
  • 21. Example PORTUGUESE 5 HOURS BEHIND OK SPEED @UXdna
  • 22. 2. Who? Recruitment B. BEHAVIOURS What do I need to know about them? TIGHTLY DEFINED SET OF LOOSE SET OF BEHAVIOURS BEHAVIOURS I need to talk to teenagers who I need to talk to have an Android people who go phone and out dining and download at least check for good 3 new games a restaurants online week, every week @UXdna
  • 23. Example TIGHTLY DEFINED SET OF LOOSE SET OF BEHAVIOURS BEHAVIOURS We needed to know they were people who are likely to be our current hypothesis customers, i.e. anyone with a certain level of income (+ 5 minimum wages), who habitually looked for advice about places to eat online @UXdna
  • 24. Example TIGHTLY DEFINED SET OF LOOSE SET OF BEHAVIOURS BEHAVIOURS We needed to know they were people who are likely to be our current hypothesis customers, i.e. anyone with a certain level of income (+ 5 minimum wages), who habitually looked for advice about places to eat online @UXdna
  • 25. 2. Who? Recruitment C. CHANNEL Where can I find them? Easier Difficult List of emails from Publish an ad or Intercept as they Call centre contacts current customers post in a forum use website LIST OF EMAIL ADDRESSES or PHONE NUMBERS @UXdna
  • 26. Example Easier Difficult List of emails from Publish an ad or Intercept as they Call centre contacts current customers post in a forum use website We published a job position for NOTE: It is important to ‘site tester’ in free job posting get an idea of what sites websites and ‘Craigslist’ style people use in the country you are testing lists @UXdna
  • 27. Example Easier Difficult List of emails from Publish an ad or Intercept as they Call centre contacts current customers post in a forum use website We published a job position for NOTE: It is important to ‘site tester’ in free job posting get an idea of what sites websites and ‘Craigslist’ style people use in the country you are testing lists @UXdna
  • 28. 2. Who? Recruitment D. WHAT TO SAY? D1 D2 D3 Create a screener Write an email Collect survey or list of questions through which you results, filter and with which you will send the SURVEY select SURVEY LINK ANSWERS can filter out survey link unsuitable participants @UXdna
  • 29. 2. Who? Screener questionnaire D. WHAT TO SAY? What makes a good screener? A GOOD SURVEY MAKES IT EASY TO FILTER 1 2 3 4 List 3 List 3 behaviours Based on these Don’t ask ‘would characteristics that make factors, write you’ questions. someone should someone unlikely questions that Ask ‘what was the display to make to be your have discrete last time you...’ them likely to be a customer answers (like ‘yes’ questions customer and ‘no’ or multiple choice) @UXdna
  • 30. Example 1 2 3 4 List 3 List 3 behaviours Based on these Don’t ask ‘would characteristics that make factors, write you’ questions. someone should someone unlikely questions that Ask ‘what was the display to make to be your have discrete last time you...’ them likely to be a customer answers (like ‘yes’ questions customer and ‘no’ or multiple choice) Must live in SP Only seek Where do you live? What do you recommendations (a-d) usually do... Must have dined in from friends a restaurant about When was the last Would you which they found Always go to the time you visited a generally... reviews online in same restaurant, new restaurant? the last 7 days don’t like trying new places Which, of the sites Must have below, have you consulted at least Only dine out for visited in the last 7 1 review site very special days? occasions @UXdna
  • 31. Example 1 2 3 4 List 3 List 3 behaviours Based on these Don’t ask ‘would characteristics that make factors, write you’ questions. someone should someone unlikely questions that Ask ‘what was the display to make to be your have discrete last time you...’ them likely to be a customer answers (like ‘yes’ questions customer and ‘no’ or multiple choice) Must live in SP Only seek Where do you live? What do you recommendations (a-d) usually do... Must have dined in from friends a restaurant about When was the last Would you which they found Always go to the time you visited a generally... reviews online in same restaurant, new restaurant? the last 7 days don’t like trying new places Which, of the sites Must have below, have you consulted at least Only dine out for visited in the last 7 1 review site very special days? occasions @UXdna
  • 32. 2. Who? Screener questionnaire MY SURVEY Only show me answers from people who replied marked the option ‘last 7 days’ to the question “When was the last time you looked online for restaurant reviews?” 456 respondents = 36 fit our criteria @UXdna
  • 33. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? Hi, You are receiving this email because you replied to a job post on [name of website you used here]. My name is [your name here] and I am a researcher/CEO/Founder at [name of company here], ([website link here, if it exists]). We are working on a project about leisure habits in Brazil. We would like to count on you for a phone interview, but beforehand we would like to know a little more about you. Could you please answer the questions on the following link (link to survey, very visible) @UXdna
  • 34. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? Hi, This is why you are receiving this email from someone you don’t know. YouYou are interestedemail because yousay, keep onjob post on [name of are receiving this in what I have to replied to a reading. website you used here]. My name is [your name here] and I am a researcher/CEO/Founder at [name of company here], ([website link here, if it exists]). We are working on a project about leisure habits in Brazil. We would like to count on you for a phone interview, but beforehand we would like to know a little more about you. Could you please answer the questions on the following link (link to survey, very visible) @UXdna
  • 35. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? Hi, This is why you are receiving this email from someone you don’t know. YouYou are interestedemail because yousay, keep onjob post on [name of are receiving this in what I have to replied to a reading. website you used here]. My name is [your name here] and I am a researcher/CEO/Founder at [name of company here], I([website link here, am exists]). We are working on a project This is who am, this is what I if it working on and these are my credentials about leisure habits in Brazil. We would like to count on you for a phone interview, but beforehand we would like to know a little more about you. Could you please answer the questions on the following link (link to survey, very visible) @UXdna
  • 36. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? Hi, This is why you are receiving this email from someone you don’t know. YouYou are interestedemail because yousay, keep onjob post on [name of are receiving this in what I have to replied to a reading. website you used here]. My name is [your name here] and I am a researcher/CEO/Founder at [name of company here], I([website link here, am exists]). We are working on a project This is who am, this is what I if it working on and these are my credentials about leisure habits in Brazil. We would like to count on you for a phone interview, but beforehand we would like Weknow a little more including you in the research, but I needquestionsif you to are interested in about you. Could you please answer the to know on the are the right person first following link (link to survey, very visible) @UXdna
  • 37. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? Hi, This is why you are receiving this email from someone you don’t know. YouYou are interestedemail because yousay, keep onjob post on [name of are receiving this in what I have to replied to a reading. website you used here]. My name is [your name here] and I am a researcher/CEO/Founder at [name of company here], I([website link here, am exists]). We are working on a project This is who am, this is what I if it working on and these are my credentials about leisure habits in Brazil. We would like to count on you for a phone interview, but beforehand we would like Weknow a little more including you in the research, but I needquestionsif you to are interested in about you. Could you please answer the to know on the are the right person first following link (link to survey, very visible) VERY VISIBLE LINK TO SURVEY @UXdna
  • 38. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? We will be in touch if you are selected for the interview. You should be available for a phone call on the [day & date] or [day & date] morning. We will call you using Skype, and the whole interview shouldn’t last more than 40 minutes. You should provide us with a Skype name or a landline number we can call you on. Lastly, I would like to let you know that the payment for your time will be made via Paypal, so you must have an active Paypal account. Thank you already for your interest in participating Best regards [your name here] @UXdna
  • 39. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? We Next steps and if you are selected for the interview. You should be available, will be in touch confirmation of availability (dates). If you are not available for a phone call on the [day further or [day & date] morning. don’t bother going any & date] We will call you using Skype, and the whole interview shouldn’t last more than 40 minutes. You should provide us with a Skype name or a landline number we can call you on. Lastly, I would like to let you know that the payment for your time will be made via Paypal, so you must have an active Paypal account. Thank you already for your interest in participating Best regards [your name here] @UXdna
  • 40. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? We Next steps and if you are selected for the interview. You should be available, will be in touch confirmation of availability (dates). If you are not available for a phone call on the [day further or [day & date] morning. don’t bother going any & date] We will call you using Skype, and the whole interview shouldn’t last more than 40 minutes. You of what provide us with a have: Skype, or a landline number we Explanation should you will need to Skype name phone,computer with canunrestricted internet connection, etc. call you on. Lastly, I would like to let you know that the payment for your time will be made via Paypal, so you must have an active Paypal account. Thank you already for your interest in participating Best regards [your name here] @UXdna
  • 41. 2. Who? Email contact D. WHAT TO SAY? What makes a good contact email? We Next steps and if you are selected for the interview. You should be available, will be in touch confirmation of availability (dates). If you are not available for a phone call on the [day further or [day & date] morning. don’t bother going any & date] We will call you using Skype, and the whole interview shouldn’t last more than 40 minutes. You of what provide us with a have: Skype, or a landline number we Explanation should you will need to Skype name phone,computer with canunrestricted internet connection, etc. call you on. Lastly, I would like to let you know that the payment for your time will be made via Paypal, so you must have an active Paypal account. How I will pay for your time. If you are not happy, let me know Thank you already for your interest in participating Best regards [your name here] @UXdna
  • 42. 3. How? Show time! A. Set up B. Pilot C. Test D. Post test What you are Test your whole testing and how What can I improve setup. What can go Show time! you are collecting for next time? wrong? feedback @UXdna
  • 43. 3. How? A. SET UP What do I need to prepare for the test? @UXdna
  • 44. 3. How? A. SET UP What do I need to prepare for the test? Interview Show + Interview Beta/Online product Prototype @UXdna
  • 45. 3. How? A. SET UP What do I need to prepare for the test? Interview Show + Interview Beta/Online product Prototype @UXdna
  • 46. 3. How? A. SET UP What do I need to prepare for the test? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP @UXdna
  • 47. 3. How? A. SET UP What do I need to prepare for the test? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP @UXdna
  • 48. 3. How? A. SET UP What do I need to prepare for the test? Interview Show + Interview Beta/Online product Prototype JOURNEY JOURNEY + SCRIPT OR SCRIPT OR + A PROTOTYPE TOPIC MAP TOPIC MAP SCRIPT OR + TOPIC MAP SCRIPT OR TOPIC MAP VOICE VOICE VOICE VOICE WEBCAM WEBCAM WEBCAM WEBCAM SCREEN SHARING SCREEN SHARING SCREEN SHARING @UXdna
  • 49. Example A scenario so you can situate people: “Imagine it is Sunday morning and you need to find a place to have lunch with a friend who just Prototype arrived in town. You want to try something new” Prototype was built with wireframes. It was important to let participants know that they JOURNEY were not going to be looking at a real website, + even though buttons were clickable A PROTOTYPE + SCRIPT OR Themes and questions you would like to TOPIC MAP discuss with participants. Do they understand the order of the search results list? Do they see and understand the ratings ? VOICE WEBCAM Does my set up work? SCREEN SHARING Test all technology works and is compatible RECORDING with each other @UXdna
  • 51. 3. How? PREPARE & ANTICIPATE @UXdna
  • 52. 3. How? PREPARE & ANTICIPATE What can go wrong, will go wrong @UXdna
  • 53. 3. How? PREPARE & ANTICIPATE What can go wrong, will go wrong PILOT, PILOT, PILOT! HAVE A FALL BACK PLAN @UXdna
  • 54. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test @UXdna
  • 55. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Turn off webcam Internet connection: •Speedtest.net can give you an idea of •Kill other applications gets really slow or drops the connection speed - or at least let •Connections often better at completely you know if participant has dial speed weekends or holidays •Test out screen sharing Screen sharing: •Send ‘how to’ step by step •Alternative screen sharing drags a lot or stops completely screenshots. platform (joinme, WebEx, or participant can’t start it •Check Skype version (4.0 ok, 5.0 GoToMeeting, Co-pilot) better) Can’t hear participant •Test your mic & speakers •Call a landline and still do He can hear you, but you can’t •Test their mic & speakers screen sharing hear him - or the other way around @UXdna
  • 56. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Turn off webcam Internet connection: •Speedtest.net can give you an idea of •Kill other applications gets really slow or drops the connection speed - or at least let •Connections often better at completely you know if participant has dial speed weekends or holidays •Test out screen sharing Screen sharing: •Send ‘how to’ step by step •Alternative screen sharing drags a lot or stops completely screenshots. platform (joinme, WebEx, or participant can’t start it •Check Skype version (4.0 ok, 5.0 GoToMeeting, Co-pilot) better) Can’t hear participant •Test your mic & speakers •Call a landline and still do He can hear you, but you can’t •Test their mic & speakers screen sharing hear him - or the other way around @UXdna
  • 57. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Turn off webcam Internet connection: •Speedtest.net can give you an idea of •Kill other applications gets really slow or drops the connection speed - or at least let •Connections often better at completely you know if participant has dial speed weekends or holidays •Test out screen sharing Screen sharing: •Send ‘how to’ step by step •Alternative screen sharing drags a lot or stops completely screenshots. platform (joinme, WebEx, or participant can’t start it •Check Skype version (4.0 ok, 5.0 GoToMeeting, Co-pilot) better) Can’t hear participant •Test your mic & speakers •Call a landline and still do He can hear you, but you can’t •Test their mic & speakers screen sharing hear him - or the other way around @UXdna
  • 58. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test @UXdna
  • 59. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Recruit spare participants or ‘floaters’ for each slot allocated for test/ No show interview, who will be waiting in case •If both participant and floater participant doesn’t show up at someone doesn’t show up fail you, check if other floaters show time •They only get paid if they are needed can fill the gap. (you should make this clear when recruiting) •Send them a link to the prototype/ mock prototype (if you don’t want to Prototype/site show real thing before the test) Can participants see the prototype •Screen share it! •Don’t waste time, get your exactly as I planned? •Send link to a friend first spare participant in there Can users access the site? •Ask them to identify something on the screen @UXdna
  • 60. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Recruit spare participants or ‘floaters’ for each slot allocated for test/ No show interview, who will be waiting in case •If both participant and floater participant doesn’t show up at someone doesn’t show up fail you, check if other floaters show time •They only get paid if they are needed can fill the gap. (you should make this clear when recruiting) •Send them a link to the prototype/ mock prototype (if you don’t want to Prototype/site show real thing before the test) Can participants see the prototype •Screen share it! •Don’t waste time, get your exactly as I planned? •Send link to a friend first spare participant in there Can users access the site? •Ask them to identify something on the screen @UXdna
  • 61. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test @UXdna
  • 62. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Try recording the conversation you •Turn off your webcam Recording have during the pilot session •Record voice only Does my recording set up work •WARNING: Silverback is a little odd •Use spare participant with the screen sharing and voice? when webcam + screen sharing! • Language •You shouldn’t get to the actual I can’t understand what the •Have a conversation with the test with someone who participant is saying (mine or participant ahead of time and you will doesn’t fit. participant’s language skills aren’t be able to gauge how you •If you did, and only realise it at as good as I thought, accent is understand each other the time of test, call in a spare really difficult to understand) @UXdna
  • 63. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •Try recording the conversation you •Turn off your webcam Recording have during the pilot session •Record voice only Does my recording set up work •WARNING: Silverback is a little odd •Use spare participant with the screen sharing and voice? when webcam + screen sharing! • Language •You shouldn’t get to the actual I can’t understand what the •Have a conversation with the test with someone who participant is saying (mine or participant ahead of time and you will doesn’t fit. participant’s language skills aren’t be able to gauge how you •If you did, and only realise it at as good as I thought, accent is understand each other the time of test, call in a spare really difficult to understand) @UXdna
  • 64. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test @UXdna
  • 65. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •WARNING: first impressions can be deceiving, but talking to the Bad recruiting: participant will give you an idea. You the participant doesn’t sound like a •Spare participant can also ask extra screening customer questions, or check answers in their questionnaire. •It will always be a problem Rapport •The pilot helps! Main drawback of not being in the •If at the time of testing •Smile & be nice room with participant is: How can I participants don’t feel •Webcam, so they know you are not a make them trust me and tell me comfortable for some reason, machine stuff? don’t insist, call in your spare! •Show your surroundings, if you can (don’t ask them to do the same) @UXdna
  • 66. 3. How? B. PILOT What problems should I be trying to mitigate in a pilot? Problem What to check? Fallback during test •WARNING: first impressions can be deceiving, but talking to the Bad recruiting: participant will give you an idea. You the participant doesn’t sound like a •Spare participant can also ask extra screening customer questions, or check answers in their questionnaire. •It will always be a problem Rapport •The pilot helps! Main drawback of not being in the •If at the time of testing •Smile & be nice room with participant is: How can I participants don’t feel •Webcam, so they know you are not a make them trust me and tell me comfortable for some reason, machine stuff? don’t insist, call in your spare! •Show your surroundings, if you can (don’t ask them to do the same) @UXdna
  • 67. 3. How? C. TEST How to set it up 1. Establish Skype call with viewers (separate username) SKYPE 2. Turn off their webcam and mic VIEWER SKYPE PRO 3. Add participant to YOU the conversation SKYPE + Screen sharing 4. Ask them to turn on web cam 5. Explain who are the other people in PARTICIPANT the call 6. Begin test! @UXdna
  • 68. 3. How? Checklist C. TEST You Participant Check participant internet connection Check your own internet connection speed speed Check Skype Pro credit ($$) Check Skype mic & speakers Check screen sharing works and they Check Skype mic know how to do it Check they can see your site or prototype well Add participant to Skype contact list, so you know when they are available Check Paypal email address (might be different from the one participant used Check recording application/device when replying to ad) (Highlight, Silverback, notebook) is working Get their landline & mobile phone number @UXdna
  • 69. Thank you! sarah@uxdna..c o. uk @UXdna