Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.

Speech assistance in online surveys #GOR2018

452 Aufrufe

Veröffentlicht am

Holger Lütters, Malte Freksa and Marc Egger about speech in Online Research

Veröffentlicht in: Internet
  • Login to see the comments

  • Gehören Sie zu den Ersten, denen das gefällt!

Speech assistance in online surveys #GOR2018

  1. 1. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 1 General Online Research 2018 - Cologne, 1st March 2018 Track A2: Innovations for Online Surveys Holger Lütters – Hochschule für Technik und Wirtschaft Berlin Malte Friedrich-Freksa - GapFish GmbH Marc Egger - insius u.G. Effects of Speech Assistance in Online Questionnaires
  2. 2. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 2 Revolutionary “beyond literacy survey approach” of Ian Thomson is the inspiring framework of the presentation Georges Perec Ian Thomson
  3. 3. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 3 Digital Speech Assistance: A new technology available for market research 1
  4. 4. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 4 2013: Text2Speech - Avatars in Interviews 2018: Speech2Text - Talking instead of typing Question text read out loud by an avatar Research 2013 Text2Speech Avoidance of READING Research 2018 Speech2Text Avoidance of TYPING Lütters, Holger /Maree, Tania / Wiese, Melanie (2014): Selecting an interviewer: Considerations for Avatar use in online market research. In: Proceedings of the 26th annual SAIMS Conference, S. 264-272, The Southern African Institute for Management Scientists (SAIMS),
  5. 5. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 5 Digital Assistants conquer the lives from different entry points
  6. 6. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 6 Idea: Integration of Speech2Text technology in online survey process During an online survey mobile or PC one or several special Speech2Text question are implemented Prerequisites: ● Modern browser ● Microphone ● Browser allowance to use the microphone Respondent is talking into the device`s microphone during the survey question Spoken words are transcripted into written text Bonus technology option: Text is sent to a sentiment API during the interview to analyze the tonality of the spoken content in an ongoing interview. Different interview routes possible
  7. 7. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 7 Different Speech2Text options are available in SaaS tool questfox Background Speech2Text* without showing results of the transcript to the respondent Talk2Me* Live Speech2Text where transcript is shown to the respondent *Both questfox question types can save the audio answer in addition to the transcript of the text. www.questfox.com
  8. 8. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 8 • More open answers • More details in the open answers • Lower barriers for people with lower willingness to express themselves in a written form • Open answers with more content • More personal expressions in a less controlled setting • More System1 than System2 answers Digital Speech assistance could improve the interview interaction • Technical drop-out • Device drop-out • Tech-savvy sampling only • Mental reservation against new forms of interaction • German speech2text capabilities not yet developed on a usable level (fear of being misunderstood with a dialect) • Legal issues of dealing with audio information Research chances of the idea Risks of the approach
  9. 9. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 9 Study set-up and field report Empirical study comparing classic text input with speech assistance input in a survey approach 2
  10. 10. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 10 HTW Berlin project with the technical support from several partners Concept and programming of the study Holger Lütters Ian Thomson Software sponsor questfox Software as-a-Service with Speech2Text functionality Dima Feller Tarik Hennings Sampling sponsor with n>600 cases Malte Friedrich-Freksa Analytics sponsor with InMap specialized tool for text analytics and API development Marc Egger André Lange
  11. 11. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 11 Browser approach avoiding an App development is the opposite of device agnostic research https://caniuse.com/#search=MediaRecorder
  12. 12. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 12 2036 contacts needed to complete n=622 interviews with an online representative panel • 13 Minutes questionnaire • More Screen-outs than Drop-outs • August 2017 in Germany • 2036 contacts with EntscheiderClub by GapFish • 622 Completes n=127 n=181 n=37 n=277
  13. 13. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 13 Three different routes of interaction in the overall setup Route A Voice Only n=152 Route B Voice and Text n=151 Route C Text Only (control group) n=322 Answer Mode
  14. 14. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 14 Three routes with different question and answer modes integrated into a standard questionnaire I. Refugee situation and chancellor Merkel Bitte kommentieren Sie die folgende Aussage: “Während des Höhepunkts der Flüchtlingskrise im Jahr 2015 sagte Angela Merkel bezüglich dieses Themas den Satz "Wir schaffen das".“ II. Gay Marriage “Was ist Ihre Einstellung gegenüber der kürzlichen Legalisierung der "Homo-Ehe" in Deutschland?“ III. Women in the workplace Bitte kommentieren Sie die folgende Aussage: “Es wird genug für die Gleichberechtigung der Frau am Arbeitsplatz getan.” Three “delicate” questions to provoke open answers (NO randomization) Questionnaire example Example Link
  15. 15. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 15 The automated transcript largely depends on the technical setup nein wir schaffen das nicht das ist hochverrat am deutschen volk und dann muss national auf nationaler ebene dafür gesorgt werden das erst die menschen der heimatländer in nicht mehr in armut leben sondern gut leben das ist was wichtiges wir schaffen das nicht wir werden kaputt gehen das finanzsystem das wird zerbrechen und alles kaputt gehen hier Automated transcriptExamples extended answers Example 1 Question 1: Chancellor Merkel; Open answer recorded
  16. 16. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 16 A new phenomen of open answers pops-up: spoken bullshit answers • lalalalala • käsekuchen • ja was ist du arschloch • elfriede manchester Spoken „asdf“ type answers Question 2: Gay marriage; Open answer recorded an transcripted
  17. 17. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 17 Results of the comparison between typed-in and spoken answers 3
  18. 18. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 18 Voice technology comes at higher cost of recruiting Route A Voice Only Route B Voice and Text Route C Text Only
  19. 19. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 19 Average Word Count per Response shows fatigue when typing more than one open answer
  20. 20. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 20 Less response time for spoken answers with longer input with immediate fatigue/satisficing after the first task
  21. 21. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 21 Estimated time in comparison to real time is in favor for speech. Speech feels less time consuming than it actually is. Self-estimated time as indicated by respondent Real time measured Route A (n=152) Real time measured Route B (n=151) Real time measured Route C (n=106) Under 10 mins 14.65 13.88 10.33 From 10 to under 15 mins 15.93 15.73 12.36 From 15 to under 20 mins 19.67 20.06 13.88 Above 20 mins 24.69 26.76 21.64
  22. 22. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 22 Check for concepts shows more detailed answerswith speech Topic I Chancellor Merkel and the refugee situation Analysis with insius InMap http://insius.com/de/produkte/inmap/
  23. 23. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 23 People express more in a speaking mode compared to writing Topic II Gay marriage Analysis with insius InMap http://insius.com/de/produkte/inmap/
  24. 24. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 24 The burden of answering is lowered through the microphone usage Topic III Women in the workplace Analysis with insius InMap http://insius.com/de/produkte/inmap/
  25. 25. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 25 The concepts can be compared in terms of content. Overall result: More concepts occur in the speech version Analysis with insius InMap http://insius.com/de/produkte/inmap/
  26. 26. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 26 Evaluation of the question type Speech2Text 4
  27. 27. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 27 Summary Speech2Text opens a new door in the world of interview research • Speech as an answer type is a thrilling new way of getting more open ended answers … but not necessarily to the respondent :-( • People leave more answers and more detailed concepts in less time at the cost of 50% technical drop-out • The technical issues will be sorted out. In a panel approach this negative effect can be controlled. In river sampling approaches this remains an issue with strong effects • It will take some time before it will become a standard tool, but it will definitely change the research landscape for more qualitative input from less educated respondents • Further automation in the field of voice analytics will again change the rules of the game
  28. 28. Lütters/Freksa/Egger · Effects of Speech Assistance in Online Questionnaires #GOR18 · Cologne 1st March 2018 · @luetters 28 Thanks to the supporters of the study Authors Prof. Dr. Holger Lütters – HTW Berlin Malte Friedrich-Freksa - GapFish GmbH Dr. Marc Egger - insius u.G. Contact Prof. Dr. Holger Lütters Hochschule für Technik und Wirtschaft Berlin Holger.Luetters@htw-berlin.de www.luetters.com Thanks to Ian Thomson for his support Thanks to questionnaire www.gapfish.com www.insius.com with www.questfox.com questfox® is a development of pangea labs www.pangealabs.com

×