We explore the shortcomings of today's voice user interfaces (like Alexa and Google Home) to chart a course for where the future should take us. Presented on October 20, 2017 at Webdagene in Oslo, Norway.
1. THE FUTURE OF VOICE
WEBDAGENE 2017
CHERYL PLATZ
Owner, IDEAPLATZ
Senior Designer, MICROSOFT
2. Iâve been designing for voice and
multimodal interfaces since 2006.
AT AMAZON:
First designer on Echo Look and
Alexa Notifications
AT MICROSOFT:
Designer for voice and multimodal
interfaces on Windows Automotive
and Cortana
WEBDAGENE 2017
COMPUTER, WHO IS CHERYL?
CHERYL PLATZ // @MUPPETAPHRODITE
3. WEBDAGENE 2017
HUMANS HAVE DEVELOPED THE
ART OF CONVERSATION FOR
THOUSANDS OF YEARS.
CHERYL PLATZ // @MUPPETAPHRODITE
4. The accessibility benefits are vast, and not just limited
to those with permanent accessibility challenges.
WEBDAGENE 2017
Voice user interfaces
leverage this experience
to improve lives.
CHERYL PLATZ // @MUPPETAPHRODITE
5. â
â
My wife passed away 4 years ago leaving me, not only a
widow, but a widowed quadriplegic trying to survive on
his own⊠Alexa has been a blessing beyond my
imagination. She has given me an opportunity that I
never thought would be possible.
AMAZON ECHO REVIEW FROM MICHAEL DAVIS, FEB 2017
DESCRIBING ECHOâS AID IN HIS LIFE AS A QUADRIPLEGIC
WEBDAGENE 2017 CHERYL PLATZ // @MUPPETAPHRODITE
6. WEBDAGENE 2017 CHERYL PLATZ // @MUPPETAPHRODITE
35.6 MILLION
AMERICANS USE A
VOICE-ACTIVATED
ASSISTANT DEVICE AT
LEAST ONCE A MONTH.
SOURCE: eMarketer
7. WEBDAGENE 2017
VOICE UI IS NOW MAINSTREAM, BUT
ITâS FAR FROM MATURE.
IN TODAYâS WEAKNESSES LIE THREE
KEY OPPORTUNITIES FOR THE FUTURE
OF VOICE UI.
CHERYL PLATZ // @MUPPETAPHRODITE
8. Limited training data and a an affluent user base is
excluding underrepresented groups with inaccuracy.
WEBDAGENE 2017
Todayâs voice interfaces
are inherently biased.
CHERYL PLATZ // @MUPPETAPHRODITE
OPPORTUNITY 1
9. â
â
ââŠlooking at race, I found that Caucasian
speakers had by far the lowest error rate. African-
American speakers and speakers with a mixed
racial background had higher error rates.
DR. RACHEL TATMAN, LINGUISTICS, UNIVERSITY OF WASHINGTON
ON ACCURACY OF SIRI FOR VARIOUS DEMOGRAPHIC GROUPS
KUOW, SEPTEMBER 19 2017
WEBDAGENE 2017 CHERYL PLATZ // @MUPPETAPHRODITE
10. GENDER
Systems were initially
trained with internal data
collection â at companies
where engineering teams
are still largely male.
ETHNICITY
Training data expands to
include early adopters,
often affluent.
This may exclude
underrepresented
ethnicities due to wage
gaps.
ACCENT
The North American
focus of most of todayâs
products mean we have
yet to attain critical mass
of training data for
second-language
speakers.
WEBDAGENE 2017
DECONSTRUCTING VOICE UI BIAS
CHERYL PLATZ // @MUPPETAPHRODITE
11. WEBDAGENE 2017 CHERYL PLATZ // @MUPPETAPHRODITE
Biased
Training Data
Poor Accuracy
for Excluded
Groups
High Attrition
by Excluded
Groups
BIAS SPIRAL
12. WE MUST FIND A WAY TO BREAK THE BIAS SPIRAL,
AND MAKE THE FUTURE OF VOICE UI VIABLE FOR ALL.
13. We are wasting time re-implementing the same basic
tasks on multiple systems. Most systems emphasize a
single modality at a time.
WEBDAGENE 2017
Todayâs voice interfaces
are simple and siloed.
CHERYL PLATZ // @MUPPETAPHRODITE
OPPORTUNITY 2
14. We currently have an ecosystem of voice assistants
chasing each othersâ tails.
What could we accomplish if we relied on each otherâs
expertise?
WEBDAGENE 2017
TIME LOST TO TIMERS
CHERYL PLATZ // @MUPPETAPHRODITE
17. WEBDAGENE 2017
DO WE NEED ONE ASSISTANT TO RULE THEM ALL?
CHERYL PLATZ // @MUPPETAPHRODITE
18. â
â
Through its collaboration with
Microsoft, Amazon said, Alexa users
will get answers to some of the same
questions that Cortana can now
answer â for instance, when is the next
budget review with the boss?
NICK WINGFIELD, NEW YORK TIMES
AUGUST 30, 2017
ILLUSTRATION: MENGXIN LI
WEBDAGENE 2017 CHERYL PLATZ // @MUPPETAPHRODITE
19. LETâS BUILD A CHOIR OF HARMONIOUS
VOICE INTERFACES TOGETHER.
20. Alexa, Google Home and Cortana essentially allow only
command-and-control scenarios.
WEBDAGENE 2017
Todayâs voice UIs arenât
conversational â yet.
CHERYL PLATZ // @MUPPETAPHRODITE
OPPORTUNITY 3
21. IT LOOKS LIKE YOU MIGHT BE IN THE
AWKWARD EARLY STAGES OF
CONVERSATIONAL UI. CAN I HELP?
PLEASE NO RUN AWAY
22. AUDIBLE CUES PHYSICAL CUES
WEBDAGENE 2017
Tone
Speed
Volume
Eye contact & gaze
Heart rate
Posture
Gesture
SPOKEN CONVERSATION IS MORE THAN WORDS
CHERYL PLATZ // @MUPPETAPHRODITE
23. WEBDAGENE 2017 CHERYL PLATZ // @MUPPETAPHRODITEClip from âHerâ: Warner Brothers / Anapurna Pictures
25. WEBDAGENE 2017
WHAT BENEFIT CAN HUMANS
GAIN FROM TRUSTING THESE
ASSISTANTS?
CHERYL PLATZ // @MUPPETAPHRODITE
26. â
â
The other night, I found Gary playing his own
version of a memory game with Alexa. He was
trying to come up with songs he remembered and
hadn't heard for awhile and would ask her to play
them.
AMAZON ECHO REVIEW FROM ALEX S.
DESCRIBING ECHOâS AID IN HUSBANDâS STRUGGLE WITH PARKINSONâS
WEBDAGENE 2017 CHERYL PLATZ // @MUPPETAPHRODITE
28. â
â
People have serious conversations with Siri. People talk
to Siri about all kinds of things, including when theyâre
having a stressful day or have something serious on
their mind. They turn to Siri in emergencies or when they
want guidance on living a healthier life.
APPLE JOB POSTING, SIRI SOFTWARE ENGINEER, HEALTH AND WELLNESS
APRIL 4, 2017
WEBDAGENE 2017 CHERYL PLATZ // @MUPPETAPHRODITE
29. WEBDAGENE 2017 CHERYL PLATZ // @MUPPETAPHRODITEClip from âHerâ: Warner Brothers / Anapurna Pictures
30. âȘ How can we (ethically) model a relationship over time?
âȘ What information is saved, and what is discarded?
âȘ What level of transparency and control is required?
âȘ Does the assistantâs personality adapt, or remain fixed?
WEBDAGENE 2017
WHAT DOES A RELATIONSHIP LOOK LIKE?
CHERYL PLATZ // @MUPPETAPHRODITE
32. âȘ Inclusive and unbiased speech recognition
âȘ Harmonious cross-product partnerships
âȘ Semantic web represents common knowledge
âȘ Trust built over time with shared context
âȘ Conversation informed by non-verbal cues
WEBDAGENE 2017
THE FUTURE OF VOICE
CHERYL PLATZ // @MUPPETAPHRODITE
33. WEBDAGENE 2017
THESE ADVANCES WILL COMBINE
TO OPEN NEW OPPORTUNITIES
AND A NEW ERA IN HUMAN
EMPOWERMENT.
CHERYL PLATZ // @MUPPETAPHRODITE
37. WEBDAGENE 2017 CHERYL PLATZ // @MUPPETAPHRODITEClip from Star Trek IV: The Voyage Home / Paramount Pictures
38. WEBDAGENE 2017
LETâS BUILD A FUTURE OF
INTERFACES WHERE OUR
HUMANITY IS AMPLIFIED,
NOT ATROPHIED.
CHERYL PLATZ // @MUPPETAPHRODITE
39. May the voice be with you.
http://ideaplatz.com
WEBDAGENE 2017
CHERYL PLATZ
Owner, IDEAPLATZ -- Senior Designer, MICROSOFT
Twitter & Medium: @MuppetAphrodite