Weitere Ă€hnliche Inhalte Ăhnlich wie Voxeo Summit 2010: Best Practices in Speech Technology (12) KĂŒrzlich hochgeladen (20) Voxeo Summit 2010: Best Practices in Speech Technology1. State of the Art/Best Practices in
Speech Technology
Dan Burnett, Director of Speech Technologies
3. Speech is the natural human interface
ïŹâŻ 15% of world population has a personal
computer
ïŹâŻ Greater than 60% of world population has a
mobile phone
© Voxeo Corporation
6. But for IVRs . . .
Your
Customer
You
(Your untuned speech-enabled IVR)
© Voxeo Corporation
7. So why do we tune?
ïŹâŻ For better communication, which leads to
ïŒâŻMore satisfied customers
ïŒâŻShorter call durations
© Voxeo Corporation
8. What can we tune?
Your untuned speech-enabled IVR
© Voxeo Corporation
9. What can we tune?
Your untuned speech-enabled IVR
© Voxeo Corporation
10. What we say â prompts
ïŹâŻ Goal: naturally reduce variability in caller's
responses
ïŹâŻ Because: predictability simplifies grammars
and increases recognition accuracy
© Voxeo Corporation
11. Prompt tuning
ï⯠Vocabulary
âąâŻ Use the words your customers use
âąâŻ For sales, say âsalesâ; For billing, say âbillingâ; ...
âąâŻ Are you calling to learn more about our products, to
fix a problem with your bill, or âŠ
ï⯠Keep in mind
âąâŻ Speech allows your customer to describe things
THEIR way rather than to use your internal company
description
âąâŻ Make it easier for them to do that!
© Voxeo Corporation
12. Prompt tuning
ï⯠Prompt specificity
âąâŻ General: âWhat would you like?â
âąâŻ More specific: âWhich department would you like?â
âąâŻ Precise: âWould you like A, B, C, or something else?â
ï⯠Keep in mind
âąâŻ The caller will often use the exact words YOU use
© Voxeo Corporation
13. Ever heard this before?
ï⯠For Sales, press 1
ï⯠For Billing, press 2
ï⯠For option I can't remember, press 3
ï⯠For another option I can't remember, press 4
ï⯠For yet another option I can't remember, press 5
ï⯠For more of the same, press 6
ï⯠Blah blah, press 7
ï⯠For help with this menu, press 8
ï⯠To hear these options again, press 9
© Voxeo Corporation
14. Prompt tuning
ï⯠Prompt length
âąâŻ Keep it short: less than a few sentences total, only
one of which asks for input
âąâŻ Or: provide pauses (at least one second long) for
interruption
ï⯠Keep in mind
âąâŻ Speech communication is only natural if it's not
drawn out
âąâŻ Primacy and recency effects
© Voxeo Corporation
15. What can we tune?
Your untuned speech-enabled IVR
© Voxeo Corporation
16. What we listen for â grammars
ïŹâŻ Goal: Cover everything they are likely to say,
and nothing more
ïŹâŻ Because: Accuracy in grammar coverage
directly affects recognition accuracy
© Voxeo Corporation
17. Grammar tuning
ï⯠Cover everything they say
âąâŻ Pre- and post- phrases such as please, I would like,
and thank you
âąâŻ Synonyms such as (for yes/no) yeah, sure, absolutely
not
ï⯠Keep in mind
âąâŻ Recognizers can only hear it if it's in the grammar
© Voxeo Corporation
18. Grammar tuning
ï⯠Include only what they say
âąâŻ Write grammars that don't overgenerate
âąâŻ If matching numbers/digits, only include valid strings
if at all possible
ï⯠Keep in mind
âąâŻ Every unnecessary grammar phrase is a potential
misrecognition
© Voxeo Corporation
19. What can we tune?
Your untuned speech-enabled IVR
© Voxeo Corporation
20. How we listen â parameter optimization
ïŹâŻ Goal: Optimize recognizer parameter settings
ïŹâŻ Because: Better accuracy, of course!
© Voxeo Corporation
21. Parameter optimization â which parameters?
ïŹâŻ Rejection threshold
ïŹâŻ Endpointer settings (sensitivity)
ïŹâŻ Large grammar parameters
© Voxeo Corporation
22. Rejection threshold â what is it?
False
Rejections
Misrecognitions
0
Rejection Threshold
100
© Voxeo Corporation
23. Rejection threshold â what is it?
Cutoff value for the recognizer conïŹdence below False
which the speaker's utterance will be rejected
Rejections
Misrecognitions
0
Rejection Threshold
100
© Voxeo Corporation
24. Rejection threshold â total error
False
Rejections
Misrecognitions
0
Rejection Threshold
100
© Voxeo Corporation
26. Rejection threshold â comparison
ASR
Engine A
ASR
Engine B
Optimal thresholds
0
Rejection Threshold
100
© Voxeo Corporation
27. Rejection threshold â another
comparison
ASR
Engine A
Optimal thresholds
ASR
Engine B
0
Rejection Threshold
100
© Voxeo Corporation
28. Parameter optimization
ï⯠Rejection threshold
âąâŻ Generally largest impact on accuracy
âąâŻ Optimum varies across recognition engines
âąâŻ Optimum varies by set of active grammars
ï⯠Keep in mind
âąâŻ Optimizing the rejection threshold is the SINGLE
MOST IMPORTANT parameter tuning you can do
© Voxeo Corporation
30. Parameter optimization
ï⯠Endpointer sensitivity
âąâŻ Second-largest impact on accuracy
âąâŻ Unnecessarily high and low sensitivity are both bad
âąâŻ Optimum should be set once, checked annually
ï⯠Keep in mind
âąâŻ If the recognizer can't hear you, it can't understand
what you say
© Voxeo Corporation
31. Parameter optimization
ï⯠Large grammar parameters
âąâŻ Typically need to be adjusted if grammar has more
than 5000 entries
âąâŻ Typically consumes more memory and/or CPU
âąâŻ Vary by ASR engine, so ask
ï⯠Keep in mind
âąâŻ If your grammar has many options, your recognizer
needs to âthinkâ more than the default settings
usually allow
© Voxeo Corporation
32. What can we tune?
Your untuned speech-enabled IVR
© Voxeo Corporation
33. Summary â Keep in mind
ï⯠Speech allows your customer to describe things THEIR way rather than to
use your internal company description â make it easy for them!
ï⯠The caller will often use the exact words YOU use
ï⯠Speech communication is only natural if it's not drawn out
ï⯠Recognizers can only hear it if it's in the grammar
ï⯠Every unnecessary grammar phrase is a potential misrecognition
ï⯠Optimizing the rejection threshold is the SINGLE MOST IMPORTANT
parameter tuning you can do
ï⯠If the recognizer can't hear you, it can't understand what you say
ï⯠If your grammar has many options, your recognizer needs to âthinkâ more
than the default settings usually allow
© Voxeo Corporation
35. State of the Art/Best Practices in
Speech Technology
Dan Burnett, Director of Speech Technologiesâš
dburnett@voxeo.com