Speech Technology and how it will Transform Medicine

HEALTHCARE SOLUTIONS
Nick van Terheyden, MD,
Chief Medical Information Officer – CLU
Nuance Communications

Outline
• Medicine and the Importance of Documentation
• History of Speech Recognition
• Types of Speech Recognition technology
• Natural Language Understanding and its link to Speech Recognition
technology in health care
• Current and future issues

Medicine used to be simple, ineffective and
relatively safe
Now it is complex, effective and potentially
dangerous
Sir Cyril Chantler, Kings Fund Chantler C. The role and education of doctors in the delivery of health care.
Lancet 1999;353:1178-81u

Challenge – Clinical Knowledge-Processing Burden
Knowledge processing requirement
Years ago Today
This gap
injures patients
Knowledge processing capacity
“Current medical
practice relies
heavily on the
unaided mind to
recall a great
amount of detailed
knowledge – a
process which, to
the detriment of all
stakeholders, has
repeatedly been
shown unreliable”
Crane and Raymond
The Permanente Journal
Winter 2003 Volume 7 No.1
Kaiser Permanente Institute for
Health Policy
Slide Courtesy of Dr Michael Bainbridge

Unstructured
Data
Direct data entry,
not physician
Structured Data
Dictation
and
Transcriptio
n
System
generated or
interfaced
data
Direct data entry,
physician
Handwritten
Current Methods for Data Capture

First Success
Bell Labs 1939 World’s Fair Voder

History Past, Present and Future
• 1964 Star Trek
"Computer, compute to the last digit the value of pi" -- Spock (Wolf in
the Fold episode)
• 1968 2001 A Space Odyssey
Dave Bowman: Open the pod bay doors, HAL.
HAL: I’m sorry Dave, I’m afraid I can’t do that[i].

Speech Recognition Time Line
• 1870’s Alexander Graham Bell
• 1939 Voder Speech Synthesizer
• 1952 Bell Labs Speech Recognition 0-9 digits over the telephone
• 1970’s HARPY Carnegie Mellon – 50 computers to process
• 1970’s Hidden Markov Modeling approach
• 1990’s Discrete Speech Recognition
• 1995 Continuous Speech Recognition Power PC Chip with Intels
Pentium 200Hz chip

What is Speech Recognition?
Automatic conversion of spoken words to
computer text

Converting Sounds to Speech

Tools for Converting Sound to Text
• Statistical Language Model
• Grammars

Types of Speech Recognition in Healthcare
• Front End or Real Time
• Server Based Speech Recognition
• Physician Self Edit

Hurdles to Adoption
• Pronunciation
• Separating Background Noise
• Resolving Ambiguity
– I made her duck
– I cooked waterfowl for her
– I cooked waterfowl belonging to her
– I created the (ceramic) duck she owns
– I caused her to quickly lower her head or body
– I waved my magic wand and turned her into a undifferentiated waterfowl
• Grammar and Punctuation

What you Speak is Not what you get
• Clinicians do not typically speak in structured coherent format
• Literal speech recognition does not provide enough value
• Newer solutions provide speech recognition combined with some
elements of Natural Language Processing to develop and
Understanding of What was meant vs what was said

Challenges – A perfect speaker?
POSTOPERATIVE_DIAGNOSIS
[AAHHMMMMM] Status post complete oral and dental rehabilitation PERIOD
NEXT_PARAGRAPH [OOHHMMMMM]
[NEXT SECTION]
OPERATION PERFORMED Complete oral and dental rehabilitation PERIOD
NEXT_PARAGRAPH
[cough]
ANESTHESIA
General anesthesia by the anesthesia staff PERIOD NEXT_LINE
Duration of surgery: Close to 45 minutes.
Incision: none.
NEXT_PARAGRAPH
[NEXT SECTION]
[________________][paper rustling] [________________]
FINDINGS [AAHHMMMMM] The patient is a 22-year-old male with missing teeth and permanent dentition
COMMA mental retardation COMMA seizure disorder COMMA microcephaly COMMA and
gastroesophageal reflux disease PERIOD Since [__] March first 2004 COMMA there has not been much
change in the appearance of the patient PERIOD [AAHHMMMMM] The patient also had periodontal
disease evidenced by moderate bone loss COMMA hyperplastic gingiva and to three to seven millimeter
pockets PERIOD
NEXT_PARAGRAPH
This is the end of dictation, thank you

Final Report
POSTOPERATIVE DIAGNOSIS
Status post complete oral and dental rehabilitation.
OPERATION PERFORMED
Complete oral and dental rehabilitation.
ANESTHESIA
General anesthesia by the anesthesia staff.
Duration of surgery: Close to 45 minutes.
Incision: none.
FINDINGS
The patient is a 22-year-old male with missing teeth and permanent dentition, mental retardation, seizure
disorder, microcephaly, and gastroesophageal reflux disease. Since March 1st 2004, there has not been
much change in the appearance of the patient. The patient also had periodontal disease evidenced by
moderate bone loss, hyperplastic gingiva and to 3-7 millimeter pockets.
John Smith, MD
Philips Hospital
Bethesda Medical Center
Maryland

Natural Language Processing
The Technology and How it Works

Speech is Reinventing the relationship
between people and technology
• At the forefront of the next
chapter of human-computer
interaction
• Deeply invested in creating
effortless and natural user
experiences
• Rapid advances in voice-recognition
technology

CHIEF COMPLAINT
PAST MEDICAL HISTORY

PAST MEDICAL HISTORY
CHIEF COMPLAINT

From Clinical Narrative to Facts
CLINICAL LANGUAGE UNDERSTANDING
63 %
93%
86 %
Oct ‘10 Feb ‘12
Rules: explicit linguistic models
Machine learning: discovery of new patterns
Syntactic parsing & Statistical semantic processing

In EHR
MT-editing
Feedback
EHR
Quality
Reporting
Coding &
Business
Intelligence
Real time
data &
feedback
EPR
Quality Metrics
Codes / Clinical Doc
Improvement
Outcomes
Clinical Facts
Narrative
With a
RIS/PACS
In EHR
Self-editing
On the Go
On a PC
At a Dictation
Station
On an
MFP

Possibly the first step in this
journey could be instant
access on mobile platform

NLU Technology – Across Industries
BRINGING THE POWER OF LANGUAGE UNDERSTANDING
Across a Number of Domains
100+ apps and counting
Dragon Go! Cluzee Evi
Dragon Assistant Voice Ask Google Cleverbot
Speaktoit Siri Ask Ziggy
Andy
iris
Skyvi Pannous EVA

Customer care
What’s the current rate
for a 30 year fixed rate mortgage?
Donna: The most recent published rate for a
qualifying 30 year fixed rate mortgage is
4.750%/4.878% APR
OK, how about variable?
Donna: For a variable mortgage, the most
recent published rate is 3.5%/3.286% APR
OK. Are there any other
options I should look at?
Donna: We also offer 15 year fixed rate
mortgages – would you like to hear more
about them?
RISING EXPECTATIONS
Consumer apps creating expectations of
natural interactions
NLU promises to cut through deep menus
Transition from a ROI-model to a 'must-have’
Native apps with migration path to
browsers and standards
Multi-platform, multi-channel

Intelligent Voice Interactions
• Minimize need to remember “gate commands” &
“command order”
• Allow multiple forms of expression and handle
ambiguous input
• Allow for more conversational spoken utterances,
less utilitarian
• Reduce distance between complex spoken
commands and desired outcome or action
– finding becomes doing
• Customizable to provide levels of seamless
access to any content on device

The Current State of On-line Content
• Input is a keyword
– Not keywords – “Treatment of onychomycosis”
– Just a single keyword – “onychomycosis”
– The result is a “chapter” to scroll through
• The need is for more granular input leading to more granular results
– At a minimum, the right page, not the right chapter
– The ultimate goal is the right paragraph
– “onychomycosis treatment” is multiple keywords
• To really improve the physician experience, natural language
– “What is the treatment of oncyhomycosis?”
• Even better with real understanding
– “How do you treat toe-nail fungus?”
• And this is just the first step…

Natural language interface to allow doctor
to request information in a comfortable way.
“Show me his last chest xray”
</intent=SHOWXRAY>
</type=chest>
</time=latest>
Application maintains patient context and
assumes request is for current patient
A Possible Future Platform
Use Cases – Voice Enable a EHR
“Labs for James Jones”
“Show me lab results for Jones”
“CBC results for James Jones”
“James Jones, lab results”
“Last labs for James Jones”
DM360
Platform
NLU
</intent=LABRESULTS>
</subject=Jones, James>
Application receives XML and displays
lab results for the specified patient.

Future Hospital Label Warning
• Please be advised that this hospital uses manual, paper-based
methods for tracking the process of your care and
for implementing the orders of your physicians.
Therefore, many orders that your doctors initiate will not
be carried out as written. As a result, you may regrettably
receive the wrong medicine, the wrong dose of the right
medicine, the wrong route of administration, or possibly
the correct medicine at the wrong time.
President’s Information Technology Advisory Committee
Revolutionizing Health Care Through Information Technology
http://www.hpcc.gov/pitac/meetings/2004/20040617/20040615_hit.pdf

Prediction
• Paper, manila notes and folders will
become memorabilia of the past and
relegated to museums

Nick van Terheyden, MD CMIO, Nuance Communications
AboutMe http://about.me/obiwan
Twitter http://twitter.com/drnic1
LinkedIn http://www.linkedin.com/in/nickvt
Voice of the Doctor http://drvoice.blogspot.com/
FaceBook http://profile.to/drnick
E-Mail drnick@nuance.com, drnic1@gmail.com
Google Voice (301) 355-0877
Where You Can Find Me

Speech Technology and how it will Transform Medicine

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (18)

Ähnlich wie Speech Technology and how it will Transform Medicine

Ähnlich wie Speech Technology and how it will Transform Medicine (20)

Mehr von Nick van Terheyden

Mehr von Nick van Terheyden (20)

Speech Technology and how it will Transform Medicine