These are my #AI slides for medical deep learning using #radiology and medical imaging examples. Please use them & modify to teach your own group about medical AI.

1. Frank J. Rybicki MD, PhD
Professor and Chair, Department of Radiology Univ. of Ottawa
Introduction to Deep Learning as it Applies
to Radiology

2. Disclosure
Frank J. Rybicki
is the Medical Director
of Imagia Cybernetics M I C H E L A N G E L O
“Rondanini Pietà”
Milan, Italy

3. Please visit LinkedIn if you would like to see or download the slides

4. “…mobile computing, the proliferation of inexpensive sensors collecting
terabytes of data, and the rise of machine learning that can use that data,
will fundamentally change the way the global economy is organized.”
Fortune, “CEOs: The Revolution is coming” March 8, 2016
Reference: March 2016 issue of Fortune: “Oracles of Davos.”

5. Deep Learning
Machine Learning
Artificial Intelligence

6. Deep Learning
Machine Learning
Artificial Intelligence
Automated
Video Game
“Player”
Decision tree
algorithm: Are
we going to
play tennis?
Using a neural
network for
expert systems:
Does the data in
the CT scan
mean your
patient will live?

7. • Paucity of high-quality data.
• Pooling of resources is needed.
• Big data sets necessitate
wholesale anonymization for
integration.
• What is non-identifiable?
• Blockchain your semantically
annotated big data and promote
service interoperability?
Machine Learning can use a Decision Tree
Tennis example from: Learning from Data: Decision Trees. Amos
Storkey, School of Informatics University of Edinburgh

8. • Paucity of high-quality data.
• Pooling of resources is needed.
• Big data sets necessitate
wholesale anonymization for
integration.
• What is non-identifiable?
• Blockchain your semantically
annotated big data and promote
service interoperability?
Tennis example from: Learning from Data: Decision Trees. Amos
Storkey, School of Informatics University of Edinburgh

9. Deep Learning
A set of rules
Intended to execute an operation
With a level of independence from humans

10. Deep Learning
A set of rules
Intended to execute an operation
With a level of independence from humans

11. The Rules
Not a single entry
based on what one
person likes (e.g.
humidity). It is
based on a large set
of data that is used
to train the
computer to make a
set of rules with the
lowest error (cost). Andrej Karpathy & Li Fei-Fei
Dept of Computer Science, Stanford University

12. The Rules
I love tennis.
The first thing I see
here is this logo and
the scoreboard!
Was the computer
trained to see it?

13. Steps to Generate Rules
• Get data
• Clean, Prep, Manipulate data
• Train model
• Test model
• Improve

14. Steps to Generate Rules
• Get data
• Clean, Prep, Manipulate data
• Train model
• Test model
• Improve

15. Steps to Generate Rules
• Get data
• Clean, Prep, Manipulate data
• Train model
• Test model
• Improve

16. Steps to Generate Rules
• Get data
• Clean, Prep, Manipulate data
• Train model
• Test model
• Improve

17. Steps to Generate Rules
• Get data
• Clean, Prep, Manipulate data
• Train model
• Test model
• Improve
playground.tensorflow.org

18. TensorFlow Crash Course and Playground
If you are dedicated to learn, you can do it!
• Label (y): This is the yes/no truth: Is this a lung cancer
that will respond to a new immunotherapy
• Features (xi): The inputs. The scan data (not just a
report), clinical information, lab data
• Examples (x): The patients, each patient has features
(xi). Labeled examples are (x, y) where y is truth
• Model: What is doing the predicting. The model
outputs (x,y’). y’ is the guess of whether or not this is
lung cancer that will respond to new immunotherapy

19. TensorFlow Playground
If you are dedicated to learn, you can do it!

20. Deep Learning
A set of rules
Intended to execute an operation
With a level of independence from humans

21. What are the tasks?

22. Our task is different, because what we do is different.
globallymealliance.org
Journal of Thoracic and Cardiovascular
Surgery. 2004; 128(5):761-2

24. US FDA Commissioner Scott Gottlieb
“AI holds enormous promise for the future of medicine.” The FDA is
working on a “new regulatory framework” to allow regulators (e.g.
FDA employees) to “promote innovation in this space”… “We expect
to see an increasing number of AI-based submissions in the coming
years, starting with medical imaging devices, and we’re working with
experts in the field.”

25. Deep Learning
A set of rules
Intended to execute an operation
With a level of independence from humans

26. The brain and its neurons
• Logical
– Study protocolling
– Information integration
– Report QA/Dictation Assistance
– Quality Improvement Support
• Image-Based
– Diagnosis
– Segmentation
– Super-Resolution
– Workflow prioritization
– Automated lesion follow-up
– Image-based molecular pathologymy-ms.org Univ of Melbourne

27. Artificial Neural Network
• Logical
– Study protocolling
– Information integration
– Report QA/Dictation Assistance
– Quality Improvement Support
• Image-Based
– Diagnosis
– Segmentation
– Super-Resolution
– Workflow prioritization
– Automated lesion follow-up
– Image-based molecular pathology

28. Detect aneurysm and predict rupture

29. Will the aneurysm rupture?... Well:
How big is it?
What is the angle with the normal
artery?
Does it have lobulations?
What is the blood pressure?

30. AI pathways based on images
Transl Cancer Res. 2016 Aug; 5(4): 432–447

31. Lung Cancer AI

32. Can we automate Tb detection just inside the Arctic circle?
Photos from Iqaluit, Nunavut, 2016

33. Training for human intelligence and
artificial intelligence is completely
different. Is it astounding that the
two methods can have similar
results?
There are several challenges: training
Curry International Tb Center

34. Supervised versus Unsupervised Learning
Applications for medical AI
have used supervised
learning
Deep clinical impact of AI
will come with
unsupervised learning
gocitrusnow.com

35. Supervised versus Unsupervised Learning
Applications for medical AI
have largely used
supervised learning
Deep clinical impact of AI
will come with
unsupervised learning

36. Personal Value
Medical Value
Social Value
There are several challenges: data

37. Problems for Radiology?

38. There are big opportunities
1. If there are fewer radiologists, they
will be more informed will make
healthcare more efficient and
economical
2. AI will benchmark a new set of
human “loss functions”
3. Seeing different solutions will enable
us to think creatively, and better
position those with creativity
4. AI can only enhance natural
intelligence, it can’t decrease it
osirix-viewer.com

39. Last example: Contrast in blood vessels from
first-pass CT angiography
Enhancing Natural Intelligence

40. Angiogram
Can We Assess Blood Flow?
1. Contrast Enhancement
2. Computational Fluid Dynamics
Perfusion
Lesion
Hemodynamic
Significance
Conduit
Blood
Flow
Organ
Pathophysiology Accessible by Coronary CTA

41. 300 HU
238 HU
0
100
200
300
400
0 2 4 6 8 10
MeanHU
Distance From Ostium (cm)
@0cm
@9.5cm1 mm intervals (not to scale)
0
100
200
300
400
0 2 4 6 8 10
MeanHU
Distance From Ostium (cm)
1 mm intervals (not to scale)
Mean HU = -8.3×Distance + 316.1
r = 0.845
0
100
200
300
400
0 2 4 6 8 10
MeanHU
Distance From Ostium (cm)
HU at
ostium
HU at 2.5 mm
diameter
300 HU 238 HU
Steigner ML et al. Circ Cardiovasc Imaging. 2010;3:179-186
Contrast Gradients (TAG)

42. Normalized 64 CT Differences in Correlate with TIMI Flow
Chow BJW et al. CT Contrast Opacfication Predicts Coronary Flow. JACC, 2011(57):1280-1288

43. Relationship Between
TAG320 and FFR
“..TAG320 assessment significantly
improves both sensitivity and
specificity to detect clinically relevant
reduction in FFR.”
Some say it is good

44. Some people do it backwards

45. Is it great? Or not?

46. Let the computer do the talking…
• Logical
– Study protocolling
– Information integration
– Report QA/Dictation Assistance
– Quality Improvement Support
• Image-Based
– Diagnosis
– Segmentation
– Super-Resolution
– Workflow prioritization
– Automated lesion follow-up
– Image-based molecular pathology

47. N=254 (484 vessels) analyzed for all features. Highest predictive value
for contrast density difference
Machine learning for integrating plaque features
to predict ischemia
Dey et al. European Radiology 2018

48. Break-down of Radiology Applications
• Text-Based
– Study protocolling
– Integrating clinical data
– Report QA/Dictation Assistance
• Image-Based
– Prioritize cases and Diagnoses
– Segmentation
– Super-Resolution
– Prioritize cases and Diagnoses
- Automated lesion follow-up
– Image-based molecular pathology
imgkid.com

49. Our Future includes Algorithms to
Improve Image Quality

50. Summary
1. AI healthcare applications will follow the routine use in
our day-to-day lives
2. Deep learning will enhance our natural learning. It can’t
decrease it.
3. Please don’t confuse science fiction with medical care.
The reality for radiology is that computers have helped us
be better doctors at every step of the way. This is just one
of those “big steps”.