We present an approach using medical expert knowledge represented in formal ontologies to check the results of automatic medical object recognition algorithms for spatial plausibility. Our system is based on the comprehensive Foundation Model of Anatomy ontology which we extend with spatial relations between a number of anatomical entities. These relations are learned inductively from an annotated corpus of 3D volume data sets. The induction process is split into two parts. First, we generate a quantitative anatomical atlas using fuzzy sets to represent inherent imprecision. From this atlas we then abstract the information further onto a purely symbolic level to generate a generic qualitative model of the spatial relations in human anatomy. In our evaluation we describe how this model can be used to check the results of a state-of-the-art medical object recognition system for 3D CT volume data sets for spatial plausibility. Our results show that the combination of medical domain knowledge in formal ontologies and sub symbolic object recognition yields improved overall recognition precision.
What Are The Drone Anti-jamming Systems Technology?
Automatic Spatial Plausibility Checks for Medical Object Recognition Results Using a Spatio-Anatomical Ontology
1. 1
1/30/2015
Automatic Spatial Plausibility Checks for
Medical Object Recognition Results
Using a Spatial-Anatomical Ontology
Manuel Möller, Patrick Ernst,
Andreas Dengel, Daniel Sonntag
German Research Center for Artificial Intelligence
University of Kaiserslautern
2. 2
1/30/2015
With the shift to the application of digital imaging
techniques for medical diagnosis, such as CT, MRI,
etc., the volume of digital images produced in modern
clinics increased tremendously.
Our clinical partner,the University Hospital Erlangen
in Germany, has a total of about 50 TB of medical
images. Currently, they have about 150,000 medical
examinations producing 13 TB of data per year.
5. 5
1/30/2015
Our approach is to augment
medical domain ontologies
and allow for an automatic
detection of anatomically
implausible constellations in the
results of a state-of-the-art
system for automatic object
recognition in 3D CT scans.
The output of our system
also provides feedback which
anatomical entities are
most likely to have been located
incorrectly.
7. 7
1/30/2015
Motivation
• Automatic object
recognition algorithms
available for several
organs in 3D
• Perform reasonably well
(in many cases)
• Integration with
anatomical background
knowledge: Foundational
Model of Anatomy
Daniel Sonntag, Daniel.Sonntag@dfki.de
8. 8
1/30/2015
Proposed Solution:
Integration of automatic object
recognition algorithms with
high-level knowledge about
human anatomy.
Motivation
In some cases the automatic
object recognition is grossly
wrong.
Daniel Sonntag, Daniel.Sonntag@dfki.de
9. 9
1/30/2015
Outline
1. Goals and Prerequisites
2. Hierarchical Algorithm for Learning Spatial
Relations
3. Application to Automatic Object Recognition
4. Conclusion
Daniel Sonntag, Daniel.Sonntag@dfki.de
10. 10
1/30/2015
Goals and Prerequisites
• Goals:
– bridge semantic gap between low-level and high-level
information
– develop system integrating information from both
sources
– perform reasoning to check plausibility of object
recognition results
• Prerequisites:
Automatic object recognition algorithms
Structural anatomical knowledge
Spatial relations of human anatomy
Integration of low-level and high-level information
Daniel Sonntag, Daniel.Sonntag@dfki.de
14. 14
1/30/2015
• Statistical algorithms for the detection of
various anatomical entities
– Constrained MSL
– Hierarchical Active Shape Models
– Patch-based Deformable Models
– Trainable Boundary Detector
see: Seifert, Kelm, Möller, Mukherjee, Cavallaro,
Huber, Comaniciu: “Semantic Annotation of Medical
Images”, SPIE Medical Imaging 2010
Results:
• Meshes: 6 different organs
left/right kidney, left/right lung, urinary bladder, prostate gland
• Landmarks: 22 exposed points: top point of the liver, …
• Manually generated gold standard of automatically
annotated volume data sets: 1 017 labeled volumes
Pre-Processing
Automatic
Annotation
Fuzzy Atlas
Qualitative
Representation
Spatial
Reasoning
Corpus
Daniel Sonntag, Daniel.Sonntag@dfki.de
15. 15
1/30/2015
• Classical Logic:
leftFrom(left kidney, right kidney) {0,1}
• Fuzzy Logic:
leftFrom(left kidney, right kidney) [0,1]
• Representation of direction in 2D:
0
½π
+/-π
-½π
R
T
right
above
left
below
α
Pre-Processing
Automatic
Annotation
Fuzzy Atlas
Qualitative
Representation
Spatial
Reasoning
Corpus
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
unten rechts oben
cos^2(x)
-½π ½π0
Truthvalue
angle
below aboveright
Daniel Sonntag, Daniel.Sonntag@dfki.de
16. 16
1/30/2015
Patient 5
Patient 4
Patient 3
Patient 2
Patient 1
Natural variability in human anatomy
R Z
α=3°
„right kidney right from left kidney“:
Truth value
Absolutefrequency
10
R Z
α=0°
R
Zα=4°
R
Zα=4°
R Z
α=0°
avg= 0,92…
Pre-Processing
Automatic
Annotation
Fuzzy Atlas
Qualitative
Representation
Spatial
Reasoning
Corpus
Daniel Sonntag, Daniel.Sonntag@dfki.de
17. 17
1/30/2015
A B
(a)
A B
(b) (c)
Right
Left
C
B
A
Relation Types
Pre-Processing
Automatic
Annotation
Fuzzy Atlas
Qualitative
Representation
Spatial
Reasoning
Corpus
• Direction: „left kidney left from right
kidney“
• Adjacency: „prostate adjacent to urinary
bladder“
• Between: „bronchial bifurcation between
left and right lung“
• Evaluated with medical experts
Daniel Sonntag, Daniel.Sonntag@dfki.de
18. 18
1/30/2015
Spatial Relations in OWL Model
• Extension of the formalism in the
Foundational Model of Anatomy
[0..1]
left|right|
above|…
term
truthValue
Instance of SimpleFuzzyRelation
Anatomical Entity B
Anatomical Entity A
location
related
Object
FuzzySpatialAssoc
iationRelation
type
[0..1]
left|right|
above|…
directionalTerm
truthValue
Instance of SimpleFuzzyRelation
[0..1]
left|right|
above|…
directionalTerm
truthValue
Instance of SimpleFuzzyRelation
[0..1]
left|right|
above|…
term
truthValue
Instance of SimpleFuzzyRelation
Pre-Processing
Automatic
Annotation
Fuzzy Atlas
Qualitative
Representation
Spatial
Reasoning
Corpus
Daniel Sonntag, Daniel.Sonntag@dfki.de
19. 19
1/30/2015
• Example:
Learning spatial
relations
• Comparison
between learned
model and new
object recognition
result
• Results:
true positives 407
true negatives 431
false positives 67
false negatives 213
precision 85,7%
recall 65,5%
Patient 1Patient 2
Patient 3
Patient 4 Pre-Processing
Automatic
Annotation
Fuzzy Atlas
Qualitative
Representation
Spatial
Reasoning
Corpus
Incorrectly
located
organ
Spatial Consistency Check
Daniel Sonntag, Daniel.Sonntag@dfki.de
20. 20
1/30/2015
Medico Server
•MEDICO Ontology
•Sesame Triplestore
•>2 Mio. Triples
•Semantic
Annotation Store
•3D Volume Renderer
•Based on MITK
State of the Art Organ
and Landmark Detection
Ontology-based Visual
Navigation Application
Central Java-based Data
Exchange Application
MEDICOServer
Volume
Parser
MITK
Semantic
Navigation
Semantic
Search
XMLRPC
SPARQL
CORBA
CORBA
CORBA
Java API
CTC-WP4
Triple
Store
Query
Broker
RadSpeech
22. 22
1/30/2015
Conclusion
• Hierarchical abstraction process to learn spatial relations
from annotated volume data sets
• Method for the generation of a fuzzy anatomical atlas
from different patients
• Spatial consistency check comparing automatic object
recognition results with
Daniel Sonntag, Daniel.Sonntag@dfki.de
Pre-Processing
Automatic
Annotation
Fuzzy Atlas
Qualitative
Representation
Spatial
Reasoning
Corpus
Patient 5
Patient 4
Patient 3
Patient 2
Patient 1
R Z
α=3°
R Z
α=0°
R
Zα=4°
R
Zα=4°
R Z
α=0°
„right kidney right from left kidney“:
Truth value
Absolutefrequency
10