Model guided therapy and the role of dicom in surgery by Heinz U. Lemke

1. Model-Guided Therapy and the
role of DICOM in Surgery
Heinz U. Lemke, PhD
Chair of Working Group 24 “DICOM in Surgery“

2. Content
1. Introduction (problems and solutions)
2. Model guided therapy with TIMMS
3. Classification and model classes
4. Virtual human model examples
5. Conclusion

3. Computer Assisted Digital OR Suite for Endoscopic MISS
Problems: Multiple Data Sources
Digital endoscopic OR suite facilitates MISS
MD’s
Staff
RN, Tech
EMG
Monitoring
C-Arm
Fluoroscopy
MRI Image -
PACS
C-Arm Images
Image Manager -
Report
Video Endoscopy
Monitor
EEG Monitoring
Left side of OR
Image view
boxes
Teleconferencing
- telesurgery
Laser
generator
Courtesy of Dr. John Chiu

4. Model Guided Therapy and the
Patient Specific Model
• Model Guided Therapy (MGT) is a methodology
complementing Image Guided Therapy (IGT) with
additional vital patient-specific data.
• It brings patient treatment closer to achieving a
more precise diagnosis, a more accurate
assessment of prognosis, as well as a more
individualized planning, execution and validation
of a specific therapy.
• By definition, Model Guided Therapy is based on
a Patient Specific Model (PSM) and allows for a
patient specific intervention via an adapted
therapeutic workflow.

5. Model Guided Therapy and data structures
• Model Guided Therapy based on patient specific
modelling requires appropriate IT architectures
and data structures for its realisation.
• For PSMs, archetypes and templates allow
different levels of generalisation and
specialisation, respectively.

6. Biosensors
(physiology,
metabolism,
serum, tissue, …)
Omics EMR
Modalities
(X-ray,CT, US,
MR,SPECT,
PET,OI)
Model Based Patient Care
EBM
Workflow
IHE
Model Creation
and Diagnosis
(Data fusion,
CAD, …)
Model Maintenance
and Intervention
(Simulation,
decision support,
validation, …)
Data bases
(Atlas,
P2P repositories,
data grids, ...)
Mechatronics
(Navigation,
ablation, …)
IT Communication Infrastructure

7. Content
1. Introduction (problems and solutions)
2. Model guided therapy with TIMMS
3. Classification and model classes
4. Virtual human model examples
5. PM data structures (SDTM and OpenEHR)
6. Conclusion

8. IT Model-Centric World View
Interventional Cockpit/SAS modules
Modelling
Models
(Simulated
Objects)
Therapy Imaging and Model Management System (TIMMS)
ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy
Simulation
Kernel for
WF and K+D
Management
Visualisation
Rep. Manager
Intervention Validation
Repo-
sitory
Engine
Data Exch.
Control
IO Imaging
and
Biosensors
Images
and
signals
Modelling
tools
Computing
tools
WF and
K+D
tools
Rep.
tools
Devices/
Mechatr.
tools
Validation
tools
WF`s, EBM,
”cases”
Data and
information
Models and
intervention
records
Therapy Imaging and Model Management System (TIMMS)

9. Model Guided Therapy with TIMMS
• For a therapeutic intervention it is assumed that
human, mechatronic, radiation or pharmaceutical
agents interact with the model.
• MGT provides the scientific basis for an accurate,
transparent and reproducible intervention with the
potential for validation and other services.
• TIMMS is an IT meta architecture allowing for
interoperability of the agents to facilitate a MGT
intervention.

10. Model Guided Therapy
The basic TIMMS patient model must have the following features:
1. The TIMMS patient model must have components which
represent the patient as an n-dimensional and multiscale
(in space and time) data set.
2. The TIMMS patient model must facilitate interfacing to the
surgeon and other operative personnel, the TIMMS engines,
TIMMS repositories, and the IT infrastructure.
3. The TIMMS patient model must be capable of linking these
components, which may be static or dynamic, in a meaningful
and accurate way.
4. For dynamic components, the TIMMS patient model must be
able to process morphological and physiological data and
perform the necessary mathematical functions to maintain the
model in an up-to-date state.

11. Model Guided Therapy
5. The TIMMS patient model must be capable of being incorporated
by the TIMMS executing workflow and responding to its changes.
6. The TIMMS patient model must be amenable to be developed
using readily available, standardized informatics methodology.
Tools may include UML, XML, Visio, block diagrams, workflow
diagrams, MATLAB, Simulink, DICOM (including surgical DICOM),
Physiome, CDISC SDTM, openEHR and similar products and tools.
7. The TIMMS patient model must comply to software engineering
criteria, for example, to open standards and service-oriented
architectures to allow for multi-disciplinary information exchange.
8. The TIMMS patient model must allow for further extensions to
incorporate advances in molecular medical imaging, genomics,
proteomics and epigenetics.
9. The TIMMS patient model must be amenable to be used for clinical
trials, predictive modeling, personal health records and in the long
term contribute to a Model Based Medical Evidence (EBME)
methodology.

12. IT Model-Centric World View
Interventional Cockpit/SAS modules
Modelling
Models
(Simulated
Objects)
Therapy Imaging and Model Management System (TIMMS)
ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy
Simulation
Kernel for
WF and K+D
Management
Visualisation
Rep. Manager
Intervention Validation
Repo-
sitory
Engine
Data Exch.
Control
IO Imaging
and
Biosensors
Images
and
signals
Modelling
tools
Computing
tools
WF and
K+D
tools
Rep.
tools
Devices/
Mechatr.
tools
Validation
tools
WF`s, EBM,
”cases”
Data and
information
Models and
intervention
records
Therapy Imaging and Model Management System (TIMMS)

13. Generic and patient specific
n-D modelling tools
• Geometric modelling
• Prosthesis modelling
• Properties of cells and tissue
• Segmentation and reconstruction
• Biomechanics and damage
• Tissue growth
• Tissue shift
• Properties of biomaterials
• ...
Modelling
tools

14. Model Guided Therapy
• MGT in its simpliest instantiation is an intervention with
a subset, a single or a set of voxels representing
locations within the patient body. With this view, it is an
extension from Image (pixel) Guided Therapy (IGT) to
model (voxel) guided therapy. Examples of model
guided therapy are:
a) interventions within a subset of a voxel, e.g. cells,
organelles, molecules, etc.
b) interventions with a voxel, e.g. small tissue parts of
an organ or lesion, etc.
c) interventions with a set of voxels, e.g. part of
functional structures of organs, organ components,
soft tissue, lesions, etc.

15. Model Guided Therapy
1. 1-D signals (e.g. EEG)
2. 2-D projection and tomographic images
3. 3-D reconstructions
4. Temporal change
5. Tissue/cell type
6. Ownership to organ, lesion, system, prothesis, chronic
condition, etc.
7. Spatial occupancy/extension
8. Permeability (blood brain barrier)
9. Flow (e.g. electric, heat, liquid, perfusion, diffusion, etc.)
In a simple PSM, voxels may be associated
with several dimensions of data

16. Model Guided Therapy
10. Level of oxygenation (e.g. level of hypoxia)
11. Pharmacokinetics (e.g. effect of tissue on
pharmaceutical agent, flow parameters, time to peak,
etc.)
12. Pharmacodynamics (effect of pharmaceutical agent on
tissue, ablation parameters)
13. Biological marker types (in vitro and/or in vivo
molecular spectrum)
14. Reference coordinate system (e.g.
Schaltenbrand/Warren, Talaraich/Tourneaux)
15. Value (life critical to life threatening)
16. Neighbourhood (e.g. 3³, 5³, 7³, etc.)
17. ...
In a simple PSM, voxels may be associated
with several dimensions of data

17. Example: ENT model elements
Source: G. Strauss

18. Example: ENT model elements
Source: G. Strauss

19. Content
1. Introduction (problems and solutions)
2. Model guided therapy with TIMMS
3. Classification and model classes
4. Virtual human model examples
5. Conclusion

20. Strategies for multiscale modelling
• Modelling is essential for understanding the
knowledge of human characteristics such as, anatomy,
physiology, metabolism, genomics, proteomics,
pharmacokinetics, etc.
• Because of the complexity of integrating the
knowledge about the different characteristics the
model of a human has to be realised on different
levels (multiscale in space and time) and with different
ontologies, depending on the questions posed and
answered delivered.
• The problems associated with using reduced-form
components within large systems models stem
primarily from their limited range of validity.

21. Source: J. Bassingthwaighte

22. Patient specific and associated
modelling functions
In the Model-Centric World View a wide variety of
information, relating to the patient, can be integrated
with the images and their derivatives, providing a more
comprehensive and robust view of the patient.
By default, the broader the spectrum of different types of
interventional/surgical workflows which have to be
considered, the more effort has to be given for designing
appropriate multiscale PSM’s and associated services.

23. Patient specific and associated
modelling functions
Management of n-D and multi resolutional
knowledge (model of the biologic continuum in
space and time) is still a research and
development challenge.
If solved successfully, it will transform surgery
into a more scientifically based activity.

24. Content
1. Introduction (problems and solutions)
2. Model guided therapy with TIMMS
3. Classification and model classes
4. Virtual human model examples
5. Conclusion

25. Patient Specific CMB
Visible Human
Anatomical Template
organ surface meshes
Multimodal Imaging
(MRI, CT, Angio,..DT-MRI)
PKPD
Spitzer 2006 Virtual Anatomy
FEM Mesh (Roberts JHU)
Human Laser
Scan (CAESAR DB)
Roberts JHU

26. Content
1. Introduction (problems and solutions)
2. Model guided therapy with TIMMS
3. Classification and model classes
4. Virtual human model examples
5. Conclusion

27. Solutions and Research Focus
(medical)
• Transition from image guided to model guided
therapy (e.g. through workflow and use case
selection/creation/repositories)
• Concepts and specification of patient specific
models in a multiscale domain of discourse
• Concepts and design of a canonical set of low
level surgical functions
• Prototyping

28. IT Model-Centric World View
Interventional Cockpit/SAS modules
Modelling
Models
(Simulated
Objects)
Therapy Imaging and Model Management System (TIMMS)
ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy
Simulation
Kernel for
WF and K+D
Management
Visualisation
Rep. Manager
Intervention Validation
Repo-
sitory
Engine
Data Exch.
Control
IO Imaging
and
Biosensors
Images
and
signals
Modelling
tools
Computing
tools
WF and
K+D
tools
Rep.
tools
Devices/
Mechatr.
tools
Validation
tools
WF`s, EBM,
”cases”
Data and
information
Models and
intervention
records
Therapy Imaging and Model Management System (TIMMS)
Prototyping

29. Solutions and Research Focus
(technical)
• Concepts and data structure design of patient specific
models (e.g. with archetypes and templates)
• Model management with open architectures (e.g. SOA)
• SOA modulariation with repositories, engines, LLM´s and
HLM´s
• LLM´s as adaptive (cognitive/intelligent) agents
• HLM´s as application modules (competitive differentiation)
• LLM´s possibly as open source
• Kernel (engine and repository) for adaptive workflow and
K+D management
• Cooperative and competitive R+D framework for engine
and repository building
• Therapy based open standard ( e.g. S-DICOM)
• Transition from CAD to CAT modelling

30. IT Model-Centric World View
Interventional Cockpit/SAS modules
Modelling
Models
(Simulated
Objects)
Therapy Imaging and Model Management System (TIMMS)
ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy
Simulation
Kernel for
WF and K+D
Management
Visualisation
Rep. Manager
Intervention Validation
Repo-
sitory
Engine
Data Exch.
Control
IO Imaging
and
Biosensors
Images
and
signals
Modelling
tools
Computing
tools
WF and
K+D
tools
Rep.
tools
Devices/
Mechatr.
tools
Validation
tools
WF`s, EBM,
”cases”
Data and
information
Models and
intervention
records
Therapy Imaging and Model Management System (TIMMS)
Archetypes and Templates

31. Solutions and Research Focus
(medical and technical)
• Transition from image guided to model guided therapy (e.g.
through workflow and use case
selection/creation/repositories)
• Use cases for adaptive workflow, exception handling and
K+D management for selected interventions
• Cooperative and competitive R+D framework for low
(open source) and high level (competitive differentiation)
surgical function computerisation
• Information/model flow from diagnosis (e.g. CAD) to CAT
(i.e. interdisciplinary cooperation)
• Development of standards for patient modelling in
WG24 “DICOM in Surgery”

32. IT Model-Centric World View
Interventional Cockpit/SAS modules
Modelling
Models
(Simulated
Objects)
Therapy Imaging and Model Management System (TIMMS)
ICT infrastructure (based on DICOM-X) for data, image, model and tool communication for patient model-guided therapy
Simulation
Kernel for
WF and K+D
Management
Visualisation
Rep. Manager
Intervention Validation
Repo-
sitory
Engine
Data Exch.
Control
IO Imaging
and
Biosensors
Images
and
signals
Modelling
tools
Computing
tools
WF and
K+D
tools
Rep.
tools
Devices/
Mechatr.
tools
Validation
tools
WF`s, EBM,
”cases”
Data and
information
Models and
intervention
records
Candidate components for open source
Open Source

33. WG 24 “DICOM in Surgery“
Project Groups
• PG1 WF/MI Neurosurgery
• PG2 WF/MI ENT and CMF Surgery
• PG3 WF/MI Orthopaedic Surgery
• PG4 WF/MI Cardiovascular Surgery
• PG5 WF/MI Thoraco-abdominal Surgery
• PG6 WF/MI Interventional Radiology
• PG7 WF/MI Anaesthesia
• PG8 S-PACS Functions
• PG9 WFMS Tools
• PG10 Image Processing and Display
• PG11 Ultrasound in Surgery

34. Definition of Surgical Workflows (S-WFs)
• Micro Laryngeal Surgery (MLS) (PG2
ENT/CMF)
• Foreign Body Excision (PG2 ENT/CMF)
• Total Hip Replacement Surgery (PG3
Orthopaedic)
• Total Endoscopic Coronary Artery Bypass (TECAB) (PG4
Cardiovascular)
• Mitral Valve Reconstruction (MVR) (PG4
Cardiovascular)
• Laparoscopic Splenectomy (PG5
Thoraco-abdominal)
• Laparoscopic Cholecystectomy (PG5
Thoraco-abdominal)
• Laparoscopic Nephrectomy left (PG5
Thoraco-abdominal)
• Angiography with PTA and Stent (PG6
Interventional Radiology)
• Hepatic Tumor Radio Frequency Ablation (PG6
Interventional Radiology)
• Trajugular Intrahepatic Portosystemic Shunt (PG6
Interventional Radiology)

35. CARS / SPIE / EuroPACS
9th Joint Workshop on
Surgical PACS and the Digital Operating Room
Barcelona, 28 June, 2008
12th Meeting of the
DICOM Working Group WG 24 “DICOM in Surgery“
Barcelona, 28 June 2008
CARS 2008 Computer Assisted Radiology and Surgery
http://www.cars-int.org

37. WG24 “DICOM in Surgery”
Secretariat: Howard Clark, NEMA
Secretary: Franziska Schweikert, CARS/CURAC Office
fschweikert@cars-int.org
General Chair: Heinz U. Lemke, ISCAS/CURAC, Germany
Co-Chair: Ferenc Jolesz, Harvard Medical School, Boston
(Surgery/Radiology)
Co-Chair: tbd
(Industry)