Multimodality & 4 D Imaging

Multimodality & 4D Imaging:
Registration and Fusion for
Treatment Planning and Delivery
ASTRO 2007 - 49th Annual Meeting
ASTRO 2007 - 49th Annual Meeting
Wednesday, October 31, 2007
Wednesday, October 31, 2007
1:30 – 2:45 PM
1:30 – 2:45 PM

Marc L Kessler, PhD
The University of Michigan

Laura A Dawson, MD
Princess Margaret Hospital

Disclosures

• Research Grant- Varian Medical Systems

Objectives
Understand the basic mechanics of
multimodality and 4D image registration
techniques

Understand the different techniques used
to combine, display and interact with
multimodality and 4D image and dose data

Understand the clinical use and limitations
of these techniques for Tx planning, Tx
delivery and plan adaptation

Marc L Kessler, PhD - ASTRO 2007 Refresher Course - Please do not redistribute

Outline

Motivation

Mechanics !

Clinical Use


Motivation
Precision radiation therapy requires
accurate delineation of the tumor
and normal tissues in the planning
phase and accurate localization of
these structures during the delivery
phase

…with the aid of imaging

Motivation
entire
Optimization of the radiotherapy process
requires that we anticipate, measure &
adapt to changes in the patient

Imaging Planning Delivery
on-line

Imaging
off-line


Gregoire / St-Luc
Gregoire / St-Luc

Multimodality Targeting

?

X-ray CT MRI Nuc Med

We now have many cameras available
… which provide complementary data!

Repeat Imaging

?

?

Normal Tissues Target Volumes

Balter / UM
Balter / UM

4-D Imaging

… assess motion


Dawson / PMH
Dawson / PMH

Image Guided Treatment

Varian Siemens ViewRay
OBI™ PRIMATOM™ Renaissance™

Elekta TomoTherapy Resonant
Synergy™ Hi-Art™ Restitu™


The Big Picture
Tx Plan Portal
CT 3D Dose Images

MR CBCT
“Adapting” 1…n

Patient
NM Model US

3D Dose 4D
US Day n CBCT

The Goal
Ideally, we would like to have a time
dependent vector of information for
every “point” in an anatomic object

image information (MR, CT, NM, … )
physiologic information (τ )
anatomic label information
dose information … with time stamp !


The Goal
PET

MR

CT

CT+ MR + NM + Dose (τ)
Scalar Data 4-D Vectors

Mechanics
… determine the geometric transformation
that maps corresponding points from one
image series to another

Form of the transformation T
… from rigid to fully freeform
Number of degrees of freedoms β
… from 3 to 3 x N *
*N
* = number of voxels
Marc L Kessler, PhD - ASTRO 2007 Refresher Course
Marc L Kessler, PhD - ASTRO 2007 Refresher Course

Transformation
… determine the geometric transformation
that maps corresponding points from one
image series to another

XB = T ( XA , { ß })
(x,y,z) coordinates
of a point in Series B (x,y,z) coordinates
of a point in Series A


Degrees of Freedom
PET/CT MR - CT 4D CT

None ? Few Many

What is T ?
Rigid / Affine
Global, regional, or piecewise

Full 3D / 4D Deformation
Parametric models
Free-form models

What is T ?
Rigid / Affine
Global, regional, or piecewise

xB = A xA + b (up to 12 DOF)

y = m x+ b … in 3D


www.gnome.org
www.gnome.org

Affine Transformations

Study A
A Square
A Square

Study B
Translation
Translation Rotation
Rotation Scaling
Scaling Shearing
Shearing

3 3 3 3
Parallel lines stay parallel !

www.gnome.org
www.gnome.org


Study A

6 DOF
A Square
A Square

Study B
Translation
Rotation Scaling
Scaling Shearing
Shearing


www.gnome.org
www.gnome.org


Study A
A Square
A Square

Study B
Translation
Rotation Scaling
Scaling Shearing
Shearing

3 or 4 DOF

non- Affine Transformations
Study A Study B

Parallel lines don’t stay parallel!

Study A Study B

XB = T ( XA , { ß })

Study A Study B

XB = T ( XA , { ß(XA) })

Study A Study B

Transformation parameters
to apply to a particular point
depends on the location of
the point !

XB = T ( XA , { ß(XA) })

Balter / UM
Balter / UM


phase dependent ?

XB = T ( XA , { ß(XA, φ )})


… up to 3 x N

Parametric Freeform

Parametric
Various splines ( TPS , B-splines )
Other basis functions

Freeform
Finite element models
Flow models ( optical, viscous )


Each have some distinct properties
B-Splines … local
Thin-Plate splines … global

Finite element … bio-mechanical

Intensity flow … image forces
( mono-modality )


Warp Space / Warp Objects /
… Drag Objects … Drag Space

Brock / PMH

Parametric Freeform

How Do We Compute { β } ?

1 Construct a metric that measures the
mismatch (or similarity) between a
pair of datasets

2 Apply an optimization algorithm to
determine the parameters (DOF) that
minimize (maximize) this metric


How Do We Compute { β } ?

{β}

Registration Metrics

?

?

Geometry-based Intensity-based

Geometry-Based Metrics

Point Matching
Least Squares
Σ ( XB - X A ) 2
B A

Surface Matching
Chamfer Matching
Σ min distance 2

… depends on image segmentation!

Intensity-Based Metrics

Mono-modality
Sum Squared Difference
Σ ( IB - IA ) 2
B A

Multimodality Data
Σ p(IA, IB) log
A B
p(IA, IB)
A B
p(IA) p(IB)
Mutual Information A B

… depends on the image characteristics!

How About An Example?

Transformation PET
Rotate - Translate

Registration Metric
CT Mutual Information
Optimizer
Simplex Algorithm


How About An Example?

PET

CT


Balter / UM
Balter / UM

How About Deformations ?

Multiphasic CT Data

How About Deformations ?
Transformation
B-Splines ( multi-resolution )

Registration Metric
Sum Squared Difference

Optimizer
Exhale State decent
Gradient Inhale State


Multiresolution Deformations
Successively increase the resolution of
the knot spacing

Only small additional computation cost
when increasing the number of knots.

the image data

¼ Resolution
¼ Resolution Full Resolution
Full Resolution

Coarse Fine

the image data

60 x 60 x 48 mm
60 x 60 x 48 mm 4 x 4 x 3 mm
4 x 4 x 3 mm

Coarse Fine

Registration Metric vs. Iteration
2.5
2.5

2.4
Change in
2.4
Registration Metric

knot spacing
2.3
2.3 Low Res
2.2
2.2

2.1
2.1
High Res
2.0
2.0

1.9
1.9

1.8
1.8
0
0 20
20 40
40 60
60 80
80 100
100 120
120 140
140 160
160 180
180
Iteration Number

Multiresolution B-Splines
Multiphasic CT Data

Exhale State Inhale State
deformed

Ruan / UM
Ruan / UM

We Are Not Really Splines !

No “stiffness”
information

Extracted Exhale
Ribcage Deform Inhale

Add Some Physics?

Etotal = Esimilarity + α Estiffness

intensity similarity measure

tissue-dependent regularization

Evol =
∫ wc(x) |det JT(x) – 1|2 dx

Spatially Variant Stiffness


“Stiffness” Weighting

wc(x)


Ruan / UM
Ruan / UM

Using “Prior” Information

using “stiffness”
information

Extracted Exhale
Ribcage Deform Inhale

Balter / UM
Balter / UM

Tissue Sliding

Deal with different organs individually?

Segmentation + Registration
No masking Masking

Ribs driven by large Ribs not affected
lung deformations by lung registration

Brock / UM
Brock / UM

Finite Element Modeling
Exhale
Exhale

Inhale
Inhale

Take into account physical
tissue properties (directly)


Brock / PMH
Brock / PMH

Finite Element Modeling

… thorough segmentation is necessary

The Future ?

Family of
Generalized,
Customizable,
Patient Models


Is The Future Here Already?

Presenter has no commercial interest in this company

www.mimvista.com
www.mimvista.com

…from Atlas to Individual


Thompson / UCLA
Thompson / UCLA

…from Individuals to Atlas

Brain Mapping: The Disorders,, Academic Press, 1999
Brain Mapping: The Disorders Academic Press, 1999

Meyer/UM
Meyer/UM

without with
with

Segment /register / average Segment using atlas

In The Meantime …

Image Anatomy Dose

Anatomy Mapping

… map to CT
Boolean OR

Use superior MR contrast for targeting

Dong / MDACC
Dong / MDACC

Anatomy Mapping
Drawn Contours Simple Overlay
(no transform)

Planning CT “Delivery” CT

Dong / MDACC
Dong / MDACC

Anatomy Mapping
Drawn Contours Transformed and
resampled

Segmentation done w/ the aid“Delivery” CT
Planning CT of a registration!

More Than Deformations

deformation
weight loss
resection
shrinkage
… not just Δ vascular
deformation!


Dose Mapping
Dealing with volume elements that may:

change shape / appear / disappear
… need proper spatial re-sampling

don’t necessarily add in a linear fashion
… need some sort of radiobiology
exist in homogenous intensity regions
… hard to evaluate registration


Validation
How do we know how well these
registration methods perform?

build phantoms and test them
we can know the truth!

provide tools to examine results
we don’t know the truth!

1986
1986

Validation Phantoms
CT

MR

Kashani / UM
Kashani / UM

Validation Phantoms


Validation Tools
Qualitative Tools

Color gel or wash
overlay

Split /dual screen
displays

Anatomic boundary
overlay!

Validation Tools
Quantitative Tools
Study A
Point Description Exhale Inhale

*
X Y Z X Y Z
1 2nd branch of bronchial tree -5.37 0.98 -3.42 -4.62 -0.22 -2.92
2 3rd branch of bronchial tree -5.73 2.12 -5.42 -5.40 0.74 -5.92
3 4th branch of bronchial tree -6.50 2.77 (x , y , z )
-8.42
AA
-6.24
A -8.12
A A
A
0.80 -9.42
4 Vessel bifurcation 1 -8.12 3.37 -9.92 1.40 -11.42
5 Vessel bifurcation 2 -8.06 -1.95 -4.42 -7.67 -3.20 -3.92
6 Vessel bifurcation 3 -10.69 2.47 0.58 -10.78 1.16 1.08
Study B
all values in cm. Exhale' - Inhale

*
Exhale' ( w/ TPS alignment ) ΔX ΔY ΔZ
-4.71 -0.47 -3.36 -0.09 -0.25 -0.44
-5.35 0.58 -5.83 0.05 -0.16 0.09
-6.27 0.69 (x , y , z )
B
-9.51
B B -0.03
B BB
-0.11 -0.09
-8.19 0.91 -11.60 -0.07 -0.49 -0.18
-7.27 -2.83 -3.63 0.40 0.37 0.29
-10.85 0.87 1.24 -0.07 -0.29 0.16
σ 0.19 0.29 0.26

AAPM Task Group 132
Use of Image Registration and Data
Fusion Algorithms and Techniques in
Radiotherapy

Methods to assess the accuracy of
image registration and fusion

Issues related to acceptance testing
and quality assurance for image
registration and fusion


Opportunities & Challenges
T2
2
Flair

T1
1
Gd Diff


More than just mechanics!
What Now ?

MR volumes mapped to CT study

Summary
Taxonomy of Registration Process

Geometry Intensity

Interactive Automated

Affine non-Affine


Summary
Tools are now available to register and
integrate image, anatomy & dose for
both Tx planning and Tx delivery

These tools can be used to help build
better models of the patient and to
help customize and adapt therapy

Work towards more standard and
robust tools and validations methods
(for non-rigid) situations continues

ASTRO 2007 – 49th meeting; Wed Oct 31, 2007, 1:30 – 2:45 PM

Multimodality and 4D Imaging:
Registration and Fusion for
Treatment Planning and
Delivery: The clinical
perspective…

Laura Dawson, Toronto
Marc Kessler, Ann Arbor

Disclosures

• Research Grant, Elekta

Challenges in Radiation Therapy
1. Seeing the tumor
2. Defining the target
3. Hitting the target
4. Knowing when the tumor is dead

Imaging and image registration is
key for addressing these
challenges

Clinical Use of Image Registration
Spatial change Temporal change
• Planning • Planning
– Planning CT-MR – 4D CT
– Planning CT-Contrast CT
– Planning CT-PET • Delivery
– Planning CT-SPECT
– MV / kV fluoroscopy
• Delivery – 4D cone beam CT
– kV simulation film-MV EPI – Shape change during RT
– DRR - MV or kV PI
• Bones • Follow-up
• Fiducial markers – Diagnostic CT/ MR/ PET
– Planning CT - MV or kV volumetric to planning CT
imaging – Dose accumulation
– Patterns of recurrence
– NTCP, TCP

Other Future Uses of Image
Registration
Spatial change Temporal change
• Planning • Planning
– Facilitate contouring – Accumulating daily dose
(e.g. map atlases to pt (e.g. adaptive therapy)
dataset)

Planning: PETCT-MR
• 56 yo man with clinical T3N0 SCC of oropharynx
• PET-CT and MR obtained in treatment position,
on hard table top with mask, on study
• Rigid image registration in region of interest
including vertebral bodies

• Benefits:
• MR helped define primary tumor superiorly in
region of CT with dental artifact
• PET helped confirm suspicious node on CT as
high risk

CT

Courtesy of John Waldron and Stephen Breen, PMH

MR


PET
?
physiologic
uptake
GTV

?
tumor


PET-CT
physiologic
uptake:
• muscle
• tonsil
GTV
• vessel
muscle

The fused images are most useful
when all information can be
evaluated together

Good alignment in region of tumor
Alignment
in base of
skull not
perfect

Good
alignment
in region
of tumor

Planning: CTPET-MR
• Looking at CT-PET fusion far more helpful
than PET alone
• CT-PET registration not perfect in entire
field of view, due to residual rotations,
deformation
• Many normal variances of PET
Clinical interpretation of fused images important!

Planning: CT-MR Prostate
• Advantages of MR for prostate cancer RT
– Improve inter-observer variability
– Provide more anatomy for organ /nerve sparing
approaches etc.

• New opportunities with MRS, diffusion MR, …
– Better knowledge of gross disease
– ‘Functional imaging’
– Dose painting
– Monitoring of change during RT and adaptation

• MR can improve contouring in patients
with bilateral hip replacements

Charnley et al, British J Radiology, 2005

McLaughlin / UM
McLaughlin / UM

Excellent localization of ‘sensitive’ structures

Allows delineation not possible or difficult on CT alone

… potency sparing?

MR with endorectal coil
1. MRI, no endorectal coil
2. Planning CT
3. MRI, endorectal coil

Deformable registration to
planning CT

4. Regions of tumor burden,
functional data can then be
visualized in planning CT
space

Courtesy of Cynthia Menard and Kristy Brock, PMH

Planning: CT-MR Liver

Liver cancer: MRI can show different volumes,
more foci of tumor, especially for HCC
Tumors often easier to see
Necessary for GTV definition if CT contrast
allergy

Planning: CT-MR for liver cancer
Auto-fuse whole field of view

CT MR CT MR

Vertebral body match

CT MR CT MR

Liver match

CT MR

• Once CT and MR liver are registered, the GTV on
both can be compared
• Different phases of CT and MR can be
complimentary

CT-arterial CT-venous MR-venous

Voroney, et al, IJROBP, 2006

Liver deformation ?
MR and CT GTV comparison
Prior to Deformable Registration

coronal

sagittal

Before After
Liver Deformable Registration

GTV Volume
CT = 13.9 cc
MR = 6.7 cc
ΔVol = 7.2 cc
(52%)
FEM deformable IR, Kristy Brock, PMH

Planning: CT-MR liver GTV
comparison
• 26 patients with liver cancer investigated
• GTV defined on CT and MR
• CT Liver-MR Liver deformable registration
• Med % surface of GTVs differed > 5 mm = 26%
• Largest differences for HCC and
cholangiocarcinoma

Voroney, et al, IJROBP, 2006

Deformation
• Even though deformation is ‘scary’ and
challenging to validate, measure,describe,
we need to be aware that deformation and other
volumetric change exists.

• Volume change and deformation is another
source of error and there are strategies to deal
with it

• Deformable image registration tools not
available commercially.

Image Guided Radiotherapy (IGRT)

MV EPID kV Fluoroscopy + markers Ultrasound kV CT

MV cone
MV CT beam CT kV Cone-beam CT

and more… Dawson, Jaffray, JCO, 2007

Electronic Portal Imaging Devices
• Types of image registration
– In your head
– Manual
– Dot gradicule
– Template
– Automated
– Limited ROI

• Even if you didn’t know it, you have been doing
image registration for years

EPIDs - Fiducials
A B

• Manual point
matching x x
• Automated
matching x x
x x

C

Courtesy of Kristy Brock, Peter Chung, PMH

Volumetric imaging: kV cone beam
CT
Bone match, then prostate match using
prostate contour from cone beam CT

Head & Neck
kV Cone-beam
CT

IGRT: Head and neck cancer
• T4N2 NPC for combined modality therapy. GTV
‘hugging’ brainstem and chiasm
• Clivus and cavernous sinus chosen as ROI

IGRT: Head and neck cancer
• Spine curvature not reproducible
• Change in tumor
• Dosimetric consequence?

L Johnston, J Waldron, PMH

IGRT Head and Neck: MV Cone Beam
CT
Week 1 Week 3
Doses
25 Gy Change in
45 Gy shape
54 Gy
70 Gy
74 Gy Increased
Dose Difference (%) cord dose
>5%
>10%

Courtesy of J Pouliot, UCSF

Lung Cancer IGRT (SBRT 20Gyx3)
Cone beam CT #1
Reconstructed CBCT
Dataset sent to
Pinnacle

CBCT Dataset
Registered w/ GTV
from Planning
GTV
PTV

Determine CouchFx #1
CBCT Shift

Verification by MV
Portal Imaging and/or
kV Fluoroscopy and/or
kV Fluoroscopy
CBCT

Courtesy of T Purdie, PMH

Lung Cancer IGRT
Non peripheral
lung tumors not
always well
visualized

Surrogates for
tumor can improve
setup accuracy

e.g carina for
central tumors

Courtesy of J Higgins,PMH

Lung Cancer based on skin
Setup IGRT

Courtesy of J Higgins, PMH

Lung Cancer IGRT
Carina Match

Courtesy of J Higgins, PMH

Lung Cancer IGRT
Bone match

Lung Cancer IGRT
Carina Match

Prior to IGRT

Pre-correction setup Courtesy of G Hugo, WBH, Michigan
(online & template)

Tumor IGRT residuals (online & of G Hugo, WBH, Michigan
Courtesy template)

A note of caution

• When region of interest for image matchig
is small, watch what happens to critical
normal tissues outside of matching
volume!

4D image matching

Respiratory
Correlated CT
(4DCT) for
Planning

Respiration
Correlated CBCT
on Treatment Unit

Liver Cancer IGRT - PMH
• MV, kV orthogonal imaging
– Diaphragm, exhale – CC positioning
– Vertebral body – ML and AP positioning
• kV CBCT - liver and/or liver tumor for 3D guidance
• Real time MV BEV images

MV imaging kV fluoroscopy kV CBCT

MV Orthogonal Image
Alignment
DRR (exhale) MV Portal Image (exhale)

Diaphragm
+ + used for CC
AP alignment

Vertebral bodies
used for AP,ML
alignment

Lat + +

MV Real Time Imaging
• 47 MV BEV movies from treatment fields
including air-diaphragm interface
– Manual check
– Automated comparison of MV exit field to
planned field

BEV DRR BEV PI

Dawson, IJROBP, 2005

kV Orthogonal Image Alignment
DRR (exhale) kV image (exhale)

Diaphragm
used for CC
AP alignment

Vertebral
bodies used
for AP,ML
alignment
Lat

Repositioning for offsets > 3 mm

Liver-liver registration for planning and
IGRT
• The liver can be used as a surrogate for the
GTV, as it is most often not visible on cone
beam CT

Planning CT MR kV cone beam CT

Volumetric Image Guidance
automated liver to liver
registration

Intra-fraction ChangePost-RT
Pre-RT in position

Difference Display

Free
breathing
cone beam
CT

Natural fiducial markers
Planning CT: Metastases with calcifications

kV Cone beam CT

Iatrogenic fiducial markers
• TACE: Trans hepatic arterial chemo-embolization
Lipiodol contrast stable in tumor for > 1 year

Liver position following MV
guidance
• Ave. residual deformation: 95% volume
deforming < 2.3 mm in each direction
• 4 cases : deformation > 5mm in CC and ML

Hawkins, Dawson et al, IJROBP 2006

Challenges with Registration in
IGRT
• Rotations
• Deformations
• New artifacts
• Free breathing changes
• Need for clinical input as to what region of interest
matters most for registration
• Do not forget rest of tissues, as they may move
MORE as smaller volume is used for registration

What do differences in images
mean?

Daisne et al. 233 (1): 93. (2004)

Daisne / St -Luc
Daisne / St -Luc

Ceci est une tumeur?
Macroscopy
CT Scan
FDG-PET

Jean-François Daisne MD, et al.. Radiology 2004;233:93-100
Jean-François Daisne MD, et al Radiology 2004;233:93-100

Radiology-pathology correlation:
Liver

Dawson, ASTRO 2007

Conclusions
• Image registration (IR) is a mandatory tool for
multi-modality planning and IGRT.
• IR is not perfect; impossible to represent the
‘whole patient’ at ‘all times’.
• As volume for matching is smaller, critical organ
doses outside matching volume may increase.
• IR can facilitate research in rad-path correlation
studies, patterns of recurrence analysis,
autocontouring, dose accumulation, adaptive
therapy, observer variability studies,…
• Clinical input and QA important despite
technological advances and automation in IR.

Acknowledgements
PMH
Outside PMH
Cynthia Menard
JJ Sonke, NKI
Andrea Bezjak
Geoff Hugo, WBH
John Waldron
Jake Van Dyk, London
Charles Catton
Jean Pouliot, UCSF
David Jaffray
Mike Sharpe
The 100s of people we have ever
Doug Moseley discussed image registration,
Jeff Siewerdsen multimodality imaging or IGRT
Tom Purdie with
Jean Pierre Bissonnette
Kristy Brock
Cynthia Eccles Funding:
Jane Higgins ASCO CDA
Robert Case NCIC
Regina Tse Canadian Cancer Society
Maria Hawkins
Elekta
Mark Lee

Multimodality & 4 D Imaging

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Andere mochten auch

Andere mochten auch (18)

Ähnlich wie Multimodality & 4 D Imaging

Ähnlich wie Multimodality & 4 D Imaging (20)

Mehr von fondas vakalis

Mehr von fondas vakalis (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Multimodality & 4 D Imaging