1. Optimal Control of chronic myeloid leukemia treatment
F.Angaroni
May 24, 2018
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 1 / 36
2. 1 The disease
Differential equations model
2 Optimal control
Analytic solution: the Pontryagins maximum principle
Numerical solution: the Pontryagins maximum principle
Optimal control: the CML therapy examples
Optimal control: the CML therapy
3 Future works
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 1 / 36
3. The disease: Mathematical hypothesis
This disease is driven by the BRC-ABL oncogene.
Since BCR-ABL mutation is present in all leukemic cells the ratio of
cancer cells respect to healthy cells is a ”simple” measurements.
About 2000 follow up could be found in literature. Their are an
example of measurements of the in vivo kinetics (F.Michor)
Exponential decay of cancer cells, and exponential blast after the stop
of the therapy, tell us that N(t) should be an exponential (solution of
first order differential equation) or described by a Poisson process.
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 2 / 36
4. The disease: Mathematical hypothesis
We divide the cells of the tiusse under study in ne non intersecting
ensambles, every ensamble represents a certain stage of cell differentiation.
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 3 / 36
5. Compartments Model
In this case, we divide the cells in
four ensambles:
1 Stem cells (SC)
2 Precursors Cells (PC)
3 Differentiated cells (PC)
4 Terminally differentiated cells
(TD)
Every collection of these ensembles
is a branch.
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 4 / 36
6. Differential equations model
Since BCR-ABL mutation is present in all leukemic cells we can distinguish
between
Healthy branch
Leukemic branch
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 5 / 36
7. Differential equations model
We attach a function ci,l (t) to every ensemble, it represents the number
of cells in one ensemble.
We will study transition rate between classes.
Given the following parameters:
pi,k the division rate of the cells
di,k the death rate of the cells
ai,k ∈ [0, 1] the probability of self-renewal
λ the probability for unit of time for a SC to develop the mutation
s(t) = 1
1+k( 4
i=1
h
j=l ci,j (t))
biochemical signal that regulate the cells
proliferation, it depends only on the number of mature cells
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 6 / 36
8. Differential equations model
Considering only symmetric differentiation, the transition rate for the ci,k
ensamble are:
+2pi,kai,kci (t)s(t) a in-going flux caused by the replication, this
contribute is absent for the TD ensembles,
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 7 / 36
9. Differential equations model
Considering only symmetric differentiation, the transition rate for the ci,k
ensamble are:
+s(t)pi−1,k(1 − ai−1,k)ci−1,k(t) a in-going flux due to the
differentiation in the previous ensemble of the branch, it is not present
for the SC (i = 1)
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 8 / 36
10. Differential equations model
Considering only symmetric differentiation, the transition rate for the ci,k
ensamble are: :
−di,kci,k out-going flux due to the death of cells,
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 9 / 36
11. Differential equations model
Considering only symmetric differentiation, the transition rate for the ci,k
ensamble are:
−s(t)pi ci (t) out-going flux due to the differentiation,
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 10 / 36
12. Differential equations model
Considering only symmetric differentiation, the transition rate are give by:
λc1,k in-going flux for c1,k+1 and out-going flux for c1,k. It represents
the generation of the tumor SC from the healthy SC.
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 11 / 36
14. Differential equations model
This tumor has a molecularly targeted therapy: Imatinib an inhibitor of
BCR-ABL gene.
The therapy is simulate as a decrease of 3 order of magnitude of division
rate of tumoral cells.
Figure: From:Dynamics of chronic myeloid leukemia F.Michor et al.
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 13 / 36
15. Differential equations model
The model is in a good agreement with the experimental data,
defining a figure of merit (FOM)
T(t) =
4
i=1 ci,l (t)
4
i=1
h
k=l ci,k(t)
. (2)
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 14 / 36
16. Differential equations model
The model presents a steady state where the tumoral stem cells are
constant: life-long disease.
c∗
1,l =
λc∗
1,h
(2a1,h − 1)p1,hs∗ − d1,l
, (3)
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 15 / 36
17. Differential equations model
The steady state is present only if:
d1,l c∗
1,l = [(2a1,l − 1)p1,l s∗
)]c∗
1,l + λc∗
1,h (4)
An early estimation of parameters could help in clinical management
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 16 / 36
18. Optimal control: motivation
The therapy has several drawbacks:
1 Fails in eradicating disease
Figure: From:Dynamics of chronic myeloid leukemia F.Michor et al.
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 17 / 36
19. Optimal control: motivation
The therapy has several drawbacks:
1 Fails in eradicating disease
2 Too expensive to be effective in a epidemiology contest
Figure: From:The price of drugs for chronic myeloid leukemia (CML) is a
reflection of the unsustainable prices of cancer drugs: from the perspective of a
large group of CML experts
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 18 / 36
20. Optimal control: motivation
The therapy has several drawbacks:
1 Fails in eradicating disease
2 Too expensive to be effective in a epidemiology contest
3 Is a life-long therapy (?)
Figure: From:Early molecular response and female sex strongly predict stable
undetectable BCR-ABL1, the criteria for imatinib discontinuation in patients with
CML
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 19 / 36
21. Optimal control: motivation
The therapy has several drawbacks:
1 Fails in eradicating disease
2 Too expensive to be effective in a epidemiology contest
3 Is a life-long therapy (?)
4 The therapy fails in approximately 15 − 25% of patients due to the
presence of resistant subclones, but Pharmacologic inhibitors for
imatinib-resistant CML exist and dose escalation can improve the
response in a subset of patients with resistance to standard dose
imatinib
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 20 / 36
22. Control Theory
Control Theory deals with systems that can be controlled, i.e. whose
evolution can be influenced by some external agent described by
u ∈ U ⊆ Rn, e.g.:
dx
dt
= f (x, u) (5)
There are two classes of control:
1 Open loop. Choose u as function of time t,
2 Closed loop or Feedback. Choose u as function of space variable x
Optimal control belongs to the first class.
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 21 / 36
23. Open loop control: the problem
Given the dynamics:
dx
dt
= f (x, u) (6)
T is the ensemble of final possible state x(T) at time t = T, if
x(0) = x0
L(x(t), u)dt is the running cost
φ(x(T)) is the pay-off
the optimal control problem is to find u∗(t) such that:
min[φ(x(T)) +
t
0
dtL(x(t), u(t))] x(0) = x0 x(T) ∈ T (7)
One approach to the optimal problem is the maximum principle :
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 22 / 36
24. Optimal control: Pontryagins Maximum Principle
The cornerstone of optimal control is the Pontryagins maximum
principle.
If u∗(t) and x∗(t) are optimal optimal solution and H(t, x(t), u(t), λ(t)) is
an Hamiltonian defined as follows:
H(t, x(t), u(t), λ(t)) = λf (t, u(t), x(t)) + L(u(t), x(t)) (8)
then there exists a piecewise differentiable adjoint variable λ(t) such that:
dλ(t)
dt
= −
∂H(t, x(t), u(t), λ(t))
dx
(9)
λ(T) = φ(x(T)) (10)
H(t, x∗
(t), u(t), λ(t)) ≤ H(t, x∗
(t), u∗
(t), λ(t)) (11)
with
∂H
∂u u=u∗
= 0 (12)
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 23 / 36
25. Optimal control: Numerical solution
Consider an optimal control system:
1) Cost
max φ(x(T)) +
T
t0
L(t, x(t), u(t))dt (13)
2) Dynamics
x(t0) = a
dx(t)
dt
= f (t, x(t), u(t)) (14)
3) The dynamics of the adjoint
λ(T) = φ(x(T))
dλ
dt
= −
∂H
∂x
(15)
4) Characterization of the optimal control
∂H
∂u u=u∗
= 0 (16)
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 24 / 36
26. Numerical solution: Forward-Backward Sweep Method
A rough outline of the algorithm to solve the optimal control problem is
the following:
1 Make an initial guess for u over the interval.
2 Solve Forward in time the Dynamics:
x(t0) = a
dx(t)
dt
= f (t, x(t), u(t)) (17)
3 Solve Backward in time the dynamics of the adjoint:
λ(T) = φ(x(T))
dλ
dt
= −
∂H
∂x
(18)
4 Update u using λ(t) and x(t) into the characterization of the
optimal control
∂H
∂u u=u∗
= 0 (19)
5 Check convergence. If values of the variables in this iteration and the
last iteration are negligibly close, output the current values as
solutions. If values are not close, return to Step 2.
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 25 / 36
28. Optimal control: the CML therapy
Where the control is given by:
U = {u(t)|∀t, 1 ≤ u(t)≤103}; (21)
We use the following cost:
C(t, x(t), u(t)) =
T
0
dt{Au2
(t) + B
4
i=1
c2
i,l (t)} (22)
where:
A economic cost per dose
u(t) is the control,
B 4
i=1 c2
i,l represents a running cost that depends on the size of the
tumor
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 27 / 36
29. Optimal control: the CML therapy
We define an Hamiltonian:
H = Au(t)2
+ B
4
i=1
c2
i,l (t) +
4
i=1
h
k=l
λi,k
dc1,k(t)
dt
(23)
We have 8 adjoint equation:
dλi,k
dt
= −
∂H
∂ci,h
i = 1, 2, 3, 4 k = l, h
and the characterization of the optimal control:
0 = −2Au(t) + log(u(t))(2a1,l − 1)p1,l s(t)c1,l +
+ log(u(t))(2(1 − a1,l )p1,l c1,l (t) + (2a2,l − 1)p2,l s(t)c2,l (t)))+
+ log(u(t))2(1 − a2,l )p2,l s(t)c2,l
(24)
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 28 / 36
30. Optimal control: the CML therapy
The Greedy controller (A = 100, B = 0 =⇒ u(t) = 1)
150 200 250 300
-4
-3
-2
-1
0
1
2
t (days)
Log[T]+2
150 200 250 300
0
2000
4000
6000
8000
10000
t (days)
cost
The Careful controller (A = 0, B = 1 =⇒ u(t) = 103)
150 200 250 300
-4
-3
-2
-1
0
1
2
t (days)
Log[T]+2
150 200 250 300
0
5.0×1014
1.0×1015
1.5×1015
t (days)
cost
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 29 / 36
31. Optimal control: the CML therapy
Problems:
the optimal solution exists and it is unique only for certain costs
Optimal control solutions implies continuous measurements and
action on the system: Optimal control is very complicated to apply to
a real system,
Since the functional space is very rich compared to a “discrete” space
the optimal control gives the upper bond of a decision process and
could be used to evaluate the sustainability of a therapy
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 30 / 36
32. Therapy control: future works
Using a random optimizer:
1 Consider the real pharmacodynamics and pharmacokinetics
Figure: From:Pharmacokinetics and pharmacodynamics of dasatinib in the chronic
phase of newly diagnosed chronic myeloid leukemia
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 31 / 36
33. Therapy control: future works
Using a random optimizer:
1 Consider the real pharcodynamics
2 Consider a more realistic cost, not quadratic
C(t, x(t), u(t)) =
T
0
dt{Ad(u) + Bd(u) + D
4
i=1
ci,l (t)} (25)
where:
d(u) the function that represent the dosage of the molecular therapy,
A economic cost per dose, B quality of life cost per dose (toxicity)
D 4
i=1 ci,l (t) represents a cost that depends on the number of
leukemic cells
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 32 / 36
34. Therapy control: future works
Using a random optimizer:
1 Consider the real pharcodynamics
2 Consider a more realistic cost
3 Consider a resistant branch and then manage the switch of the
therapy
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 33 / 36
35. Therapy control: future works
Using a random optimizer:
1 Consider the real pharcodynamics
2 Consider a more realistic cost
3 Consider a resistant branch and then manage the switch of the
therapy
4 Consider a discrete follow up
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 34 / 36
36. Therapy control: future works
Using a random optimizer:
1 Consider the real pharcodynamics
2 Consider a more realistic cost
3 Consider a discrete follow up
4 Consider resistant branch and then the switch of the therapy
5 Consider a hybrid therapy with “conventional“ radiotherapy or
chemotherapy with the aim of eradicate residual cancer stem cells
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 35 / 36
37. Thank you
F.Angaroni (U. Bicocca) Optimal Control of the chronic myeloid leukemia treatment 23 May 2018 36 / 36