An approach to algorithmically synthesize control strategies for set-to-set transitions of uncertain discrete-time switched linear systems based on a combination of tree search and reachable set computations in a stochastic setting is proposed in this presentation. The initial state and disturbances are assumed to be Gaussian distributed, and a time-variant hybrid control law stabilizes the system towards a goal set. The algorithmic solution computes sequences of discrete states via tree search and the continuous controls are obtained from solving embedded semi-definite programs (SDP). These program taking polytopic input constraints as well as timevarying probabilistic state constraints into account. An example for demonstrating the principles of the solution procedure with focus on handling the chance constraints is included.

1. Control & System
Theory
Probabilistic Control of Switched Linear Systems
with Chance Constraints
Leonhard Asselborn Olaf Stursberg
Control and System Theory
University of Kassel (Germany)
l.asselborn@uni-kassel.de
stursberg@uni-kassel.de
European Control Conference 2016
www.control.eecs.uni-kassel.de 01.07.2016

2. Introduction: Motivation in Uncertain Environment Control & System
Theory
• dynamic system with different modes (1. fast straight motion, 2. slow
curved motion, etc.)
• stochastic disturbances (wind)
• stochastic initialization (GPS coordinates)
• input constraints: uk ∈ U
• state constraints (safety critical): xk ∈ Xk cannot be guaranteed for all
disturbances → state chance constraint: Pr(xk ∈ Xk) ≥ δx
• steer the system into a terminal region T with confidence δ
x0
p
p
xN
estimated
position of
an obstacle
T
Introduction Problem Definition Method Example Conclusion Appendix 2

3. Relevant Literature (Excerpt): Control & System
Theory
Switched Linear Systems
• Liberzon [2003], Sun [2006], Sun and Ge [2011]: stability conditions for arbitrary
switching, no input constraints
Chance Constraint Optimization
• Galafiore and Campi [2005], Blackmore et al. [2010], Asselborn et al. [2012],
Prandini et al. [2015]: scenario-based approach
• van Hessem et al. [2001,2002], Ma and Borelli [2012]: set-based approach
• Prekopa [1999], Blackmore and Ono [2009], Blackmore et al [2011], Vitus and
Tomlin [2011]: Boole’s inequality for separate handling of chance constraints
Reachability sets for stochastic hybrid systems
• Hu et al. [2000], Blom and Lygeros [2006], Cassandras and Lygeros [2006],
Kamgarpour et al. [2013], Abate et al. [2008]: control design
Previous own work
• Controller synthesis for nonlinear systems: [NOLCOS, 2013]
• Synthesis for stochastic discrete-time linear systems: [ROCOND, 2015]
• Synthesis for stochastic discrete-time switched linear systems: [ADHS, 2015]
Introduction Problem Definition Method Example Conclusion Appendix 3

4. Contribution Control & System
Theory
Controller synthesis based on probabilistc reachability computation for
discrete-time stochastic hybrid systems with state chance constraints
Solution Approach:
• forward propagation of ellipsoidal reachable sets Xδ
k with confidence δ
Xδ
0
Xk
Xδ
N
T
• offline controller synthesis by semi-definite programming (SDP) and tree
search
• approximation of chance constraints via Boole’s inequality
Introduction Problem Definition Method Example Conclusion Appendix 4

5. Sets, Distributions and Dynamic System (1) Control & System
Theory
Set representation:
• Ellipsoid: E := ε(q, Q) = x ∈ Rn
| (x − q)T
Q−1
(x − q) ≤ 1
with q ∈ Rn
, Q ∈ Rn×n
• Polytope: P := {x ∈ Rn
| Rx ≤ b} with R ∈ Rnp×n
, b ∈ Rnp
Multivariate Normal Distribution:
ξ ∼ N(µ, Ω)
The sum of two Gaussian variables ξ1 ∼ N(µ1, Ω1) and ξ2 ∼ N(µ2, Ω2) is
again a Gaussian variable:
ξ1 + ξ2 ∼ N(µ1 + µ2, Ω1 + Ω2)
Introduction Problem Definition Method Example Conclusion Appendix 5

6. Sets, Distributions and Dynamic System (2) Control & System
Theory
Probabilistic Switched Affine Systems (PSAS):
xk+1 = Azk xk + Bzk uk + Gzk vk, k ∈ {0, 1, 2, . . .}
x0 ∼ N(q0, Q0), xk ∈ Rn
Xk = {xk | Rx,kxk ≤ bx,k}
vk ∼ N(0, Σ), vk ∈ Rn
, iid
uk ∈ U = {uk | Ruuk ≤ bu} ⊆ Rm
zk ∈ Z = {1, 2, . . . , nz}
Feasible system execution for k ∈ N0:
1. select zk ∈ Z to determine the tuple (Azk , Bzk , Gzk )
2. sample the disturbance vk ∼ N(0, Σ)
3. choose a suitable input uk ∈ U
4. evaluate the continuous dynamics to get the new state xk+1
Introduction Problem Definition Method Example Conclusion Appendix 6

7. Probabilistic Reachable Sets with Confidence δ Control & System
Theory
• Surfaces of equal density for ξ ∼ N(µ, Ω)
(Krzanowski and Marriott [1994]):
(ξ − µ)T
Ω−1
(ξ − µ) = c
with χ2
-distributed random variable and:
δ := Pr (ξ ∈ ε(µ, Ωc)) = Fχ2 (c, n)
cumulative distribution function
• Initial state confidence ellipsoid:
Xδ
0 := ε(q0, Q0c) with Pr(x0 ∈ Xδ
0 ) = δ contour of pdf
samples of ξ ∼ N (µ, Ω)
ξ2
ξ1
• Evolution of the state distribution:
qk+1 = Azk qk + Bzk uk, Qk+1 = Azk QkAT
zk
+ Gzk ΣGT
zk
Xδ
k+1 := ε(qk+1, Qk+1c
=:Qδ
k+1
)
Xδ
k+1 is the confidence ellipsoid for xk+1 with confidence δ.
Introduction Problem Definition Method Example Conclusion Appendix 7

8. Problem Definition Control & System
Theory
Problem
Given PSAS, determine a hybrid control law κk = (λk(xk), νk) for which it
holds that:
• uk = λk(xk) ∈ U, zk = νk ∈ Z and xk ∈ Xδ
k
∀ k ∈ {0, 1, . . . , N − 1}, N ∈ N
• the chance constraint Pr(xk ∈ Xk) ≥ δx is satisfied ∀k ∈ {0, . . . , N − 1}
• Xδ
N ⊆ T for a finite N ≤ Nmax.
Thus, any initial state x0 ∈ Xδ
0 has to be transferred into the terminal set T
with probability δ after N steps.
Introduction Problem Definition Method Example Conclusion Appendix 8

9. Main Idea Control & System
Theory
Solution procedure:
1. tree search algorithm (Asselborn and Stursberg [2015]) to determine the
sequence of discrete control inputs νk ∈ Z
2. solution of an SDP provides continuous control law:
uk = λ(xk) = −Kkxk + dk
with closed-loop dynamics:
xk+1 = Azk xk + Bzk uk + Gzk vk
= (Aνk − Bνk Kk)
:=Acl,k,νk
xk + Bνk dk + Gνk vk
Main Challenge:
• state chance constraint with multi-dimensional integral:
Pr(xk ∈ Xk) =
w∈Xk
N (qx,k, Qx,k) dw ≥ δx
• no closed form solution available
Idea: separate consideration of each half-space of Xk
Introduction Problem Definition Method Example Conclusion Appendix 9

10. Approximation of State Chance Constraints Control & System
Theory
• projection of the random variables onto each hyperplane
rx,k,1
rx,k,2
Rx,k
xk ≤
bx,k,1
bx,k,2
bx,k
yk,i := rx,k,i · xk ∼ N(qy,k,i, Qy,k,i)
qx,k
rx,k,1
rx,k,2
• yk,1 is a univariate random variable (Blackmore and Ono [2009])
qy,k,i = rx,k,iqx,k,i, Qy,k,i = rx,k,iQx,k,irT
x,k,i
• consideration of the violation of the chance constraint: Pr(yk,i > bx,k,i)
• with Boole’s inequality Pr nx
i=1 yk,i > bx,k,i ≤ nx
i=1 Pr(yk,i > bx,k,i)
the following Lemma holds:
Lemma
The chance constraint Pr(xk ∈ Xk) ≥ δx is satisfied, if it holds for
i ∈ {1, 2, . . . , nx} that:
ǫi := Pr(yk,i > bx,k,i),
nx
i=1
ǫi < 1 − δx
Introduction Problem Definition Method Example Conclusion Appendix 10

11. Approximation of State Chance Constraint (2) Control & System
Theory
Transformation into a standard normal distribution
yk,i ∼ N(qy,k,i, Qy,k,i), bx,k,i → yk,i ∼ N(0, 1),
bx,k,i − qy,k,i
Qy,k,i
Evaluation of the probability with the cumulative distribution function:
Pr(yk,i > bx,k,i) = 1 − cdf
bx,k,i − qy,k,i
Qy,k,i
with:
cdf(x) =
1
√
2π
x
−∞
e− z2
2 dz, no closed form solution available
In Soranzo [2014]:
cdf(x) ≈ fcdf (x) = 2−221−41x/10
with max error: 1.27e − 4 for x ≥ 0
Introduction Problem Definition Method Example Conclusion Appendix 11

12. Solution based on SDP (1) Control & System
Theory
Linearization of fcdf (x):
¯fcdf (qy,k,i, Qy,k,i, bx,k,i) :=fcdf (¯qy, ¯Qy, bx,k,i) + . . .
∇qy fcdf (qy,k,i − ¯qy) + ∇Qy fcdf (Qy,k,i − ¯Qy)
Approximation of Pr(xk ∈ Xk) ≥ δx by:
ǫi = 1 − ¯fcdf (qy,k,i, Qy,k,i, bx,k,i) ,
nx
i=1
ǫi < 1 − δx, ∀i = {1, 2, . . . nx}
Introduction Problem Definition Method Example Conclusion Appendix 12

13. Solution based on SDP (2) Control & System
Theory
• Convergence of the covariance matrix of N(qx,k+1, Qx,k+1):
Sk+1 ≥ Qx,k+1 = Acl,k,zk
Qx,kAT
cl,k,zk
+ GΣGT
or with Schur complement:


Sk+1 Acl,k,zk
Qx,k GΣ
Qx,kAT
cl,k,zk
Qx,k 0
ΣGT
0 Σ

 ≥ 0
• Convergence of the expected value qx,k
use of flexible Lyapunov functions. (Lazar et al. [2009] ) suitable for
switched dynamics
V
k
Introduction Problem Definition Method Example Conclusion Appendix 13

14. Solution based on SDP (3) Control & System
Theory
Proposition
The input constraint uk = −Kkxk + dk ∈ U holds for Kk, dk and all xk ∈ Xδ
k
if:
(bu,i − ru,idk)In −ru,iKk(Qδ
x,k)− 1
2
−(Qδ
x,k)− 1
2 KT
k rT
u,i bu,i − ru,idk
≥ 0 ∀i = {1, . . . , nu}.
• ru,i and bu,i denote the i−th row of Ru and bu, respectively.
• Xδ
k is mapped into a unit ball by a suitable coordinate transformation
h(xk)
• the Euclidean norm ||h(xk)||2 ≤ 1 can be expressed as LMI, which results
in the above formulation
• complete proof can be found in Asselborn et al. [2013]
Introduction Problem Definition Method Example Conclusion Appendix 14

15. Determination of the Continuous Controller Control & System
Theory
Semidefinite program to be solved for chosen zk ∈ Z:
min
Sk+1,Kk,dk
Jk,zk
center point convergence:



qT
x,k+1,zk
Lqx,k+1,zk
− ρqT
x,kLqx,k ≤ αk
qx,k+1,zk
= (Azk − Bzk Kk)qx,k + Bzk dk
αk ≤ maxl∈{1,...,k} ωl
αk−l
ellipsoidal shape convergence:






Sk+1 Acl,k,zk
Qx,k Gzk Σ
Qx,kAT
cl,k,zk
Qx,k 0
ΣGT
zk
0 Σ


 ≥ 0
trace(Sk+1) ≤ trace(Qk)
input constraint:



(bu,i − ru,idk)In −ru,iKk(Qδ
x,k)− 1
2
−(Qδ
x,k)− 1
2 KT
k rT
u,i bu,i − ru,idk
≥ 0,
∀i = {1, . . . , nu}
state chance constraint:
ǫi = 1 − ¯fcdf (qy,k+1,i, Qy,k+1,i, bx,k+1,i) ,
nx
i=1 ǫi < 1 − δx, ∀i = {1, 2, . . . nx}
Introduction Problem Definition Method Example Conclusion Appendix 15

16. Determination of the Hybrid Controller Control & System
Theory
Probabilistic Ellipsoidal Control Algorithm (PECA)
Given: x0 ∼ N(qx,0, Qx,0), vk ∼ N(0, Σ), U = {uk | Ruuk ≤ bu},
Xk = {xk | Rx,kxk ≤ bx,k}, T, δ, δx, γmin, ω, ρ, α0
Define: k := 0, γ0 = γmin, O0 = ∅
while Xδ
k T and γk ≥ γmin do
(1) for i = 1, . . . , nz do
◮ compute Xδ
k, and solve the SDP problem for z = i and Xδ
k
◮ if solution exists do Ok := Ok ∪ {i} else Ok := Ok end
end
(2) if Ok = ∅ do choose the tuple (Kk, dk, νk) with best performance
else
if k = 0 do Termination without success
else k = k − 1, Ok := Ok {νk}, go to step (2) end
end
(3) compute (qx,k+1, Qx,k+1) with the selected controller (Kk, dk, νk)
(4) check state chance constraint:
if nx
j=1 1 − fcdf (qy,k+1,j, Qy,k+1,j, bx,k,j ) > 1 − δx do mark νk = i as
infeasible solution, go to step 2
(5) compute γk+1 = qk+1 − qk , k := k + 1, end while
Introduction Problem Definition Method Example Conclusion Appendix 16

17. Termination with success Control & System
Theory
Lemma
The control problem with a confidence δ, an initialization x0 ∼ N(qx,0, Qx,0),
vk ∼ N(0, Σ), (λk(xk), νk) ∈ U × Z ∀ k, and Pr(xk ∈ Xk) ≥ δx is
successfully solved with selected parameters γmin, ω, ρ and α0, if PECA
terminates in N steps with Xδ
N ⊆ T.
Proof: by construction
If no success: adjust δ, γmin, ω, ρ, tree search strategy
Introduction Problem Definition Method Example Conclusion Appendix 17

18. Numerical Example (1) Control & System
Theory
Initial distribution and disturbance:
x0 ∼ N(qx,0, Qx,0) with qx,0 =
−10
50
, Qx,0 =
1 0
0 1
vk ∼ N(0, Σ) with Σ =
0.02 0.01
0.01 0.02
.
Discrete input set: Z = {1, 2, 3}
The continuous dynamic is specified by the following system matrices:
A1 =
1.35 −0.06
0.11 0.95
, A2 =
0.82 0.05
−0.14 1.10
, A3 =
0.86 0.05
−0.09 0.99
B1 =
0.58 −0.03
0.03 0.97
, B2 =
0.48 0.01
1.01 0.53
, B3 =
0.49 0.01
0.98 0.50
G1 = G2 = G3 =
0.1 0.05
0.08 0.2
Note that all three subsystems are chosen to have unstable state matrices.
Introduction Problem Definition Method Example Conclusion Appendix 18

19. Numerical Example (2) Control & System
Theory
Input constraints:
uk ∈ U =



u ∈ R2
|




1 0
0 1
−1 0
0 −1



 u ≤




3
3
3
3







,
Target set:
T = ε 0,
0.96 0.64
0.64 0.8
Cost function:
Jk = trace
Sk+1 0
0 0.8 qx,k+1
State chance constraint:
Pr(xk ∈ Xk) ≥ δx = 0.95, Xk = {x ∈ Rn
| Rx,kxk ≤ bx,k}
with:
Rx,k =
1 0 −1 0 − 2√
2
3
0 1 0 −1 − 1√
2
1
T
bx,k = 5 60 20.5 5 2.5 + 0.2k 40
T
Parameters: δ = 0.95, γmin = 0.01, α0 = 10−4
, ω = 0.8 and ρ = 0.98
Introduction Problem Definition Method Example Conclusion Appendix 19

20. Numerical Example (3) Control & System
Theory
x2
x1
Xδ
0
T
−20 −15 −10 −5 0 5
0
10
20
30
40
50
60
• Termination with N = 38 steps
in 80s using 2.8 Ghz Quad-Core
CPU
• Implementation with Matlab
7.12.0, YALMIP 3.0, SeDuMi
1.3, and ellipsoidal toolbox ET
(Kurzhanskiy and Varaiya
[2006]).
Xδ
27
Xδ
26
Xδ
25
x2
x1
−8 −7.5 −7 −6.5 −6
2
3
4
5
6
7
Introduction Problem Definition Method Example Conclusion Appendix 20

21. Conclusion and Outlook Control & System
Theory
Summary:
• Algorithm for control of PSAS with state chance constraints
• Offline control law synthesis based on a combination of probabilistic
reachability analysis and tree search
• Explicit consideration of input constraints
• Tight approximation of state chance constraints
Future work:
• Consideration of piecewise-affine systems with autonomous switching
• Explore measures to reduce the computational complexity
Introduction Problem Definition Method Example Conclusion Appendix 21

22. End Control & System
Theory
Thank you for your attention!
Introduction Problem Definition Method Example Conclusion Appendix 22

23. Attractivity and Stochastic Stability Control & System
Theory
Stability with confidence δ
The switched uncertain linear system is called attractive with confidence δ on a
bounded time domain [0, N], if for any initial condition x0 ∈ Xδ
0 and any
vk ∈ ε(0, Σc), finite parameters ¯q ∈ Rn
and ¯Q ∈ Rn×n
exist such that:
||qN || ≤ ||¯q||, ||QN || ≤ || ¯Q||.
The system is said stable with confidence δ on a bounded time domain [0, N] if
in addition
||qk+1|| ≤ ||qk||, ||Qk+1|| ≤ ||Qk||.
holds for any 0 ≤ k ≤ N − 1.
Interpretation:
• qk converges to a finite neighborhood of the origin
• covariance matrix Qk converges, such that the confidence ellipsoid is of
decreasing size over k (while rotation is still possible).
Introduction Problem Definition Method Example Conclusion Appendix 23

24. References Control & System
Theory
Liberzon, D.
Switching in Systems and Control
Birkhaeuser, 2003
Sun, Z.
Switched Linear Systems: Control and Design
Springer, 2006
Sun, Z. and Ge, S.S.
Stability theory of switched dynamical systems;
Springer. 2011
Blackmore, L. and Ono, M.
Convex chance constrained predictive control without sampling
Proceedings of the AIAA Guidance, Navigation and Control Conference, 2009
Blackmore, L., Ono, M., William, B.C.
Chance-constrained optimal path planning with obstacles
IEEE Transactions on Robotics, 2011
Vitus, M.P. and Tomlin, C.J.
Closed-loop belief space planning for linear gaussian systems
IEEE Conference on Robotics and Automation, 2011
Calafiore, G. and Campi, M.C.
Uncertain convex programs: randomized solutions and confidence levels
Mathematical Programming, 2005
Blackmore, L., Ono, M., Bektassov, A., Williams, B.C.
A probabilistic particle-control approximation of chance-constrained stochastic predictive control
IEEE Transactions on Robotics, 2010
Introduction Problem Definition Method Example Conclusion Appendix 24

25. References Control & System
Theory
Asselborn, L., Jilg, M., and Stursberg, O.
Control of uncertain hybrid nonlinear systems using particle filters
In 4th IFAC Conf. on Analysis and Design of Hybrid Systems, 2012
Prandini, M., Garatti, S., Vignali, R.
Performance assessment and design of abstracted models for stochastic hybrid systems through a randomized approach
Autmatica, vol. 50, 2014
Hu, J., Lygeros, J., and Sastry, S.
Towards a theory of stochastic hybrid systems;
In Hybrid systems: Computation and Control, volume 1790, 160-173. Springer. 2000
Blom, H.A. and Lygeros, J.
Stochastic hybrid systems: theory and safety critical applications;
volume 337. Springer. 2006
Cassandras, C.G. and Lygeros, J.
Stochastic hybrid systems;
CRC Press.2006
Kamgarpour, M., Summers, S., and Lygeros, J.
Control design for specifications on stochastic hybrid systems;
In Hybrid systems: computation and control, 303-312. ACM. 2013
Abate, A., Prandini, M., Lygeros, J., and Sastry, S.
Probabilistic reachability and safety for controlled discrete time stochastic hybrid systems;
Automatica,44(11), 2724-2734. 2008
Asselborn, L., Gross, D., and Stursberg, O.
Control of uncertain nonlinear systems using ellipsoidal reachability calculus;
In 9th IFAC Symp. on Nonlinear Control Systems, 50-55. 2013
Introduction Problem Definition Method Example Conclusion Appendix 25

26. References Control & System
Theory
Asselborn, L. and Stursberg, O.
Probabilistic control of uncertain linear systems using stochastic reachability;
In 8th IFAC Symp. on Robust Control Design. 2015
Asselborn, L. and Stursberg, O.
Robust Control of Uncertain Switched Linear Systems based on Stochastic Reachability
In 5th IFAC Conf. on Analysis and Design of Hybrid Systems, 2015
Soranzo, E.E.A.
Very simple explicit invertible approximation of normal cumulative and normal quantile function
Applied Mathematical Science, 2014
Boyd, S.P., El Ghaoui, L., Feron, E., and Balakrishnan, V.
Linear matrix inequalities in system and control theory;
volume 15. SIAM. 1994
Lazar, M.
Flexible control lyapunov functions;
In American Control Conf., 102-107.2009
Kurzhanski, A. and Varaiya, P.:
Ellipsoidal Calculus for Estimation and Control;
Birkh¨auser, 1996.
Introduction Problem Definition Method Example Conclusion Appendix 26