This document summarizes Frances Kuo's work applying Quasi-Monte Carlo (QMC) methods to solve partial differential equations (PDEs) with random coefficients. It introduces a motivating example of modeling groundwater flow with uncertainty in porous medium properties. It then provides an overview of QMC methods, including advantages over Monte Carlo, construction techniques like lattice rules, and three theoretical settings for applying QMC to PDEs with random coefficients. The document outlines Kuo's collaborations applying QMC to problems with uniform and lognormal random coefficients under these different settings.

1. Frances Kuo @ UNSW Sydney, Australia p.1
Application of QMC methods to
PDEs with random coefficients
Frances Kuo
f.kuo@unsw.edu.au
University of New South Wales, Sydney, Australia
joint work with
Ivan Graham and Rob Scheichl (Bath), Dirk Nuyens (KU Leuven),
Christoph Schwab (ETH Zurich), Elizabeth Ullmann (TU Munich),
James Nichols (Paris 6), Mahadevan Ganesh (Colorado), Mike Giles (Oxford),
Grzegorz Wasilkowski (Kentucky), Leszek Plaskota (Warsaw),
Michael Feischl, Alexander Gilbert,
Josef Dick, Thong Le Gia, and Ian Sloan (UNSW)

2. Frances Kuo @ UNSW Sydney, Australia p.2
Outline
Motivating example – a flow through a porous medium
Introduction to QMC methods
QMC theory
Application of QMC to PDEs with random coefficients
Cost reduction strategies
Software
Concluding remarks

3. Frances Kuo @ UNSW Sydney, Australia p.3
Motivating example
Uncertainty in groundwater flow
eg. risk analysis of radwaste disposal or CO2 sequestration
Darcy’s law q + a ∇p = κ
mass conservation law ∇ · q = 0
in D ⊂ Rd
, d = 1, 2, 3
together with boundary conditions
Uncertainty in a(xxx, ω) leads to uncertainty in q(xxx, ω) and p(xxx, ω)

4. Frances Kuo @ UNSW Sydney, Australia p.4
Expected values of quantities of interest
To compute the expected value of some quantity of interest:
1. Generate a number of realizations of the random field
(Some approximation may be required)
2. For each realization, solve the PDE using e.g. FEM / FVM / mFEM
3. Take the average of all solutions from different realizations
This describes Monte Carlo simulation.
Example: particle dispersion
p = 1 p = 0
∂p
∂n = 0
∂p
∂n = 0
◦release point →
•
0
0.2
0.4
0.6
0.8
1
0 0.2 0.4 0.6 0.8 1

5. Frances Kuo @ UNSW Sydney, Australia p.5
Expected values of quantities of interest
To compute the expected value of some quantity of interest:
1. Generate a number of realizations of the random field
(Some approximation may be required)
2. For each realization, solve the PDE using e.g. FEM / FVM / mFEM
3. Take the average of all solutions from different realizations
This describes Monte Carlo simulation.
NOTE: expected value = (high dimensional) integral
s = stochastic dimension
→ use quasi-Monte Carlo methods
10
3
10
4
10
5
10
6
10
−5
10
−4
10
−3
10
−2
10
−1
error
n
n−1/2
n−1
or better QMC
MC

6. Frances Kuo @ UNSW Sydney, Australia p.6
MC v.s. QMC in the unit cube
[0,1]s
f(yyy) dyyy ≈
1
n
n
i=1
f(ttti)
Monte Carlo method Quasi-Monte Carlo methods
ttti random uniform ttti deterministic
n−1/2
convergence close to n−1
convergence or better
0 1
1
64 random points
•
•
•
•
•
•
•
•
•
•
•
•••
•
•
• •
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
• •
•
••
•
•
•
•
•
••
•
•
•
•
•
•
••
•
0 1
1
First 64 points of a
2D Sobol′ sequence
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
0 1
1
A lattice rule with 64 points
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
more effective for earlier variables and lower-order projections
order of variables irrelevant order of variables very important
use randomized QMC methods for error estimation

7. Frances Kuo @ UNSW Sydney, Australia p.7
QMC
Two main families of QMC methods:
(t,m,s)-nets and (t,s)-sequences
lattice rules
0 1
1
First 64 points of a
2D Sobol′ sequence
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
0 1
1
A lattice rule with 64 points
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
(0,6,2)-netHaving the right number of
points in various sub-cubes
A group under addition modulo Z
and includes the integer points
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
••
•
••
• •
•
•
•• •
•
•
•
•
•
•
• ••
•
•
•
•
•
•
•
•
•
•
•
•
•
Important developments:
component-by-component (CBC) construction, “fast” CBC
higher order digital nets
Niederreiter book (1992)
Sloan and Joe book (1994)
Lemieux book (2009)
Dick and Pillichshammer book (2010)
Dick, K., Sloan Acta Numerica (2013)
Leobacher and Pillichshammer book
(2014)

8. Frances Kuo @ UNSW Sydney, Australia p.8
Lattice rules
Rank-1 lattice rules have points
ttti = frac
i
n
zzz , i = 1, 2, . . . , n
zzz ∈ Zs
– the generating vector, with all components coprime to n
frac(·) – means to take the fractional part of all components
0 1
1
A lattice rule with 64 points
64
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
∼ quality determined by the choice of zzz ∼
n = 64 zzz = (1, 19) ttti = frac
i
64
(1, 19)

9. Frances Kuo @ UNSW Sydney, Australia p.9
Randomly shifted lattice rules
Shifted rank-1 lattice rules have points
ttti = frac
i
n
zzz + ∆∆∆ , i = 1, 2, . . . , n
∆∆∆ ∈ [0, 1)s
– the shift
shifted by
∆∆∆ = (0.1, 0.3)
0 1
1
A lattice rule with 64 points
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
0 1
1
A shifted lattice rule with 64 points
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
∼ use a number of random shifts for error estimation ∼

10. Frances Kuo @ UNSW Sydney, Australia p.10
Component-by-component construction
Want to find zzz for which some error criterion is as small possible
CBC algorithm [Korobov (1959); Sloan, Reztsov (2002); Sloan, K., Joe (2002),. . . ]
1. Set z1 = 1.
2. With z1 fixed, choose z2 to minimize the error criterion in 2D.
3. With z1, z2 fixed, choose z3 to minimize the error criterion in 3D.
4. etc.
[Hickernell (1997); Sloan, Wo´zniakowski (1998)]
[K. (2003); Dick (2004)]
Optimal rate of convergence O(n−1+δ
) in “weighted Sobolev space”,
independently of s under an appropriate condition on the weights
[Nuyens, Cools (2006)]
Cost of algorithm for “product weights” is O(n log n s) using FFT
Extensible/embedded variants
[Hickernell, Hong, L’Ecuyer, Lemieux (2000); Hickernell, Niederreiter (2003)]
[Cools, K., Nuyens (2006)]
[Dick, Pillichshammer, Waterhouse (2007)]
http://people.cs.kuleuven.be/~dirk.nuyens/fast-cbc/
http://people.cs.kuleuven.be/~dirk.nuyens/qmc-generators/

11. Frances Kuo @ UNSW Sydney, Australia p.11
Application of QMC to PDEs with random coefficients
[0] Graham, K., Nuyens, Scheichl, Sloan (J. Comput. Physics, 2011)
[1] K., Schwab, Sloan (SIAM J. Numer. Anal., 2012)
[2] K., Schwab, Sloan (J. FoCM, 2015)
[3] Graham, K., Nichols, Scheichl, Schwab, Sloan (Numer. Math., 2015)
[4] K., Scheichl, Schwab, Sloan, Ullmann (Math. Comp., to appear)
[5] Dick, K., Le Gia, Nuyens, Schwab (SIAM J. Numer. Anal., 2014)
[6] Dick, K., Le Gia, Schwab (SIAM J. Numer. Anal., 2016)
[7] Graham, K., Nuyens, Scheichl, Sloan (in progress)
Also Schwab
Le Gia
Dick, Le Gia, Schwab
Dick, Gantner, Le Gia, Schwab
Gantner, Schwab
Gantner, Herrmann, Schwab
Kazashi
Harbrecht, Peters, Siebenmorgen
Ganesh, Hawking
Robbe, Nuyens, Vandewalle
Scheichl, Stuart, Teckentrup
Graham, Parkinson, Scheichl
Gilbert, Graham, K., Scheichl, Sloan
Feischl, K., Sloan
Ganesh, K., Sloan
etc.

12. Frances Kuo @ UNSW Sydney, Australia p.12
Application of QMC to PDEs with random coefficients
Three different QMC theoretical settings:
[1,2] Weighted Sobolev space over [0, 1]s
and “randomly shifted lattice rules”
[3,4] Weighted space setting in Rs
and “randomly shifted lattice rules”
[5,6] Weighted space of smooth functions over [0, 1]s
and (deterministic)
“interlaced polynomial lattice rules”
Uniform Lognormal Lognormal
KL expansion Circulant embedding
Experiments only [0]*
First order, single-level [1] [3]* [7]*
First order, multi-level [2] [4]*
Higher order, single-level [5]
Higher order, multi-level [6]*
The * indicates there are accompanying numerical results
K., Nuyens (J. FoCM, 2016) – a survey of [1,2], [3,4], [5,6] in a unified view
Accompanying software package
http://people.cs.kuleuven.be/~dirk.nuyens/qmc4pde/

13. Frances Kuo @ UNSW Sydney, Australia p.13
5-minute QMC primer
Common features among all three QMC theoretical settings:
Separation in error bound
(rms) QMC error ≤ (rms) worst case errorγγγ × norm of integrandγγγ(rms) QMC error ≤ (rms) worst case errorγγγ × norm of integrandγγγ
Weighted spaces
Pairing of QMC rule with function space
Optimal rate of convergence
Rate and constant independent of dimension
Fast component-by-component (CBC) construction
Extensible or embedded rules
Application of QMC theory
Estimate the norm (critical step)
Choose the weights
Weights as input to the CBC construction

14. Frances Kuo @ UNSW Sydney, Australia p.14
Overview
Motivating example
Introduction to QMC
Application
PDEs with random coefficients
uniform
lognormal
Setting 1
domain [0, 1]s
randomly shifted lattice rules
mixed first derivatives
first order convergence
Setting 2
domain Rs
randomly shifted lattice rules
mixed first derivatives
first order convergence
Setting 3
domain [0, 1]s
interlaced polynomial lattice rules
mixed higher derivatives
higher order convergence

15. Frances Kuo @ UNSW Sydney, Australia p.15
Uniform v.s. lognormal
−∇ · (a(xxx,yyy) ∇u(xxx,yyy)) = κ(xxx), xxx ∈ D ⊂ Rd
, d = 1, 2, 3
u(xxx,yyy) = 0, xxx ∈ ∂D
Uniform [Cohen, DeVore, Schwab (2011)]
a(xxx,yyy) = a(xxx) +
j≥1
yj ψj(xxx), yj ∼ U[−1
2
, 1
2
]
Goal:
(−
1
2
,
1
2
)N
G(u(·,yyy)) dyyy G – bounded linear functional
Lognormal [many references]
a(xxx,yyy) = exp(Z(xxx,yyy))
Karhunen–Loève expansion:
Z(xxx,yyy) = Z(xxx) +
j≥1
yj
√
µj ξj(xxx), yj ∼ N(0, 1)
Goal:
RN
∗
G(u(·,yyy))
j≥1
φnor(yj) dyyy

16. Frances Kuo @ UNSW Sydney, Australia p.16
Error analysis (the uniform case, single-level algorithm)
[K., Nuyens (2016) – survey]
(−
1
2
,
1
2
)N
G(u(·, yyy)) dyyy ≈
1
n
n
i=1
G(us
h(·,ttti)) (1) dimension truncation
(2) FE discretization
(3) QMC quadrature
Balance three sources of error
QMC error
Estimate the norm G(us
h) γγγ by “differentiating the PDE”
Choose the weights to minimize (bound on ewor
γγγ )×(bound on G(us
h) γγγ )
Get POD weights (product and order dependent) ⇒ feed into fast CBC construction

17. Frances Kuo @ UNSW Sydney, Australia p.17
TUTORIAL: Differentiate the PDE (the uniform case)
1. Variational formulation
D
a(xxx, yyy) ∇u(xxx, yyy) · ∇w(xxx) dxxx =
D
κ(xxx) w(xxx) dxxx ∀w ∈ H1
0 (D)
2. Mixed derivatives in yyy with multi-index ννν = 000
D
∂ννν
a(xxx, yyy) ∇u(xxx, yyy) · ∇w(xxx) dxxx = 0 ∀w ∈ H1
0 (D)
3. Leibniz product rule
D mmm≤ννν
ννν
mmm
(∂mmm
a)(xxx, yyy) ∇(∂ννν−mmm
u)(xxx, yyy) · ∇w(xxx) dxxx = 0 ∀w ∈ H1
0 (D)
4. Uniform case
a(xxx, yyy) = a(xxx) +
j≥1
yj ψj(xxx) =⇒ (∂mmm
a)(xxx, yyy) =



a(xxx, yyy) if mmm = 000,
ψj(xxx) if mmm = eeej,
0 otherwise
5. Separate out the mmm = 000 term
D
a(xxx, yyy) ∇(∂ννν
u)(xxx, yyy) · ∇w(xxx) dxxx
= −
j≥1
νj
D
ψj(xxx) ∇(∂ννν−eeej u)(xxx, yyy) · ∇w(xxx) dxxx ∀w ∈ H1
0 (D)

18. Frances Kuo @ UNSW Sydney, Australia p.18
TUTORIAL: Differentiate the PDE (the uniform case)
5. Separate out the mmm = 000 term
D
a(xxx, yyy) ∇(∂ννν
u)(xxx, yyy) · ∇w(xxx) dxxx
= −
j≥1
νj
D
ψj(xxx) ∇(∂ννν−eeej u)(xxx, yyy) · ∇w(xxx) dxxx ∀w ∈ H1
0 (D)
6. Take w = (∂ννν
u)(·,yyy) and apply Cauchy-Schwarz
amin
D
|∇(∂ννν
u)(xxx, yyy)|2
dxxx
≤
j≥1
νj ψj L∞
D
|∇(∂ννν−eeej u)(xxx, yyy)|2
dxxx
1/2
D
|∇(∂ννν
u)(xxx, yyy)|2
dxxx
1/2
7. Cancel one common factor
∇(∂ννν
u)(·, yyy) L2 ≤
j≥1
νj bj ∇(∂ννν−eeej u)(·, yyy) L2 , bj =
ψj L∞
amin
8. Induction hypothesis
∇(∂ννν
u)(·, yyy) L2 ≤
j≥1
νj bj |ννν − eeej|! bbbννν−eeej
κ H-1
amin
= |ννν|! bbbννν κ H-1
amin

19. Frances Kuo @ UNSW Sydney, Australia p.19
Extensions
Lognormal a(xxx,yyy) = exp(Z(xxx,yyy))
circulant embedding [Graham, K., Nuyens, Scheichl, Sloan (in progress)]
Z(xxxi, yyy) for i = 1, . . . , M ⇒ M × M covariance matrix R
Assume stationary covariance function and regular grid
Extend R to an s × s circulant matrix Rext = AAT
Compute matrix-vector multiplication Ayyy using FFT
H-matrix [Feischl, K., Sloan (in review)]
Does not require stationary covariance function or regular grid
Approximate R by an H2
-matrix of interpolation order p, Rp = BBT
Iterative method to compute Byyy to some prescribed accuracy
QMC for PDE eigenvalue problems (neutron diffusion)
[Gilbert, Graham, K., Scheichl, Sloan (in progress)]
QMC for wave propagation models [Ganesh, K., Sloan (in progress)]
QMC for Bayesian inverse problems [several research groups]

20. Frances Kuo @ UNSW Sydney, Australia p.20
Overview
Motivating example
Introduction to QMC
Application
PDEs with random coefficients
uniform
lognormal
Setting 1
domain [0, 1]s
randomly shifted lattice rules
mixed first derivatives
first order convergence
Setting 2
domain Rs
randomly shifted lattice rules
mixed first derivatives
first order convergence
Setting 3
domain [0, 1]s
interlaced polynomial lattice rules
mixed higher derivatives
higher order convergence
KL expansion
circulant embedding
H-matrix
Cost reduction
Saving 1
Saving 2
Saving 3
Software

21. Frances Kuo @ UNSW Sydney, Australia p.21
Saving 1: multi-level and multi-index
Multi-level method [Heinrich (1998); Giles (2008); . . . ; Giles (2016)]
Telescoping sum
f =
∞
ℓ=0
(fℓ − fℓ−1), f−1 := 0 AML(f) =
L
ℓ=0
Qℓ(fℓ − fℓ−1)
|I(f) − AML(f)| ≤ |I(f) − IL(fL)|
≤ 1
2
ε
+
L
ℓ=0
|(Iℓ − Qℓ)(fℓ − fℓ−1)|
≤ 1
2
ε
choose L specify parameters for Qℓ
Successive differences fℓ − fℓ−1 become smaller
Require less samples for higher dimensional sub-problems
Lagrange multiplier to minimize cost subject to a given error threshold
Multi-index method [Haji-Ali, Nobile, Tempone (2016)]
Apply multi-level simultaneously to more than one aspect of the problem
e.g., quadrature, spatial discretization, time discretization, etc.
[Giles, K., Sloan (in review)]

22. Frances Kuo @ UNSW Sydney, Australia p.22
Saving 2: MDM
Multivariate decomposition method [. . . ; Wasilkowski (2013); . . . ]
[K., Nuyens, Plaskota, Sloan, Wasilkowski (2017)]
Decomposition
f =
u⊂{1,2,...},|u|<∞
fu AMDM(f) =
u∈A
Qu(fu)
|I(f) − AMDM(f)| ≤
u /∈A
|Iu(fu)|
≤ 1
2
ε
+
u∈A
|(Iu − Qu)(fu)|
≤ 1
2
ε
choose A specify parameters for Qu
Assume bounds on the norms fu Fu ≤ Bu are given
Qu can be QMC or other quadratures including Smolyak
Cardinality of active set A might be huge, but cardinality of u ∈ A is small
Implementation [Gilbert, K., Nuyens, Wasilkowski (in progress)]
Anchored decomposition fu(xxxu) = v⊆u(−1)|u|−|v|f(xxxv; aaa)
Explore nestedness/embedding in the quadratures
Efficient data structure to reuse function evaluations
Application to PDEs with random coefficients
[Gilbert, K., Nuyens, Nguyen, Sloan (in progress)]

23. Frances Kuo @ UNSW Sydney, Australia p.23
Saving 3: QMC fast matrix-vector mult.
QMC fast matrix-vector multiplication [Dick, K., Le Gia, Schwab (2015)]
Want to compute yyyiA for all i = 1, . . . , n
Row vectors yyyi = ϕ(ttti) where ttti are QMC points
Write Y ′
=



yyy1
...
yyyn−1


 = CP
C(n−1)×(n−1) is circulant
P has a single 1 in each column
and 0 everywhere else
Can compute Y ′
aaa in O(n log n) operations for any column aaa of A
Where does this appy?
Generating normally distributed points with a general covariance matrix
PDEs with uniform random coefficients
PDEs with lognormal random coefficients involving FE quadrature

24. Frances Kuo @ UNSW Sydney, Australia p.25
Software
http://people.cs.kuleuven.be/~dirk.nuyens/qmc4pde
Construction script in Python
Input: generic bound on mixed derivatives (weights are set automatically)
|∂ννν
F (yyy)| (|ννν| + a1)!
d1
s
j=1
(a2Bj)νj exp(a3Bj|yj|)
F (yyy) =



G(us
h) single-level
G(us
hℓ
− us
hℓ−1
) multi-level



a3 = 0 uniform case
a3 > 0 lognormal case
Output: generating vector for lattice sequence z.txt
## uniform case, 100-dim rule, 2ˆ10 points, with specified bounds b:
./lat-cbc.py --s=100 --m=10 --d2=3 --b=”0.1 * j**-3 / log(j+1)”
## lognormal case, 100-dim rule, 2ˆ10 points, with algebraic decay:
./lat-cbc.py --s=100 --m=10 --a2=”1/log(2)” --a3=1 --d2=3 --c=0.1
Output: generating matrices for interlaced polynomial lattice rule Bs53.col
## 100-dim rule, 2ˆ10 points, interlacing 3, with bounds from file:
./polylat-cbc.py --s=100 --m=10 --alpha=3 --a1=5 --b_file=bounds.txt

25. Frances Kuo @ UNSW Sydney, Australia p.26
Software
http://people.cs.kuleuven.be/~dirk.nuyens/qmc4pde
Point generators in Matlab/Octave (also C++ and Python)
Lattice sequences
load z.txt % load generating vector
latticeseq_b2(’init0’, z) % initialize the procedural generator
Pa = latticeseq_b2(20, 512); % first 512 20-dimensional points
Pb = latticeseq_b2(20, 512); % next 512 20-dimensional points
Interlaced polynomial lattice rules
load Bs53.col % load generating matrices
digitalseq_b2g(’init0’, Bs53) % initialize the procedural generator
Pa = digitalseq_b2g(100, 512); % first 512 100-dimensional points
Pb = digitalseq_b2g(100, 512); % next 512 100-dimensional points
Interlaced Sobol′
sequences
load sobol_alpha3_Bs53.col % load generating matrices
digitalseq_b2g(’init0’, sobol_alpha3_Bs53) % initialize the procedural gen
Pa = digitalseq_b2g(50, 512); % first 512 50-dimensional points
Pb = digitalseq_b2g(50, 512); % next 512 50-dimensional points

26. Frances Kuo @ UNSW Sydney, Australia p.27
Summary
Motivating example
Introduction to QMC
Application
PDEs with random coefficients
uniform
lognormal
Setting 1
domain [0, 1]s
randomly shifted lattice rules
mixed first derivatives
first order convergence
Setting 2
domain Rs
randomly shifted lattice rules
mixed first derivatives
first order convergence
Setting 3
domain [0, 1]s
interlaced polynomial lattice rules
mixed higher derivatives
higher order convergence
KL expansion
circulant embedding
H-matrix
Cost reduction
Saving 1 multi-level and multi-index methods
Saving 2 multivariate decomposition method (MDM)
Saving 3 QMC fast matrix-vector multiplication
Software