The document presents a unified perspective called probability functional descent (PFD) that views generative adversarial networks (GANs), variational inference, and reinforcement learning as optimizing probability functionals. PFD frames these domains as minimizing an objective function J over probability distributions using functional derivatives. This allows first-order descent algorithms to be generalized for probability optimization. The document shows how GANs, variational inference, and adversarial PFD are specific instantiations of PFD, providing a unified framework and enabling transfer of techniques between domains.

1. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Probability Functional Descent
A Unifying Perspective on GANs, Variational Inference,
and Reinforcement Learning
Casey Chu 1 Jose Blanchet 1 Peter Glynn 1
1Stanford University
presentation by yangjq, 2019.9
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 1 / 24

2. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Generative Adversarial Networks
Figure: images generated by progressive GAN
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 2 / 24

3. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Generative Adversarial Networks
GAN composes of two deep networks, the generator, and the
discriminator.
The generator:
Figure: how the generator works
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 3 / 24

4. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Generative Adversarial Networks
Figure: A generator in DCGAN
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 4 / 24

5. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Generative Adversarial Networks
The generator and discriminator act as adversarial.
Figure: typical training procedure of GAN
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 5 / 24

6. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Generative Adversarial Networks
Minimax GAN (Goodfellow et al., 2014)
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 6 / 24

7. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Generative Adversarial Networks
GAN is actually optimizing JS-divergence.
min
G
max
D
V (D, G) = Ex∼pr(x)[log D(x)] + Ez∼pz(z)[log(1 − D(G(z)))]
= Ex∼pr(x)[log D(x)] + Ex∼pg(x)[log(1 − D(x)]
=
∫
x
(pr(x) log D(x) + pg(x) log(1 − D(x))) dx
(1)
If G is fixed the optimal Discriminator D∗ is
D∗
(x) =
pr(x)
pr(x) + pg(x)
(2)
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 7 / 24

8. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Generative Adversarial Networks
min
G
V (D∗
, G) = 2DJS (pr∥pg) − 2 log 2 (3)
DJS(µ∥ν) =
1
2
DKL
(
µ∥
1
2
µ +
1
2
ν
)
+
1
2
DKL
(
ν∥
1
2
µ +
1
2
ν
)
(4)
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 8 / 24

9. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Variational Inference
The posterior distribution
p(z|x) =
p(x|z)p(z)
p(x)
=
p(x|z)p(z)
∫
p(x|z)p(z)dz
(5)
The posterior is difficult to compute due to the presence of the integral.
Variational inference turn this problem from an integral to sampling and
optimizing.
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 9 / 24

10. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Variational Inference
Variational inference therefore reframes this computation as an
optimization problem in which a variational posterior q(z) approximates
the true posterior by solving
inf
q
DKL(q(z)∥p(z|x)) (6)
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 10 / 24

11. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Variational Inference
Black-box variational inference
KL(q(z)∥p(z|x)) = E[log q(z)] − E[log p(z, x)] + log p(x) (7)
= log p(x) − Ez∼q(z)
[
log
p(x|z)p(z)
q(z)
]
ELBO
(8)
This leads to the following practical algorithm, namely stochastic gradient
descent on the objective
θ → −Ez∼qθ(z)
[
log
p(x|z)p(z)
qθ(z)
]
(9)
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 11 / 24

12. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Probability Functional Descent
The objective:
min
µ∈P(X)
J(µ) (10)
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 12 / 24

13. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Probability Functional Descent
Intuitively, the Gateaux differential is a generalization of the directional
derivative.
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 13 / 24

14. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Probability Functional Descent
dJµ(χ) = Ex∼ν [Ψµ(x)] − Ex∼µ [Ψµ(x)] (11)
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 14 / 24

15. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Probability Functional Descent
The Gateaux derivative and the influence function provide the proper
notion of a functional derivative, which allows us to generalize first-order
descent algorithms to apply to probability functionals such as J.
˜J(µ) = J (µ0) + dJµ0 (µ − µ0)
= J (µ0) + Ex∼µ [Ψµ0 (x)] − Ex∼µ0 [Ψµ0 (x)]
= constant + Ex∼µ [Ψµ0 (x)]
(12)
To optimize J we only need to optimize on Ex∼µ [Ψµ0 (x)]!!
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 15 / 24

16. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Probability Functional Descent
∇θJ (µθ) = ∇θEx∼µθ
[ˆΨ(x)] (13)
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 16 / 24

17. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Probability Functional Descent
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 17 / 24

18. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Probability Functional Descent
Adversarial PFD with approximation of Ψµ.
inf
µ
sup
φ
[Ex∼µ[φ(x)] − J⋆
(φ)] (14)
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 18 / 24

19. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
PFD and GAN
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 19 / 24

20. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
PFD and GAN
ΨJS = arg max
φ∈C(X)
[
Ex∼µ[φ(x)] +
1
2
Ex∼ν
[
log
(
1 − e2φ(x)+log 2
)]]
(15)
GAN update:



ϕ → −1
2Ex∼ν [log Dϕ(x)] − 1
2Ex∼µθ
[log (1 − Dϕ(x))]
θ → 1
2Ex∼µθ
[log (1 − Dϕ(x))]
(16)
So the influence function ΨJS(x) = 1
2 log (1 − D∗(x)) is approximated
using the learned discriminator.
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 20 / 24

21. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
PFD and Variational Inference
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 21 / 24

22. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Probability Functional Descent
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 22 / 24

23. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Probability Functional Descent
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 23 / 24

24. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Conclusion
1. The transfer of insight and specialized techniques from one domain to
another.
2. Simultaneous development of new algorithms for GANs, variational
inference, and reinforcement learning.
3. This paper unlocks the possibility of applying probability functional
descent to new problems.
Probability Functional Descent A Unifying Perspective on GANs, Variational Inference, and2019.5 24 / 24