Generative Adversarial Nets

1. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Generative Adversarial Nets
JIN HO LEE
2019-3-28
JIN HO LEE Generative Adversarial Nets 2019-3-28 1 / 8

2. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
3. Adversarial nets
3. Adversarial nets
• Adversarial modeling framework 는 generator 와 discriminator 의
distribution 을 학습하게 된다.
• Generator’s distribution : input noise variables pz(z) 를 정의하고
거기에서 parameter θg 를 학습하여 G(z; θg) 를 업데이트 하는 방법으로
학습된다.
• Discriminator’s distribution : discriminator D(x; θd) 는 single scalar 의
값을 갖는 함수로 x 가 generator 의 distribution pz 에서 나오지 않았을 확률
(즉, input data 에서 나왔을 확률)을 의미한다.
• 이것을 한 번에 표시하면 아래의 min-max problem 으로 표시된 loss
function 을 통해 학습이 된다.
min
G
max
D
V(D, G) = Ex∼pdata
[log D(x)] + Ez∼pz(z)[log(1 − D(G(z)))].
JIN HO LEE Generative Adversarial Nets 2019-3-28 2 / 8

3. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4. Theoretical Results
4. Theoretical Results
JIN HO LEE Generative Adversarial Nets 2019-3-28 3 / 8

4. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4. Theoretical Results 4.1 Global Optimality of pg = pdata
4.1 Global Optimality of pg = pdata
Proposition
For G fixed, the optimal discriminator D is
D∗
G(x) =
pdata(x)
pdata(x) + pg(x)
Proof.
G 가 고정되면 식 (1) 에 의해 우리는 max
D
V(G, D) 문제를 풀게 된다. Law
of the unconscious statistician 에 의해 다음을 얻을 수 있다. 여기서
V(G, D) =
∫
x
pdata(x) log(D(x))dx +
∫
z
pz(z) log(1 − D(g(z)))dz
=
∫
x
(pdata(x) log(D(x)) + pg(x) log(1 − D(x))) dx
JIN HO LEE Generative Adversarial Nets 2019-3-28 4 / 8

5. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4. Theoretical Results 4.1 Global Optimality of pg = pdata
Proof.
임의의 (a, b) ∈ R2 {(0, 0)} 에 대해서 함수 f(y) = a log(y) + b log(1 − y)
는 f ′(y) = a
y − b
1−y 를 이용하면 y = a
a+b 에서 최대값을 갖는다.
위의 식에서는 a ≡ pdata(x), b ≡ pg(x) 를 대입하면
a
a + b
≡
pdata(x)
pdata(x) + pg(x)
를 얻을 수 있다.
JIN HO LEE Generative Adversarial Nets 2019-3-28 5 / 8

6. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4. Theoretical Results 4.1 Global Optimality of pg = pdata
For the optimal discriminator D∗
G(x) = pdata(x)
pdata(x)+pg(x), we have
C(G) = max
D
V(C, D)
= Ex∼pdata
[log D∗
G(x)] + Ez∼pz [log(1 − D∗
G(G(z)))]
= Ex∼pdata
[log D∗
G(x)] + Ez∼pg [log(1 − D∗
G(x))]
= Ex∼pdata
[
log
pdata(x)
pdata(x) + pg(x)
]
+ Ex∼pg
[
log
(
1 −
pdata(x)
pdata(x) + pg(x)
)]
= Ex∼pdata
[
log
pdata(x)
pdata(x) + pg(x)
]
+ Ex∼pg
[
log
pg(x)
pdata(x) + pg(x)
]
JIN HO LEE Generative Adversarial Nets 2019-3-28 6 / 8

7. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4. Theoretical Results 4.1 Global Optimality of pg = pdata
Theorem
The global minimum of the vortual training criterion C(G) is achieved iff
pg = pdata. At that point, C(G) achieves the value − log 4.
Proof.
If pg = pdata, then
D∗
G =
pdata(x)
pdata(x) + pg(x)
=
pg
pg + pg
=
1
2
.
C(G) = Ex∼pdata
[
log
pdata
pdata + pg
]
+ Ex∼pg
[
log
1
2
]
= log
1
2
+ log
1
2
= − log 4
JIN HO LEE Generative Adversarial Nets 2019-3-28 7 / 8

8. .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
4. Theoretical Results 4.1 Global Optimality of pg = pdata
C(G) = Ex∼pdata
[
log
pdata
pdata + pg
]
+ Ex∼pg
[
pg
pdata + pg
]
= KL(pdata||pdata + pg) + KL(pg||pdata + pg)
= −2 log 2 + KL
(
pdata||
pdata + pg
2
)
+ KL
(
pg||
pdata + pg
2
)
= − log 4 + 2JSD(pdata||pg)
≥ − log 4
JIN HO LEE Generative Adversarial Nets 2019-3-28 8 / 8