Weitere ähnliche Inhalte Ähnlich wie NeuralArt 電腦作畫 (20) Kürzlich hochgeladen (20) NeuralArt 電腦作畫2. A
Neural
Algorithm
of
Ar7s7c
Style
• 作者:
– Leon
A.
Gatys.
– Alexander
S.
Ecker.
– MaAhias
Bethge
• 所屬單位:
– Werner
Reichardt
Centre
for
Integra7ve
Neuroscience
and
Ins7tute
of
Theore7cal
Physics,
University
of
Tubingen,
Germany.
– Bernstein
Center
for
Computa7onal
Neuroscience,
Tubingen,
Germany.
6. 神經元
• Neuron
• Ac7on
Poten7al
Dendrite
Axon
Cell
Body
Time
Voltage
Threshold
10. 電腦視覺
• Neural
Networks
• Convolu7onal
Neural
Networks
• VGG
19
11. Neural
Networks
n
W1
W2
x1
x2
b
Wb
y
nin = w1x1 + w2x2 + wb
nout =
1
1 + e nin
Sigmoid
Rec7fied
Linear
nout =
⇢
nin if nin > 0
0 otherwise
12. Neural
Networks
x
y
n11
n12
n21
n22
b b
z1
z2
Input
Layer
Hidden
Layer
Output
Layer
W12,y
W12,x
W11,y
W11,b
W12,b
W11,x
W21,11
W22,12
W21,12
W22,11
W21,b
W22,b
15. Convolu7onal
Neural
Networks
• Pooling
Layer
1
3
2
4
5
7
6
8
0
0
4
4
6
6
0
0
4
5
3
2
no
overlap
no
padding
no
weights
depth
=
1
7
8
6
4
Maximum
Pooling
Average
Pooling
16. Convolu7onal
Neural
Networks
Convolu7onal
Layer
Convolu7onal
Layer
Pooling
Layer
Pooling
Layer
Recep7ve
Fields
Recep7ve
Fields
Input
Layer
17. Convolu7onal
Neural
Networks
Input
Layer
Convolu7onal
Layer
with
Recep7ve
Fields:
Max-‐pooling
Layer
with
Width
=3,
Height
=
3
Filter
Responses
Filter
Responses
Input
Image
18. VGG
19
• Karen
Simonyan
&
Andrew
Zisserman.
Very
Deep
Convolu7onal
Networks
for
Large-‐scale
Image
Recogni7on.
• ImageNet
Challenge
2014
• 19
(+5)
layers
– 16
Convolu7onal
layers
(width=3,
height=3)
– 5
Max-‐pooling
layers
(width=2,
height=2)
– 3
Fully-‐connected
layers
19. VGG
19
depth=64
conv1_1
conv1_2
maxpool
depth=128
conv2_1
conv2_2
maxpool
depth=256
conv3_1
conv3_2
conv3_3
conv3_4
depth=512
conv4_1
conv4_2
conv4_3
conv4_4
depth=512
conv5_1
conv5_2
conv5_3
conv5_4
maxpool
maxpool
maxpool
size=4096
FC1
FC2
size=1000
sogmax
22. 內容產生
計算
兩者
差異
Filter
Responses
VGG19
修
正
差
異
相片
畫布
修正後
Width*Height
Depth
23. 內容產生
Layer
l’s
Filter
l
Responses:
ent(Pl
, Cl
) =
1
2
X
i,j
(Cl
i,j Pl
i,j)2
Layer
l’s
Filter
Responses:
Lcontent(P l
, Cl
) =
1
2
X
i,j
(Cl
i,j P l
i,j)2
Input
Photo:
Lcontent(p, c, l) =
1
2
X
i,j
(Cl
i,j Pl
i,j)2
Lcontent(p, x, l) =
1
2
X
i,j
(Xl
i,j Pl
i,j)2
@Lcontent(p, x, l)
@Xl
i,j
= Xl
i,j Pl
i,j
Xl
Xl
i,j
Input
Canvas:
x
Width*Height
(j)
Depth
(i)
Width*Height
(j)
Depth
(i)
24. 內容產生
• Backward
Propaga7on
Layer
l’s
Filter
l
Responses:
Xl
Input
Canvas:
x
VGG19
@Lcontent
@x
=
@Lcontent
@Xl
@Xl
x
x x ⌘
@Lcontent
@x
Update
Canvas
Learning
Rate
28. 畫風產生
VGG19
畫作
G
G
Filter
Responses
Gram
Matrix
Width*Height
Depth
Depth
Depth
Posi7on-‐
dependent
Posi7on-‐
independent
29. 畫風產生
1.
.5
.5
.5
1.
1.
.5
.25
1.
.5
.25
.5
.25
.25
1.
.5
1.
Width*Height
Depth
k1
k2
k1
k2
Depth
Depth
Layer
l’s
Filter
Responses
Gram
Matrix
Fl
1
Fl
2
Fl
3
Fl
4
Fl
1
Fl
2
Fl
3
Fl
4
G
Gl
i,j = Fl
i · Fl
j
Gl
4,1 = Fl
4 · Fl
1
= 1 ⇥ 1 + 0 ⇥ 0.5 + 0 ⇥ 0 + ...
= 1
31. 畫風產生
Lstyle(a, x, l) =
1
2
X
i,j
(Xl
i,j Al
i,j)2
@Lstyle(a, x, l)
@Fl
i,j
= ((Fl
)T
(Xl
Al
))j,i
Layer
l’s
Filter
Responses
Layer
l’s
Gram
Matrix
Layer
l’s
Gram
Matrix
Fl
i,j
Al
i,j Xl
i,j
Input
Artwork:
Input
Canvas:
a x
33. 畫風產生
VGG19
Conv1_1
Conv1_1
Conv2_1
Conv1_1
Conv2_1
Conv3_1
Conv1_1
Conv2_1
Conv3_1
Conv4_1
Conv1_1
Conv2_1
Conv3_1
Conv4_1
Conv5_1
34. 作品產生
Ltotal = ↵Lcontent + Lstyle
a
ent(p, c, l) =
1
2
X
i,j
(Cl
i,j Pl
i,j)2
x
x x ⌘
@Ltotal
@x
x
Filter
Responses
VGG19
Lcontent(p, x)
Lstyle(a, x)
Gram
Matrix
35. 作品產生
VGG19
VGG19
Lcontent(p, x) Lstyle(a, x)
Conv1_1
Conv2_1
Conv3_1
Conv4_1
Conv5_1
Conv4_2
Ltotal = ↵Lcontent + Lstyle
37. 作品展示
• 內容 v.s.
畫風
• 不同起始狀態
• 不同VGG
Layers
• 素描、水彩
• 詩中有畫、畫中有詩
40. 不同VGG
Layers
Conv1_1
Conv2_1
Conv1_1
Conv2_1
Conv3_1
Conv1_1
Conv2_1
Conv3_1
Conv4_1
Conv1_1
Conv2_1
Conv3_1
Conv4_1
Conv5_1
↵
= 0.002
43. 延伸閱讀
• A
Neural
Algorithm
of
Ar7s7c
Style
– hAp://arxiv.org/abs/1508.06576
• Texture
Synthesis
Using
Convolu7onal
Neural
Networks
– hAp://arxiv.org/abs/1505.07376
• Convolu7onal
Neural
Network
– hAp://cs231n.github.io/convolu7onal-‐networks/
• Neural
Network
Back
Propaga7on
– hAp://cpmarkchang.logdown.com/posts/277349-‐neural-‐
network-‐backward-‐propaga7on
• 電腦賦詩:
– hAp://www.slideshare.net/ckmarkohchang/computa7onal-‐
poetry
44. 程式碼
• Python
Tensorflow
– hAps://github.com/ckmarkoh/neuralart_tensorflow
• Python
Theano
– hAps://github.com/woonketwong/ar7fy
• Python
Theano
(ipython
notebook)
– hAps://github.com/Lasagne/Recipes/blob/master/
examples/styletransfer/Art%20Style
%20Transfer.ipynb
• Python
deeppy
– hAps://github.com/andersbll/neural_ar7s7c_style
Hinweis der Redaktion n_{in} = w_{1} x_{1} + w_{2}x_{2}+w_{b}
n_{out} = \frac{1}{1+e^{-n_{in}}}
L_{content}(\textbf{p},\textbf{x}, l) = \frac{1}{2} \sum_{i,j}(X^{l}_{i,j}-P^{l}_{i,j})^2
\dfrac{\partial L_{content}(\textbf{p},\textbf{x},l) }{\partial X^{l}_{i,j}} = X^{l}_{i,j}-P^{l}_{i,j}
X_^{l} \textbf{x} \leftarrow \textbf{x} - \eta \dfrac{\partial L_{content}}{\partial \textbf{x}}
\textbf{x} \leftarrow \textbf{x} - \eta \dfrac{\partial L_{content}}{\partial \textbf{x}}
G^{l}_{i,j} = F^{l}_{i} \cdot F^{l}_{j}
& G^{l}_{4,1} = F^{l}_{4} \cdot F^{l}_{1} \\
& = 1 \times 1 + 0 \times 0.5 + 0\times 0 + ...\\
&= 1 A^{l}_{i,j}
L_{style}(\textbf{a},\textbf{x}, l) = \frac{1}{2} \sum_{i,j}(X^{l}_{i,j}-A^{l}_{i,j})^2
\dfrac{\partial L_{style}(\textbf{a},\textbf{x},l) }{\partial F^{l}_{i,j}} = ( (F^{l})^{T} (X^{l}-A^{l}) )_{j,i}
L_{style}
L_{total} = \alpha L_{content} + (1-\alpha) L_{style}
\dfrac{\partial L_{total}}{\partial \textbf{x}}
\textbf{x} \rightarrow \textbf{x} - \eta \dfrac{\partial L_{total}}{\partial \textbf{x}}
L_{style}(\textbf{a},\textbf{x})
L_{content}(\textbf{p},\textbf{x})