Generative Adversarial Networks for Anime Character Generation

1. #denatechcon #denatechcon

2. #denatechcon

3. #denatechcon

4. #denatechcon 00./ 0

6. #denatechcon

7. #denatechcon

8. #denatechcon // . /

10. #denatechcon

11. #denatechcon

12. #denatechcon

13. #denatechcon

14. #denatechcon

15. #denatechcon AGENDA

16. #denatechcon (@hamadakoichi) Mobage 2010 6 Mobage2011 4 DeNA DeNA2014 10 : ( ) TokyoWebmining - 1500 - 2010 60 DeNA 8 AI AI AI

17. #denatechcon 50 /

18. #denatechcon

19. #denatechcon , , , , , . “ ”. 232 (7/20/2017). 0 3

20. #denatechcon 5 4 /0 / 5 4 /0 : 6 / .

21. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCV Workshop 2018. (ECCV: European Conference on Computer Vision)

24. #denatechcon

25. #denatechcon 1 3 5 7 2 4 6 8

26. #denatechcon 1 3 5 7 2 4 6 8

27. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019) 1 3 5 7 2 4 6 8

28. #denatechcon ProgressiveGAN (Karras et al., ICLR 2018) BigGAN (Brock et al., ICLR 2019)

31. #denatechcon

32. #denatechcon Generative Adversarial Nets. Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde- Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio. arXiv:1406.2661. In NIPS 2014.

33. #denatechcon

36. #denatechcon Progressive Growing of GANs for Improved Quality, Stability, and Variation. Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen. In ICLR 2018. (1024X1024) (256x256)

37. #denatechcon .441 7 545 7 4 / Progressive Growing of GANs for Improved Quality, Stability, and Variation Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen. In ICLR 2018.

38. #denatechcon / 5. 44 5 Progressive Growing of GANs for Improved Quality, Stability, and Variation Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen. In ICLR 2018.

41. #denatechcon + Spectral Normalization on Generator + Self Attention + Two Time Scale Update Rule (512x512) + Spectral Normalization on Discriminator + Projection Discriminator SNGAN with Projection (Miyato+, ICLR’18) SAGAN (Zhang+, 18) BigGAN (Brock+, ICLR’19) + Large Batch Size (256→2048) + Large Channel (64→96) + Shared Embedding + Hierarchical Latent Space + Truncation Trick + Orthogonal Regularization + First Singular Value Clamp + Zero-centered Gradient Penalty Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2018.

42. #denatechcon (512x512) Generator Typical Architecture Res Block Architecture for ImageNet at 512x512 Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019.

43. #denatechcon (512x512) Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019.

44. #denatechcon Large Scale GAN Training for High Fidelity Natural Image Synthesis. Andrew Brock, Jeff Donahue, Karen Simonyan. arXiv:1809.11096. In ICLR 2019. (512x512)

54. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0

56. #denatechcon

57. #denatechcon

58. #denatechcon

59. #denatechcon

60. #denatechcon

61. #denatechcon

62. #denatechcon

63. #denatechcon

64. #denatechcon

65. #denatechcon

68. #denatechcon

71. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 0 0 0 0 0 0 0 0 0 0 0

72. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018.

74. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 0 0 0 0 0 0 0 0 0 0 0

80. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. 00./ 0 Full-body anime generation at 1024x1024 with Progressive Structure-conditional GANs

82. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. // . 0/0 Adding action to full-body anime characters with Progressive Structure-conditional GANs

84. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. (ICLR’18)

85. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. (ICLR’18)

86. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. (ICLR’18) (NIPS’17) (NIPS’17)

87. #denatechcon Full-body High-resolution Anime Generation with Progressive Structure-conditional Generative Adversarial Networks Koichi Hamada, Kentaro Tachibana, Tianqi Li, Hiroto Honda, and Yusuke Uchida. In ECCVW 2018. (ICLR’18) (NIPS’17) (NIPS’17)

89. #denatechcon

90. #denatechcon

92. #denatechcon n e n 4 3 33 36 4 n 0231 33 36 4 n c b a )3 ( 0 06I J@ N LT A F ( D T

93. #denatechcon n n n

94. #denatechcon n n

95. #denatechcon n C n 0 1 . n 0:: /1:0 6 ..:. 21 )()

96. #denatechcon n CEAD A n ) 17: n 7 2 -( ) 17: 0/ 1 7 0 7 . :0. 0 7 1 7 :

98. #denatechcon

99. #denatechcon : ( . ) / .

100. #denatechcon • • • • : ( . ) / .

101. #denatechcon

102. #denatechcon O .10 2:: ( / /:7 10 * .) A L A

103. #denatechcon O .10 2:: ( / /:7 10 * .) A L A

104. #denatechcon • O .10 2:: ( / /:7 10 * .) A L A

105. #denatechcon • • O .10 2:: ( / /:7 10 * .) A L A

106. #denatechcon O .10 2:: ( / /:7 10 * .) A L A • •

107. #denatechcon

108. #denatechcon ) ) 3

109. #denatechcon ) ) 3

110. #denatechcon

111. #denatechcon

112. #denatechcon

113. #denatechcon

114. #denatechcon 8 1 1 1 Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation. Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, Jan Kautz. In CVPR 2018.

115. #denatechcon /30 480 6 2/81 4C + 60 2 • 8 , Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation. Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, Jan Kautz. In CVPR 2018. https://youtu.be/MjViy6kyiqs Research at NVIDIA: Transforming Standard Video Into Slow Motion with AI

116. #denatechcon 7 1 Video Frame Synthesis using Deep Voxel Flow. Ziwei Liu, Raymond A. Yeh, Xiaoou Tang, Yiming Liu, Aseem Agarwala. In ICCV 2017.

117. #denatechcon N I 7 B7 =: B = P 77 = 7: :=D • /0 (+ /0 , Video Frame Synthesis using Deep Voxel Flow. Ziwei Liu, Raymond A. Yeh, Xiaoou Tang, Yiming Liu, Aseem Agarwala. In ICCV 2017. BB F=CBC 67 ?. / 3: 1 B Video Frame Synthesis using Deep Voxel Flow

118. #denatechcon D F 6 + 23C • 1 76 , P SV P J IOM S R • ( ,24 c SV P J ,24 cP / ++ C • 1 76 , P J SV P • 4 8 4 0 L a Super SloMo(Adobe) Super SloMo Deep Voxel Flow Video Frame Synthesis using Deep Voxel Flow. Ziwei Liu, Raymond A. Yeh, Xiaoou Tang, Yiming Liu, Aseem Agarwala. In ICCV 2017. Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation. Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, Jan Kautz. In CVPR 2018. F

119. #denatechcon 2

120. #denatechcon

121. #denatechcon F 8 0 O T PL n S 4 ) n F 4 ( W4000301 32 ( 6 A 7

122. #denatechcon F F F 0 -1 12 W n n S L ***7* 7 A A OT 0 -1 12

123. #denatechcon

124. #denatechcon

125. #denatechcon

126. #denatechcon n n

127. #denatechcon n n n

128. #denatechcon n n n n F

129. #denatechcon n n n n F

130. #denatechcon G I0 I4 Optical Flow I1, I2, I3 (Generated)

131. #denatechcon G I0 I4 D Optical Flow I1, I2, I3 (Generated) Ground Truth

132. #denatechcon G I0 I4 D Optical Flow I1, I2, I3 (Generated) Ground Truth MSE

133. #denatechcon G I0 I4 D Optical Flow Ground Truth I1, I2, I3 (Generated) Ground Truth Generated MSE MSE

134. #denatechcon G I0 I4 D Optical Flow Ground Truth I1, I2, I3 (Generated) Ground Truth Generated MSE MSE

135. #denatechcon 2

136. #denatechcon Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Local Discriminator “Real” or “Fake” Local Patch (16×16pix)

137. #denatechcon Conv-BN-ReLU Conv-BN-ReLU Generated Image Sequense Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU Conv-BN-ReLU FC Local Discriminator Temporal Discriminator “Real” or “Fake” Local Patch (16×16pix) Image Sequense “Real” or “Fake”

138. #denatechcon

139. #denatechcon Video

140. #denatechcon image0 image1 image2 image3 image4Video

141. #denatechcon ⁃ step size = 4 7FPS -> 30FPS 001.png, 005.png, 009.png, 013.png, 017.png ⁃ step size = 1 30FPS -> 120FPS 001.png, 002.png, 003.png, 004.png, 005.png

142. #denatechcon ⁃ step size = 4 7FPS -> 30FPS 001.png, 005.png, 009.png, 013.png, 017.png ⁃ step size = 1 30FPS -> 120FPS 001.png, 002.png, 003.png, 004.png, 005.png

143. #denatechcon

144. #denatechcon Frame Frame Deep Voxel FlowInput // . / Experimental Results: “Anime Frame Generation with Structure-consistent Prediction GANs”

145. #denatechcon step size = 1 step size = 4 step size = 7 step size = 10 Input SPGAN (Ours) Deep Voxel Flow 4 // . / Experimental Results: “Anime Frame Generation with Structure-consistent Prediction GANs”

146. #denatechcon Deep Voxel Flow Ours 1.average PSNR/SSIM on test dataset step size=4 PSNR SSIM Deep Voxel Flow 23.32 0.9294 SPGAN(Ours) 24.27 0.9407

155. #denatechcon

156. #denatechcon * 10 2 1

157. #denatechcon

158. #denatechcon L0*** * 2 D 1 A 7 1 D 1 7

159. #denatechcon TZ ... ./ 0 KD SL KA N O N KD SL K N W (3 Frame) AI (x16 ) Input Frames Generated Frames 78 : : 102*0DeNA AI :

160. #denatechcon

Generative Adversarial Networks for Anime Character Generation

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Generative Adversarial Networks for Anime Character Generation

Ähnlich wie Generative Adversarial Networks for Anime Character Generation (11)

Mehr von Koichi Hamada

Mehr von Koichi Hamada (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Generative Adversarial Networks for Anime Character Generation