This document provides random thoughts on implementing machine learning papers. It discusses what types of papers to implement, including computer vision, NLP, reinforcement learning and more. It recommends specific papers and code repositories. It also discusses whether to use TensorFlow or PyTorch and mentions grants and competitions for implementing papers.

1. Random thoughts on
Paper Implementation
Taehoon Kim / carpedm20

2. Taehoon Kim / carpedm20 /

3. Paper Implementation

4. Release of
2015.11.08

5. Sukhbaatar, Sainbayar, Jason Weston, and Rob Fergus. "End-to-end memory networks." Advances in neural information processing systems. 2015.

6. Sukhbaatar, Sainbayar, Jason Weston, and Rob Fergus. "End-to-end memory networks." Advances in neural information processing systems. 2015.

7. Sukhbaatar, Sainbayar, Jason Weston, and Rob Fergus. "End-to-end memory networks." Advances in neural information processing systems. 2015.

8. https://github.com/facebook/MemNN/tree/mater/MemN2N-lang-model

9. https://github.com/carpedm20/MemN2N-tensorflow

10. Lua Torch →

11. Lua Torch →
Simple Translation

12. →
https://github.com/carpedm20/MemN2N-tensorflow/blob/master/model.py
Lua Torch 0.5.0
Translation

13. 20+ implementations
900 days
https://carpedm20.github.io/#personal-projects

14. 13,000+

15. Received questions about implementation

16. 4 random thoughts

17. 1. What to implement
2. What to read
3. TensorFlow or PyTorch?
4. Grants and Competitions

18. 1. What to implement

19. Computer Vision
Natural Language Processing
Reinforcement Learning
Semi/Un-supervised Learning
…

20. Computer Vision
Natural Language Processing
Reinforcement Learning
Semi/Un-supervised Learning
…

21. GAN
Generative Adversarial Network

22. Too many GANs..
Relatively easier to implement many but not much of a difference

23. Reasoning
Question Answering
Controllable Text Generation
Unsupervised Embedding
Exploration Without Reward
Neural Programmers
Neural Architecture Search
Hierarchical Representation
Learning
Video Generation
Program Induction
Self-Play Learning
Adversarial Attack
Multi-agent RL
Model-based RL
Meta Learning
Learned Data Augmentation
Speech Synthesis
Music Generation

24. Meta (Reinforcement) Learning
Transfer Learning

25. https://blog.openai.com/evolved-policy-gradients/
Adaptive learning from prior experience
with Evolution Strategies (ES)

26. https://blog.openai.com/evolved-policy-gradients/
Easiest way : learning from pretrained network
But how can we do better?

27. https://blog.openai.com/evolved-policy-gradients/
𝜃
Loss : 𝑓 𝑥; 𝜃 − 𝑦 (
Neural Network : 𝑓 𝑥; 𝜃

28. https://blog.openai.com/evolved-policy-gradients/
𝜃𝜙
Loss : 𝐿 𝑥, 𝑦; 𝜙
Neural Network : 𝑓 𝑥; 𝜃

29. https://blog.openai.com/evolved-policy-gradients/
𝜃𝜙

30. https://blog.openai.com/evolved-policy-gradients/

31. https://blog.openai.com/evolved-policy-gradients/
Evolved Policy Gradient Policy Gradient

32. Ganin, Yaroslav, et al. "Synthesizing Programs for Images using Reinforced Adversarial Learning." arXiv preprint arXiv:1804.01118 (2018).
Distributed RL + GAN

33. Ganin, Yaroslav, et al. "Synthesizing Programs for Images using Reinforced Adversarial Learning." arXiv preprint arXiv:1804.01118 (2018).

34. Ganin, Yaroslav, et al. "Synthesizing Programs for Images using Reinforced Adversarial Learning." arXiv preprint arXiv:1804.01118 (2018).

35. https://blog.openai.com/ingredients-for-robotics-research/

36. https://blog.openai.com/robots-that-learn/

37. https://blog.openai.com/competitive-self-play/

38. 2. What to Read

39. “How to implement a paper FASTER?”

40. Read others code a lot

41. Recommendations
• NLP: End-to-End Memory Network
• https://github.com/carpedm20/MemN2N-tensorflow
• Vision: (Fast) Style Transfer
• https://github.com/lengstrom/fast-style-transfer
• RL: Asynchronous Methods for Deep Reinforcement Learning
• https://github.com/openai/universe-starter-agent (TBH code is dirty but there are lots of things to learn)
• https://github.com/ikostrikov/pytorch-a3c
• Etc: Neural Turing Machine
• https://github.com/loudinthecloud/pytorch-ntm

42. More (nerdy) recommendations
• from tensorflow.contrib.seq2seq import Helper:
• https://github.com/keithito/tacotron/blob/master/models/helpers.py
• tf.while_loop:
• https://github.com/melodyguan/enas/blob/master/src/ptb/ptb_enas_child.py
• import tensorflow.contrib.graph_editor as ge:
• https://github.com/openai/gradient-checkpointing/blob/master/memory_saving_gradients.py
• Google’s production level code example:
• https://github.com/tensorflow/tensor2tensor
• The Annotated Transformer
• http://nlp.seas.harvard.edu/2018/04/03/attention.html
• Relatively safe-to-read codes:
• https://github.com/tensorflow/models

43. Two useful code snippets

44. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/train.py

45. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/train.py

46. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/train.py

48. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/image.py

49. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/image.py

50. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/image.py
64 batch image → 8 × 8 image grid summary

51. https://github.com/carpedm20/SPIRAL-tensorflow/blob/master/utils/image.py
64 batch image → 8 × 8 image grid summary

52. Read more recent and correct code
Never read carpedm20’s code (no joke. BAD examples)

53. 3. TensorFlow or PyTorch?

54. or

55. or
Mom
Dad

56. • Dirty long code
• More stressful debugging
• Faster development (1.5.0: Jan. 18, 1.6.0-1.7.0: Mar. 18, 1.8.0: Apr. 18, possibly more reliable)
• Easier (partial) save and load models (tf.Supervisor)
• Harder dynamic computation (tf.fold)
• XLA, TensorBoard, TPU, tf.eager, Multi-node distribution, Documentation

57. … compiling parts of the computational graph with XLA (a TensorFlow Just-In-Time compiler) and
…
Espeholt, Lasse, et al. "IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures." arXiv preprint arXiv:1802.01561 (2018).

58. tensorboard --port 6006 --debugger_port 6064
https://github.com/tensorflow/tensorboard/tree/master/tensorboard/plugins/debugger

59. https://github.com/tensorflow/tensorboard/tree/master/tensorboard/plugins/debugger

60. https://github.com/tensorflow/tensorboard/tree/master/tensorboard/plugins/debugger

61. https://twitter.com/jordannovet/status/977645002200252416

62. • Clean short code
• Less stressful debugging
• Slower development (0.2.0: Aug. 17, 0.3.0: Dec. 17)
• Dirty (partial) save and load models
• Easier dynamic computation

63. 4. Grants and Competitions

64. This is not about

66. https://blog.openai.com/openai-scholars/

67. https://blog.openai.com/openai-scholars/

68. Open Challenges

70. https://blog.openai.com/retro-contest/

71. https://contest.openai.com/

72. Workshop Challenges
NIPS, ICML, ICLR

73. http://www.cs.mcgill.ca/~jpineau/ICLR2018-ReproducibilityChallenge.html

74. http://www.cs.mcgill.ca/~jpineau/ICLR2018-ReproducibilityChallenge.html

75. https://openreview.net/forum?id=r1dHXnH6-
Feedback from Authors

76. https://arxiv.org/abs/1802.03198

78. So,

79. Read a paper

80. Implement a code

81. Seize the day :)
Great opportunities await you

82. One more thing...

84. 2015.06 ~

85. https://www.slideshare.net/carpedm20/ai-67616630

87. Teaching Machines to Understand Visual Manuals
via Attention Supervision for Object Assembly

88. Teaching Machines to Understand Visual Manuals via Attention Supervision for Object Assembly (work in progress)

89. https://www.slideshare.net/carpedm20/deview-2017-80824162

90. https://carpedm20.github.io/tacotron/
Person A

91. https://carpedm20.github.io/tacotron/
Person A

92. https://carpedm20.github.io/tacotron/
Person A

93. https://carpedm20.github.io/tacotron/
Person B

94. https://carpedm20.github.io/tacotron/
Person B

95. https://carpedm20.github.io/tacotron/
Person B

96. http://www.devsisters.com/jobs/

97. Thank you
@carpedm20