PPT presented DPS(https://digitalproductschool.io/) week 8, at engineering weekly. Explaining how BERT model can be used in industry.

1. Applicable BERT
Jiwon Kim Engineering Weekly 8

2. How can we
apply SOTA NLP
model to the
specific domain?

3. Can BERT be used in a
sentiment classification?

4. 1. conda create -n bert python pytorch pandas
tqdm
conda install -c anaconda scikit-learn
(Just for safeness..)
1. pip install pytorch-pretrained-bert
(Thank you hugging face 🤗)
1. Prepare your data
2. Fine-tune the model with your data
Yes, of course
Just your target data, e.g., Yelp polarity data

6. Can BERT be used in a
Question-Answering?

7. Definitely
feat, pragnakalp
1. conda create -n bert python pytorch pandas
tqdm
conda install -c anaconda scikit-learn
(Just for safeness..)
1. pip install pytorch-pretrained-bert
(Thank you hugging face 🤗)
1. Prepare your data
2. Fine-tune the model with your data

8. Can BERT be used in a
chatbot?

9. 1. conda create -n bert python pytorch pandas tqdm
conda install -c anaconda scikit-learn
(Just for safeness..)
1. pip install pytorch-pretrained-bert
(Thank you hugging face 🤗)
1. Prepare your data
2. Fine-tune the model with your data
Again….

10. Definitely
feat, pragnakalp

11. Can BERT be used in
sentence auto-completion?

12. Gmail’s Suggested Replies, Smart Compose & Google Search Autocomplete
What is autocompletion?

13. Sure you can use BERT!
1. conda create -n bert python pytorch pandas
tqdm
conda install -c anaconda scikit-learn
(Just for safeness..)
1. pip install pytorch-pretrained-bert
(Thank you hugging face 🤗)
1. Prepare your data
2. Fine-tune the model with your data
feat, Hugging face

14. Why suddenly GPT-2???? 🤗🤗🤗🤗 🤗

15. € 40K

16. Review
BERT can be used as
● Sentiment(emotion) classification
● Question-Answering
● Chatbot
● Sentence auto-completion

17. SO……….
What is a BERT?

18. It would be way more long story…...
Concepts that you should know...
● Transformer, ELMo, GPT
● Transfer Learning from Supervised Data
● Embedding model(position embedding, segment embedding, token embedding)
● Pre-trained model
● Attention mechanism(e.g., Scaled Dot-Product Attention Multi-Head Attention)
● Semi-supervised sequence learning
● Position-wise Feed-Forward Networks
● And so on…...
Paper link 1
Paper link 2

19. Henry AI Labs

20. ● BERT can be used at least….
○ Sentiment(emotion) classification
○ Question-Answering
○ Chatbot
○ Sentence auto-completion
● Feel free to ask me if you can’t find out the pre-made code.
● “Don’t try to understand everything what you use… just run the
code, and don’t go deep on theory.” - Jeremy Howard
● Still you want to understand theoretical background, let’s study.
Conclusion