73

GitHub - graykode/nlp-tutorial: Natural Language Processing Tutorial for Deep Le...

5 years ago

source link: https://github.com/graykode/nlp-tutorial
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

README.md

nlp-tutorial

nlp-tutorial is a tutorial for who is studying NLP(Natural Language Processing) using TensorFlow and Pytorch. Most of the models in NLP were implemented with less than 100 lines of code.(except comments or blank lines)

Curriculum - (Example Purpose)

1. Basic Embedding Model

1-1. NNLM(Neural Network Language Model) - Predict Next Word
- Paper - A Neural Probabilistic Language Model(2003)
1-2. Word2Vec(Skip-gram) - Embedding Words and Show Graph
- Paper - Distributed Representations of Words and Phrases and their Compositionality(2013)
1-3. FastText(Application Level) - Sentence Classification
- Site : http://fasttext.cc
- Paper : Bag of Tricks for Efficient Text Classification(2016)

2. CNN(Convolutional Neural Network)

2-1. TextCNN - Binary Sentiment Classification
- Paper - Convolutional Neural Networks for Sentence Classification(2014)
2-2. DCNN(Dynamic Convolutional Neural Network)

3. RNN(Recurrent Neural Network)

3-1. TextRNN - Predict Next Step
- Paper - Finding Structure in Time(1990)
3-2. TextLSTM - Autocomplete
- Paper - LONG SHORT-TERM MEMORY(1997)
3-3. Bi-LSTM - Predict Next Word in Long Sentence

4. Attention Mechanism

4-1. Sequence2Sequence - Change Word
- Paper - Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation(2014)
4-2. Seq2Seq with Attention - Translate
- Paper - Neural Machine Translation by Jointly Learning to Align and Translate(2014)
4-3. Bi-LSTM with Attention - Binary Sentiment Classification

5. Model based on Transformer

5-1. The Transformer - Translate
- Paper - Attention Is All You Need(2017)
5-2. BERT - Classification Next Sentence & Predict Masked Tokens
- Paper - BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding(2018)

Depencencies

Python 3.5+
Tensorflow 1.12.0+
Pytorch 0.4.1+
Plan to add Keras Version

Author

Tae Hwan Jung(Jeff Jung) @graykode
Email : [email protected]

Recommend

About Joyk

Aggregate valuable and interesting links.
Joyk means Joy of geeK