Convolutional Neural Network in Natural Language Processing

What Convolutional Neural Network is and How to Utilize it for Sentiment Analysis?

What is Convolutional Neural Network?

Neural networks are a set of algorithms designed to recognize patterns. These patterns are numbers contained in vectors that are translated from real-world data such as images, sound, text or time series. A convolutional neural network is a neural network that applies convolutional layers to local features.

To understand how a small kernel can transform the large input data, see the gif below.

Retrieved from Stack Exchange

As we can see, every kernel is small spatially (along width and height), but extends through the full depth of the input volume and transform the input data as it slides.

But what if we want to different outputs? No problem. We just need to apply different filters.

Explore CNN in Setosa.io

These filters can also be applied in multi-dimensional output. The below image is input in 3D, with the image of size 7x7 and the last dimension represents the 3 channels of color (red, blue, green). This makes the input size 7x7x3. Then we apply 2 filters w0 and w1, each with size 3x3x3 to match the input size. What should we expect for the dimension of the output? 3x3x2 with 2 the number of filters.

Retrieved from gitbook

Convolutional Neural Network in Natural Language Processing

So we understand what convolutional neural network is and get an idea of how CNN can be applied to images. But how does CNN really work in NLP? For example, if we have a sentence “I love my new iphone” how can we use CNN to classify if this sentence is negative, positive, or neutral?

Image retrieved from Stack Exchange

Short explanations of the image above from left to right:

• The inputs are words. Each word is represented by a vector of size 7.
• Apply 4 different filters on the word vectors to create convolutional feature map
• Choose the maximum value of the result from each filter vector for pooled representation
• Apply softmax to transform a vector of size 1x4 to a vector of size 1x3 for classification

Learn by Example: Sentiment Analysis with PyTorch

PyTorch is a library for Python programs that facilitates building deep learning projects. In case you don’t know about PyTorch, check my article on the topic:

If you don’t have GPU in your machine, I encourage you to use Google Colab to try out with the codes. We will use this library to perform sentiment analysis on Kera’s IMDb dataset of movie reviews. Our task is to classify whether the review is positive or negative.

To build model, we use 2D convolution with nn.Conv2d(in_channels, out_channels, kernel_size) and a layer of linear neural networks for classification with nn.Linear(in_channels, out_channels).

Training steps

Visualize our loss function

import matplotlib.pyplot as pltplt.plot(LOSS)

print("F1_test: %.5f"%(get_f1(X_test, y_test)))

Awesome! Our CNN model gives us an F1 score of 0.87!

Conclusion

Congratulations! You have learned what convolutional neural network is and how to apply for natural language processing with PyTorch. I hope this gives you a general understanding of CNN and the motivation to utilize this method for your deep learning project. In case you want to gain a better understanding of CNN, t his website provides a cool interactive visualization for how the images change when applying with filters of CNN. You can try out the code of this article here .

I like to write about basic data science concepts and play with different data science tools. Follow me on Medium to get updated about my latest articles. You could also connect with me on LinkedIn and Twitter .

Check out my other blogs on data science topics: