Stock Market Action Prediction with ConvNet

Stock Buy/Sell Prediction Using ConvNets

Inspired from Research Paper titled ‘Algorithmic Financial Trading with Deep Convolutional Neural Networks: Time Series to Image Conversion’

This project is loosely based on a research paper titled “ Algorithmic Financial Trading with Deep Convolutional Neural Networks: Time Series to Image Conversion Approach ”. I say ‘loosely’ because although I have borrowed the core idea from the paper, there are many things that I have done (or had to do) different as we will see here. The link I have is a preprint. The paid/main paper may have more details. This paper was suggested by one of the readers of my previous article on stock price prediction and it immediately caught my attention. Here is the link to the Github repo .

There is one thing I would like the readers to know — I am not here to claim that I have a ready to use trading model (although I am further exploring this method for my personal use). The idea of converting a conventional tabular or time-series data to image, and training a model on it, just seemed too exciting to resist from trying it out and sharing it with the community. Like my previous article this is an account of my experience with the project.

1. What does the research paper say?

The idea is fairly simple: Calculate 15 technical indicators with 15 different period lengths (explained below) for each day in your trading data. Then convert the 225 (15*15) new features into a 15x15 images. Label the data as buy/sell/hold based the algorithm provided in the paper. Then train a Convolutional Neural Network like any other image classification problem.

Feature Engineering:If you are not aware of what a technical indicator is, I would suggest you check the link provided above. I would explain the concept of technical indicators and time period with a Simple Moving Average (SMA) since it’s simpler. This should be enough for you to understand the idea.

A moving average for a list of numbers is like arithmetic average but instead of calculating the average of all the numbers, we calculate the average of the first ’n’ numbers (n is referred as window size or time period) and then move (or slide) the window by 1 index, thus excluding the first element and including the n+1 element and calculate their average. This process continues. Here is an example to drive this point home:

SMA example on excel sheet

This is an example of SMA on window size of 6. The SMA of first 6 elements is shown in orange. Now consider the first column above as the close price of your chosen stock. Now calculate SMA on close price for 14 other window sizes (7 to 20) concatenated on right side of sma_6. Now you have 15 new features for each row of your dataset. Repeat this process for 14 other technical indicators and drop the null rows. Now you have 225 new features. If you reshape these numbers into a 15x15 array, you have an image! (Albeit, at this point, it’s a single channel. More on this later). There is one thing to keep in mind though. While constructing these images we should keep the related technical indicators spatially close. The intuition is, when training for human face recognition, you would not label a picture as human face if it has one eye below the nose. Related pixels should be close by. I am not posting the code to calculate all the indicators for brevity. You can find them in utils.py file.

Labeling:What’s left now is to label this dataset. For that, the authors used following algorithm:

Algorithm used to label the dataset as buy/sell/hold

At first glance, it may seem formidable, but all it says is this: use a window of 11 days on close price. If the middle number is maximum within the window, label the last (11th day) as ‘sell’ or if the middle number is minimum then label the last day as ‘buy’, else label as ‘hold’. Slide the window like explained earlier and repeat. The idea is to buy at troughs and sell at crests. The competency of this algorithm is a different matter and I will get into that toward the end.

Training:Authors have used rolling window training, which is similar to the sliding window concept we saw above. If you have stock history data for the year 2000 to 2019 and you decide to train on 5 years data and test on 1 year data then, slice the data for 2000–2004 from dataset for training and 2005 year’s data for testing. Train and test your model on this data. Next select 2001–2005 as training data and 2006 as test data. Use the same model to retrain on this data. Repeat until you reach the end.

Computational Performance Evaluation:Authors have provided two types of model evaluations in the paper, computational and financial evaluation. Computational evaluation includes confusion matrix, F1 score, class wise precision etc. Financial evaluation is done by applying the model prediction to real world trading and measure the profit made. I will only discuss the computational evaluation. Financial evaluation can be done by either real world trading or backtesting on held out data, which I will discuss in the future articles.

2. Implementation

As mentioned at the beginning of this article, I have not followed the research paper strictly. I will mention the differences as and when they come up.

Data Source:I usually get stock data from Alpha Vantage which provides historical stock data for free. I had used it for my previousproject as well.

url = "https://www.alphavantage.co/query?function=TIME_SERIES_DAILY_ADJUSTED&outputsize=full&apikey=api_key&datatype=csv&symbol=company_code"
urllib.request.urlretrieve(url, path_to_save)

Data looks like this:

Feature Engineering:The first deviation is the technical indicators I used. I couldn’t find library/implementation for some of the indicators that were mentioned in the paper, like PSI. Some indicators were just not clear; for example, PPO is calculated using EMA of period 12 and 26. How can we calculate PPO for different periods? I tried to use most of the indicators mentioned in the paper for which I found open source implementations to avoid any programming bugs. I have implemented some indicators like WMA, HMA, etc, although they are quite slow and need optimization. Since I have to run it only once and save the data, it’s not an issue for me. You can use different indicators of your choice though. They have also adjusted the prices (open, high, low etc) with adjust ratio. But I haven’t followed this one because I couldn’t find any reference on how to do that adjustment.

Labeling the data:For this blog, I have used the original labeling algorithm that the authors have used. Here is a direct implementation of it: