An overview of several recommendation systems

Collaborative filtering, KNN, Deep learning, transfer learning, Tfidf…ect explore all of these

In this article we will review several recommendation algorithms, evaluate through KPI and compare them in real time. We will see in order :

• a popularity based recommender
• a content based recommender (Through KNN, TFIDF, Transfert Learning)
• a user based recommender
• a hybrid recommender
• a deep learning recommender

N.B : I was greatly inspired by the great notebook of Gabriel Moreira thanks to him https://www.kaggle.com/gspmoreira/recommender-systems-in-python-101 some model or function like user profiler and evaluation function come from his notebook.

Introduction

I will introduce the databases and define what will allow us to evaluate our models. I have anime metadata that I modified a little bit to have more reliable content recommendations.

Then a user-item database with ratings

After joining the two, we will divide them into training and test sets in a stratified way

interactions_train_df, interactions_test_df = train_test_split(interactions_full_df,
                                   stratify=interactions_full_df['user_id'], 
                                   test_size=0.20,
                                   random_state=42)#Indexing by personId to speed up the searches during evaluation
interactions_full_indexed_df = interactions_full_df.set_index('user_id')
interactions_train_indexed_df = interactions_train_df.set_index('user_id')
interactions_test_indexed_df = interactions_test_df.set_index('user_id')

Now, we can define a evaluation function.

Evaluation function

We work with Top-N accuracy metrics , which evaluates the accuracy of the top recommendations provided to a user, comparing to the items the user has actually interacted in test set.

This evaluation method works as follows:

• For each user
• For each item the user has interacted in test set
• Sample 100 other items the user has never interacted.
  Ps. Here we naively assume those non interacted items are not relevant to the user, which might not be true, as the user may simply not be aware of those not interacted items. But let’s keep this assumption.
• Ask the recommender model to produce a ranked list of recommended items, from a set composed one interacted item and the 100 non-interacted (non-relevant) items
• Compute the Top-N accuracy metrics for this user and interacted item from the recommendations ranked list
• Aggregate the global Top-N accuracy metrics

The Top-N accuracy metric choosen was Recall@N which evaluates whether the interacted item is among the top N items (hit) in the ranked list of 101 recommendations for a user.

Popularity based model

A common (and usually hard-to-beat) baseline approach is the Popularity model. This model is not actually personalized it simply recommends to a user the most popular anime that the user has not previously consumed.

item_popularity_df = interactions_full_df.groupby('anime_id')['user_rating'].sum().sort_values(ascending=False).reset_index()

Here we perform the evaluation of the Popularity model, according to the method described above.

It achieved the Recall@5 of 0.59, which means that about 60 % of interacted items in test set were ranked by Popularity model among the top-5 items (from lists with 100 random items). And Recall@10 was even higher (77 % ), as expected.

This model is very good for the effort we put into it !

Content based model

Content-based filtering approaches leverage description or attributes from items the user has interacted to recommend similar items. It depends only on the user previous choices, making this method robust to avoid the cold-start problem. For textual items, like articles, news and books, it is simple to use the raw text to build item profiles and user profiles.

In order to have personalized recommendations from a content-based model we will implement the following steps :

for each user :

• get his watchlist

for each anime in his watchlist

• recovers the similarity of the anime compared to all other anime (thanks to a distances/features matrix) like tfidf one
• recovers every user-rating

finaly make a weighted average of items values we got from the content model (our distance matrix) with item strength from ratings.

Term Frequency Inverse Document Frequency (TFIDF)

We will using a very popular technique in information retrieval (search engines) named TF-IDF . This technique converts unstructured text into a vector structure, where each word is represented by a position in the vector, and the value measures how relevant a given word is for an article. As all items will be represented in the same Vector Space Model , it is to compute similarity between articles.

word_vectorizer = TfidfVectorizer(ngram_range =(1,4),
                             min_df=5, max_df=0.9,
                             strip_accents='unicode',
                             stop_words = 'english',
                             analyzer = 'word',
                             use_idf=1,
                             smooth_idf=1,
                             sublinear_tf=1)

tfidf_matrix = word_vectorizer.fit_transform(anime['soup'])
tfidf_feature_names = word_vectorizer.get_feature_names()

""" compute users profiles """
U2P = Users_Profiler(tfidf_matrix, anime_ids)
U2tfifd = U2P.build_users_profiles()

now thanks to our base recommender and user profiler class we can recommend anime to a specific user :

ContentBasedmodel = BaseRecommender(U2tfifd, tfidf_matrix, anime)ContentBasedmodel.recommend_anime(8)

Let’s evaluate it

print('Evaluating Content-Based Filtering model...')cb_global_metrics, cb_detailed_results_df = evaluate_model(ContentBasedmodel)print('\nGlobal metrics:\n%s' % cb_global_metrics)cb_detailed_results_df = cb_detailed_results_df[['_user_id', 'watched_count', "hits@3_count", 'hits@5_count','hits@10_count', 
                                                'recall@3','recall@5','recall@10']]
cb_detailed_results_df.head(10)

It achieved the Recall@5 of 0.755 , which means that about 75 % of interacted items in test set were ranked by Popularity model among the top-5 items (from lists with 100 random items). And Recall@10 was even higher (81 % ).

This content model seems very reliable.

K-nearest neighbor

Here we use the k-nearest-neighbor this is an algorithm that attempts to determine what group a data point is in by looking at the data points around it. This one is a bit different from the classification one because instead of predict which group each data point belonging he just return 6 nearest neighbors of each data point according the cosine distance metrics. From this we will extract the distances matrix and got user preference from it.

from sklearn.neighbors import NearestNeighbors
from scipy.sparse import csr_matrix
from sklearn.preprocessing import MaxAbsScaler
mabs = MaxAbsScaler()# build dummies matrix
anime_features = pd.concat([anime["genre"].str.get_dummies(sep=","),
                            pd.get_dummies(anime[["type"]]),
                            pd.get_dummies(anime[["episodes"]]),
                            pd.get_dummies(anime[["members"]]),anime_ids = anime["anime_id"].tolist()
anime_features_scale = mabs.fit_transform(anime_features)nbrs = NearestNeighbors(n_neighbors=6, algorithm='brute', metric = "cosine").fit(anime_features_scale)distances, indices = nbrs.kneighbors(anime_features_scale)
distances =  csr_matrix(distances)

U2KNN = Users_Profiler(distances, anime_ids)
KNN_model = U2KNN.build_users_profiles()KNN_RECO = BaseRecommender(KNN_model, distances, anime)
print('Evaluating Content-Based Filtering model...')
global_metrics, detailed_results_df = evaluate_model(KNN_RECO)
print('\nGlobal metrics:\n%s' % global_metrics)

The KNN model not seems produce usefull recommendations. It achieved the Recall@5 of 0.16, which means that about 16 % of interacted items only in test set were ranked by Popularity model among the top-5 items (from lists with 100 random items).

Transfert Learning (word embedding)

For that method we will rely on pretrained embedding matrix to create an embeddings vector. In the context of neural networks, embeddings are low-dimensional, learned continuous vector representations of discrete variables. They can reduce the dimensionality of categorical variables and meaningfully represent categories in the transformed space. Here, is purpose is to finding nearest neighbors in the embedding space. These can be used to make recommendations based on user interests.

xtrain_embeddings = sc.fit_transform(xtrain_embeddings)
xtrain_embeddings = csr_matrix(xtrain_embeddings)
U2f = Users_Profiler(xtrain_embeddings, anime_ids)
U2fastt = U2f.build_users_profiles()
fastcontent = BaseRecommender(U2fastt, xtrain_embeddings, anime)

we can now use it for recommendation and evaluate it.

print('Evaluating Content-Based Filtering model...')
fglobal_metrics, fdetailed_results_df = evaluate_model(fastcontent)
print('\nGlobal metrics:\n%s' % fglobal_metrics)
fdetailed_results_df = fdetailed_results_df[['_user_id', 'watched_count', "hits@3_count", 'hits@5_count','hits@10_count', 
                                                'recall@3','recall@5','recall@10']]
fdetailed_results_df.head(10)

Transfert Learning is better than KNN but has poor performance compared to TFIDF method with almost half of his performance with 37% on recall@5.

Collaborative Filtering

Collaborative filteringis a method of making automatic predictions ( filtering ) about the interests of a user by collecting preferences or taste information from many users.

Singular Value Decomposition (SVD)

Here, we a use popular latent factor model named Singular Value Decomposition (SVD) . Latent factor models compress user-item matrix into a low-dimensional representation in terms of latent factors. One advantage of using this approach is that instead of having a high dimensional matrix containing abundant number of missing values we will be dealing with a much smaller matrix in lower-dimensional space. More factors there’s more specific the model is, it may result some overfitting if there’s to much factors.

U, sigma, Vt = svds(R, k= 300)
sigma = np.diag(sigma)
predicted_ratings = np.dot(np.dot(U, sigma), Vt) 
cf_preds_df = pd.DataFrame(predicted_ratings, 
                           columns = pivpiv.columns, 
                           index=iddd).transpose()

It achieved the Recall@5 of 0.755 , which means that about 75 % of interacted items in test set were ranked by Popularity model among the top-5 items (from lists with 100 random items). And Recall@10 was even higher (81 % ).

Hybrid model

We will build a simple hybridization method, by only multiply the SVD score with the Content-Based score, and ranking by resulting score.

evaluate our hybrid tfidf/svd model :

Here, we have almost the same KPI for each recall@n 82% which is pretty good.

Deep learning

Finally, we will use deep learning to build a deep collaborative filtering model to combine the 2 concept we saw earlier, latent factor model and embeddings.

Thanks to that model we will predict the ratings of an unknown item for a users.

history = DeepCF.fit([train.user_id, train.anime_id], train.user_rating, batch_size=512, epochs=5, 
          validation_data=([valid.user_id, valid.anime_id], 
            valid.user_rating))

I didn’t evaluate it because the prediction method is very slow but don’t hesitate to do it.

We will look at the recommendations proposed by each model for a single user.

rated, compare = comparison(25, 5)

I didn’t show the “noted” corpus of this user because there are more than 100 articles but from experience except KNN most of these recommendations seem very reliable.

To conclude

We have seen several types of recommendation systems from the very simple and to more elaborate models, you are free to improve or enrich them. It is interesting to note that some very simple models like popularity can be reliable. I hope that one of these models will be useful to your project, if you want to see others exotic recommendations model, take a look at my other article like :

• image-recommendation-engine-with-keras : https://towardsdatascience.com/image-recommendation-engine-with-keras-d227b0996667
• Image recommendation engine — leverage transfer learning : https://towardsdatascience.com/image-recommendation-engine-leverage-transfert-learning-ec9af32f5239
• Anime2vec : A sequence recommender system : https://towardsdatascience.com/anime2vec-a-sequence-recommender-1e0a3e558c44

Thank you!

The codes are available here: https://github.com/AlexWarembourg/Medium/blob/master/a-panel-of-recommender.ipynb