Breaking Down Review Articles

Getting started in a new area of research can seem like such a daunting task. Especially when the field has seen exponential growth in the number of papers published every year. This rapid growth can make it extremely difficult to keep up with current research practices. Higher turnover means there are fewer well-regarded standards as new work focuses on coming up with the next best idea rather than iterating on current methods. It can feel like a never-ending maze if you start indiscriminately reading paper after paper referenced in a widely cited paper.

So does this mean we are doomed to fail as soon as we start trying to keep up? No, not at all and especially not if you have a plan.

Review or survey articles are a great way to become oriented with a new field. They include historical information about the original problems in the field, along with past research efforts to try and solve them. Current methods are presented in a way to compare them with other methods in use. Having well-identified problem areas can help you figure out what you should focus your efforts on based on your situation or interests. Moreover, the best part of review articles is that they act as a well-curated list of references that you can use to learn more about the specifics of the method. Now I want you to walk you through how I break down review articles for both my Ph.D. research and my interests.

How I Go Through A Review Article

I recently wanted to know more about how current recommender systems integrate deep learning. In case you don’t know, recommender systems use past information about users and items to predict or recommend future user interactions with new items. For example, in a movie recommender, the movies would be the items, and individual people might be the users. Interactions for this problem vary depending on the format and your access to different kinds of data. It could be a rating system (1–5), it could be a :+1: or :-1:, or maybe it’s just whether or not the user has watched the movie before.

Finding Review Articles

Arxiv and Google Scholar are great places to look for review papers. Google Scholar can be tied to a university account so that any article you might have access to on a campus network, are available to you off-campus. Here, however, I only use Arxiv since any paper there is free for anyone reading this.

Pro tip: if you find an excellent scientific paper that you can’t get access to, reach out to the authors through email, and they will most likely send you a copy.

After searching Arxiv, I decided to choose the paper, “ Deep Learning Based Recommender System: A Survey and New Perspectives .” There were a couple of reasons that I ended up choosing this article.

1. It is a recent paper. With the latest version released in July 2019, it should contain most of the recent advancements in the field.
2. The review is published in ACM . Note, just because a paper is published does not mean that it is good. In this case, the peer-review process should help ensure that no significant contributions are missing.
3. The layout of the paper. The authors broke up the work into clear sections making it easier to parse and understand how all of the reviewed articles fit in the larger picture.

Breaking it Down

All right, now that we have a review article, we can start breaking it down piece by piece. I typically try and break the paper into different sections as a way to organize information. This is especially useful when trying to combine the ideas from multiple articles and for having a nice summary for future reference.

Reason for Writing

Some review articles provide their motivation for writing the review article. Their motivation is a great way to determine the scope of the research they hope to cover as well as find out about other articles that might be of interest to you.

In the last few years, a number of surveys in traditional recommender systems have been presented … However, there is a lack of extensive review on deep learning based recommender system . To the extent of our knowledge, only two related short surveys [7, 97] are formally published … Given the rising popularity and potential of deep learning applied in recommender system, a systematic survey will be of high scientific and practical values. We analyzed these works from different perspectives and presented some new insights toward this area. To this end, over 100 studies were shortlisted and classified in this survey [emphasis mine]

With just this one short excerpt, we know that the authors have written this paper to be an extensive systematic survey of over 100 articles on the use of deep learning in recommender systems. Also, it aims to provide a different perspective than the two previous review papers in the field.

Definition

It is always good to check and see if the authors give a detailed description of the topic that they cover in the review. Explicit definitions can help to clear up any possible misunderstandings that are a result of non-standard terminology in the field.

Recommender systems estimate users’ preference on items and recommend items that users might like to them proactively [1, 121]. Recommendation models are usually classified into three categories [1, 69]: collaborative filtering , content based and hybrid recommender system.

Now we can categorize each recommender system reviewed in the paper into one of three different categories. They then go on to define each of the categories.

Collaborative filteringmakes recommendations by l earning from user-item historical interactions , either explicit (e.g. user’s previous ratings) or implicit feedback (e.g. browsing history). Content-based recommendation is based primarily on c omparisons across items’ and users’ auxiliary information . A diverse range of auxiliary information such as texts, images and videos can be taken into account. Hybrid model refers to recommender system that integrates two or more types of recommendation strategies [8, 69].

A flowchart showing the relationship between types of recommender systems.

So, a recommender system that uses past information about how users interact with the items is collaborative filtering. Systems that use information about the users or items to determine the similarity of different items or users and then bases recommendations off that similarity are content-based recommenders. Lastly, if the recommender combines different methods, it is a hybrid system.

Applications and Data

Determining the kinds of problems that the reviewed methods aim to answer is essential for understanding both the current and future research directions. With recommender systems, we want to know the purpose of current systems and the kind of data needed for each system.

In industry, recommender systems are critical tools to enhance user experience and promote sales/services for many online websites and mobile applications [20, 27, 30, 43, 113]. For example, 80 percent of movies watched on Netflix came from recommendations [43], 60 percent of video clicks came from home page recommendation in YouTube [30] … Covington et al. [27] presented a deep neural network based recommendation algorithm for video recommendation on YouTube . Cheng et al. [20] proposed an App recommender system for Google Play with a wide & deep model. Shumpei et al. [113] presented a RNN based news recommender system for Yahoo News.

For each of these applications, we can envision there are different sources and formats of the data that is needed. Thankfully the authors include a table that breaks down the reviewed work into categories for each of the sources. Now, instead of having to look through all of the 100 reviewed papers to see if they use video as an input source, we only have four articles to read.

This table shows how the authors categorize publications based on the data source and task.

Future Research and Open Issues

In the final section of the paper, the authors discuss open issues in the field and paths for future research. If you are someone who is just getting into the field, this can be a great place to get ideas about future projects and the direction of your work.

1. Joint Learning from Users and Items
2. Explainability
3. Going Deeper
4. Machine Reasoning
5. Cross-Domain
6. Multi-Task Learning
7. Scalability
8. Better, More Unified, and Harder Evaluations

Any of these would be a great place to start working on a new project after you feel comfortable.

Putting It All Together

After going through the paper, we have a better understanding of the what, why, and where of recommender systems. Sadly at this point, we still aren’t anywhere close to being experts in the field. However, we can start to see how the research in the field is categorized and what papers we can read to get more info.

I hope this has been a useful exercise to see how review papers can be broken down to help get a better understanding of new areas of research. While this is certainly not the only way to review papers, these are some of the techniques I have found helpful for me.

Leave a comment below if you have questions or you want to share some of the techniques that you use for reviewing papers.