Artificial intelligence is often used in recommendation systems, which are the foundation of the most popular VOD apps and platforms.

What are recommendation systems?

The key to successful sales of products and services lies in close scrutiny of customers, especially in relation to their reactions to particular trade offers. Being able to get insight into customers’ tastes enables businesses to adjust their offer to satisfy their audience. Recommendation systems respond to this need, as their goal is to predict how a given person might rate a specific product. By analysing the visits to your online shop using a recommendation engine, you can adapt the way its content is viewed to users so that it draws their attention and, as a consequence, increases the value of the shopping cart.

What are recommendations good for?

Recommender systems have a number of applications, but their primary purpose is to address every user as a unique individual. A person is the focus of recommendations and the task of the system is to get to know this person, especially when it comes to the individual characteristics that differentiate them from other people. Based on the acquired information, the system browses the shop offer to find the products which are best suited to the particular customer’s needs. As a result, the offer presented to the customer is carefully adjusted to their individual preferences. This mechanism can be used in various industries including healthcare.

These are the main goals of recommendation systems:

• Increasing sales volumes – one of the principal methods of increasing the sales of regular offers is marketing. Marketing activities involve addressing the whole population or a selected target group and directing special offers and promotional material to them. As far as recommendation systems are concerned, sales offers and promotional activities are directed to specific users. This approach increases the likelihood of selling the promoted products.
  

• Customer retention – one of the greatest problems of modern companies, in particular those based on the subscription model, is the churn rate. Retaining customers for a long time is necessary to get a return on the investment made to acquire them. Popular methods of customer retention, such as discounts and coupons, are often rather costly and with short-term effects. This problem can be overcome by adopting an individual approach to customers. Recommendation systems help in understanding the needs and objectives of customers and, eventually, in selecting the correct activities to maintain the relationship with them.

Research has shown that reducing customer churn by a mere 5% can increase your profits by 25–125%.

Joshua Paul

Socious

• Increasing customer satisfaction – the use of recommendation systems helps increase customer satisfaction by enhancing the functionality of the sales service. Spotify, the popular music streaming service, offers 70 million tracks – trying to find the music that you like in such a crowd may be a cumbersome task. Here’s where recommendation systems come in handy: they shorten the distance between the customer’s need and satisfaction. Not only do they help find the search product; they also discover the needs that customers are not even aware they have.

How do recommendation systems work?

Recommendation engines are based on state-of-the-art functions of machine learning.

ML algorithms build mathematical models to make predictions or decisions without being explicitly programmed to do so by humans. They are used in a variety of applications where it is difficult or unfeasible to develop conventional algorithms to perform the needed tasks. The training data for algorithms includes, for example, user behaviour (ratings, clicks, purchase history), user demographics (age, education, income, location), or product attributes (book genre, film cast, food origin).

How to present the recommendations

Preparing the presentation

Since lists are the most popular method of presenting the recommended items, a proper ranking model is necessary to assign rating to every item and rank them from the most to the least attractive ones.

The basic ranking function which increases consumption is the popularity of the item. However, popularity is the opposite of personalisation. If a list is based on popularity, every user gets the same order. This is why it is essential to personalise the experience.

Personalisation can be achieved by completing the system with ratings given by the user to other items. And yet, this is not enough either, because in this approach, the prioritised items might be too niche and the most interesting suggestions might get omitted. It’s easy to picture a situation where a user gives a low rating to a film with a terrible plot even though they love the particular film genre. To make up for this, instead of focusing on popularity or predicted rating only, the ranking should balance these factors.

Learning to rank

There are numerous ways of constructing the ranking function, from simple rating methods to pair preferences and ranking optimisation. One of the available options is combining the popularity factor with the user rating predictions.

You must bear in mind, though, that in the case of ranking recommendations, personalisation is essential: the goal is not to create a universal concept of accuracy but to find ways of optimising customised models.

With large sets of data available, in terms of both the amount and the type, it is vital to adopt an in-depth approach to model selection, training, and testing. This is why all-encompassing approaches to ML algorithms are usually adopted: from unsupervised learning methods, such as cluster analysis, to a series of supervised classification methods, which have given optimum results in various contexts.

Supervised learning happens when the data set supplied for training includes the expected output: for instance, a collection of emails including the information which of them is spam and which isn’t. In this case, a new email should end up in the inbox or in the spam, in accordance with its content. Unsupervised learning, on the other hand, makes it possible to process untagged data to identify previously undetected patterns, e.g. collecting press articles on similar subjects.

These are some of the most important algorithms:

• Linear regression
  
• Logistic regression
  
• Elastic nets
  
• Singular value decomposition
  
• Restricted Boltzmann machine
  
• Markov chains
  
• Latent Dirichlet allocation
  
• Association rules
  
• Gradient-boosted decision trees
  
• Random forests
  
• K-means
  
• Affinity propagation
  
• Matrix factorization
  
• Support vector machines
  
• Neural networks

In the last couple of years, there have been many new algorithms specifically designed to learn to rank, such as RankSVM and RankBoost.

There is no universal method of choosing the best model for a given ranking task. The simpler the feature set is, the simpler the model. It is easy to fall into a trap where a new attribute shows no value because the model is unable to learn it. Or another trap: deciding that a more powerful model is not useful only because there are no attributes that use its advantages.

A/B testing process

Although ratings prove very useful in deciding whether a given model deals well with training data, you cannot be sure that the results will translate into actual improvement of user experience. This is why it is necessary to implement A/B testing to test new algorithms. Tests are normally made on thousands of users and 2 to 20 elements which are variants of the basic idea. With A/B tests, you can check out brave ideas or test multiple projects at once. All in all, their key benefit is that they help you make data-based decisions.

To measure the model performance, a number of factors can be taken into account: from ranking measures, such as NDCG (Normalised Discounted Cumulative Gain), mean mutual rank, and fraction of concordant pairs, to classification indexes, such as accuracy, precision, recall, and F-score.

The analysis shows to what extent these factors correlate with tangible results in A/B tests. However, since mapping is not perfect, performance is used only as a guideline to making informed decisions concerning further tests. If model tests confirm the hypothesis, A/B tests are designed and run to prove that the new functionality is relevant from the user’s point of view.

Application

One of the most popular recommender systems has been created by Netflix. Netflix is based on the subscription business model, which means they need to not only acquire new users but also to retain the current ones. For this reason, it is very important for them to retain the users’ attention and to encourage them to extend their subscription.

The current video base of Netflix includes over 5500 TV shows and films and it is regularly updated. The wide range of content allows Netflix to satisfy their customers’ needs but there is also a risk involved. If users need to take a lot of time looking for a film they want to watch, they might become uninterested and cancel the subscription as a result.

To prevent this, Netflix uses a recommendation engine that helps users find new films. The effectiveness of this approach is confirmed by the fact that most shows watched by Netflix users get through to them by means of the recommendation system. The system enables Netflix to retain their regular customers, which means they also retain profit and, additionally, acquire new users. If recommendations are effective, they attract new users, who want to watch shows they find interesting without spending a lot of time looking for them.

Other online services also use effective AI-based recommendation engines. One of them is Spotify. Spotify puts together a unique list called “Discover Weekly” for every user. The list contains 30 songs selected based on the user’s personal preferences.

As far as machine learning is concerned, Spotify uses a model based on the multi-armed bandit problem that balances between exploitation and exploration. In this case, exploitation refers to providing recommendations in the app, based on the previous music and podcast choices.

Exploration is the opposite of exploitation. It is based on users’ uncertain engagement and it serves as a research tool that provides information on how people interact with the suggested content. Exploration makes it possible to discover new interesting items for users, something they haven’t heard before. Thanks to this balanced approach, shelves and cards are personalised for new and current users.

Unlike traditional department stores, online sellers such as Zalando need to use digital representations of their offer to attract clients. This normally means that there is a database gathering information about the properties, prices, and images of particular items. And yet, the relationship between these attributes and the features that customers actually look for (e.g. how much a given object fits their style) is much more complex.

The recommendation engine is able to provide accurate suggestions concerning a given product or recommend an accessory to match the outfit. One of such networks is Style2Vec, which allows saving an image of a piece of clothing so that its qualities can be easily modified: for instance, you can change the colour, the length, or the cut. Browsing the base with Style2Vec consists in adding and subtracting traits of the clothes.

The results can be used to generate entire outfits and collections based on users’ preferences. This allows:

• recommending similar or matching products,
  
• following the trends and, consequently, launching new products that meet the customers’ needs,
  
• shortening the shopping time.

Another popular platform using an impressive recommendation engine is YouTube. The system consists of two neural networks responsible for different sorts of data filtering.

The first one filters user data to find correlations between users’ preferences, whereas the other sorts videos based on their characteristics and other data, including their ratings, number of views, comments, and descriptions. YouTube recommendations are displayed both on home screen and in the “Skip to next” section, where suggestions show up as you watch a video.

Summary

Comprehensive use of data and practical application of artificial intelligence determine the success of the entertainment and e-commerce tycoons. Thanks to access to unlimited sets of data, they are able to offer extremely accurate recommendations.

The benefits of ML-based solutions outweigh the effort that must be put in their implementation. The fact that this area is still developing is also beneficial. The future may bring brand-new and incredibly effective AI-based methods of interacting with users.

Are you wondering how recommendation systems can help your company? Contact us so that we can help you find the best solution for your needs. You can also learn more about data science.