Recommender System – A must have AI tool for your business

What is a recommender system?

A recommender system is a tool to suggest relevant content/products to users. For example, in eCommerce it suggests buyers buy relevant products which have already a high probability to get bought by the user, in OTT it suggests what to watch now, in news or blogs, it suggests relevant content that the user would like to read. And it goes on to other verticals where you want to serve personalised service.

Netflix generates its 70% content views from recommender system.

Who uses recommender systems?

Almost in all industries and verticals where personalization matters most. Few potential users are:

1. Retail and eCommerce
2. OTT and entertainment platforms
3. News, Blogs and content generations
4. Lead generations, sales, marketing especially in email marketing
5. HR and recruitment
6. Banking, financial service and insurance (BFSI)
7. And the list goes on…

Types of recommender systems?

1. Content-based filtering
2. Collaborative filtering
3. Hybrid which is mixed with both content-based and collaborative filtering.

What is content-based filtering?

It captures previous search phrases by the user and searches relevant items based on the previous search phrases. Here content refers to attributes perse title of the product, images, descriptions, tags, reviews, ratings etc.  When a user searches for something in the system, it tries to understand semantically what the user wants by NLP algorithms. It embeds all textual data along with image data into mathematical vectors and searches on those vectors. For example, if a user searches “I am looking for funky blue t-shirts that really good fit for summer”, the algorithm embeds the product’s data from the title, descriptions, tags, reviews and images and tries to search semantically. The mathematics operation embeds the product’s data, store its information in such a way that semantic search can be sourced through the following breakdown:

• t-shirt -> title and description vector
• good fit for summer -> from title, description, reviews, ratings
• funky -> reviews and image vectors
• Blue -> tags and image vectors

The advantages of content-based filtering are: data collection, preprocessing and labelling are almost already done. Traning models are fairly easy and faster. The prediction system is highly scalable. Also, it can capture specific interest from a user’s search. On the other hand, features are chosen by the model maker so they can be biased and it is only based on the product’s or user’s attributes so it is not exploring other important aspects of mutual interests.

What is collaborative filtering?

Collaborative filtering takes other user’s mutual interests into the account to generate recommendations. An example we often see in Amazon is “User who bought this product, also bought these products”. It also takes other user’s interactions like product buys, views, clicks, add to cart etc as potential features of the model. So there are two types of collaborative filtering:

1. User-user collaborative filtering
2. Item-item collaborative filtering

User-user collaborative filtering

Assume there are 3 users and 5 items in the system. User 1 likes items 1,2 and 3, user 2 likes 1,3,4. User 3 likes item 5. So the system can recommend item 4 to user 1 and also recommend item 2 to user 2 having said that user 1 and user 2 has common interests or they are similar.

Item-item collaborative filtering

Unlike above it finds similarities between items rather than users. The algorithm creates an interaction matrix for the above case it will be a one-hot encoded matrix that represents users in the row and items on the columns. Now if we fill up 1 to the cells if the user has liked the item and null if the user has not interacted. In this case, we will fit a classification algorithm to predict all the null values whether 0 or 1 and these predicted 1s can be presented as recommendations to the visiting user.

Advantages of collaborative filtering are, it fits into small sets of data and provide accurate predictions and it has a proven track record of generating good results into revenue. On the other hand, it can have cold start issues and extremely time-consuming training.

Conclusion

Based on your business needs, find out which methods can be a good fit for you and always measure its performance through all means of analytics.