How Today’s Recommender Systems Use Machine Learning to Cater to Your Every Whim

If you’ve ever used the Internet, you’ve encountered a recommender system.

Recommender systems are complex sets of algorithms used by companies to make predictions about what you might want to buy, watch, hear, read, or see online. These systems essentially power your everyday experiences on the Internet. They strongly influence what you buy on Amazon, hear on Spotify, watch on YouTube and Netflix, and consume in your social media feed.

And, while they’ve been around for decades, modern recommender systems are far more sophisticated than the simple ones that tell you which products other users bought. (Though those systems are still alive and well.)

Today’s recommender systems from leading companies like Amazon, Google, Rakuten, TikTok, and others are highly sophisticated models powered by advanced machine learning. They’re able to increasingly personalize your experience online so you see, hear, and buy things that feel like they were chosen specifically for you.

What Makes Recommender Systems Tick?

Whether they recommend products, offers, or content, all recommender systems ultimately determine what makes you more or less compatible with an item or piece of content, according to Julian McAuley, a professor of computer science at University of California San Diego.

“More elaborate models leverage machine learning and capture temporal dynamics and changing user context,” said McAuley. “But the core idea is the same: they use historical interactions to learn which users and items are similar to each other.”

They use different approaches to accomplish that.

Some recommender systems are content-based systems, examining the properties of different items or pieces of content, explained Dinesh Gauri, Walmart Chair of Marketing at the University of Arkansas.

First, a content-based system extracts relevant features that describe a particular item or piece of content. Then, it matches those features to a user profile created from data on how a user has interacted with a website. User interactions could include things like a user clicking on products, past viewing or listening histories, or sharing their preferences with a website.

Collaborative filtering recommender systems, however, make recommendations based on the similarities between how users behave across a website. A very simple example of this is how Amazon tells you that people who bought Item A also typically bought Item B. The system is essentially saying that two items are similar, in the sense that the users who bought Item A overlap significantly with users who bought Item B.

Collaborative filtering recommender systems generally don’t “know” anything about you or the items you like. They don’t know your age, which movies you like, your favorite products, or anything else that might give clues as to your preferences. “They just know that somebody else watched many of the same movies as you, for example, so you might like other things they watched,” said McAuley.

In some scenarios, companies may employ hybrid systems that borrow from both approaches as needed to arrive at the most accurate, useful model, said Gauri.

In any event, despite leading to much more personalized results, the overall approach is quite impersonal. Recommender systems use massive amounts of data to match you with content or match you with other users—or both. With enough data, they’re typically able to find some combination of factors that match your behavior and preferences.

“People tend to anthropomorphize these things, imagining that a recommender system recommends movies the same way your spouse would, by knowing every intimate detail about you,” said McAuley. “Instead, they do something totally trivial, but at massive scale.”

Amazon’s Amazing Recommender Systems

One of the original and current leaders in recommender systems is Amazon. For over 20 years, Amazon has been using various types of recommender systems to suggest products that consumers might like to buy. (The first paper the company published on the subject was way back in 2003.)

Amazon’s recommender systems started out using solely collaborative filtering. But as the company’s pool of user data grew and it sold millions of items to millions of people, it began to pioneer the use of content-based recommender systems to match users to products based on the features of the individual product.

You see these recommender systems at work when Amazon makes (often accurate) recommendations on what you should buy next. However, it’s no longer like those early days. Then, you’d receive a simple recommendation solely on your purchase history. Today, you get tons of different types of recommendations based not only on what you’ve bought, but also on what you’ve viewed, what you’ve expressed interest in, and your digital shopping trends.

With modern times come modern challenges. Modern recommender systems like Amazon’s need to handle three big challenges, said Rajeev Rastogi, vice president of applied science in Amazon’s International Emerging Stores division.

The first is asymmetry, or relationships that run in only a single direction. Plenty of product recommendations are asymmetrical.: if you buy a phone, you recommend a phone case. Pretty easy. But it doesn’t go the other way: If you buy a phone case, you don’t want to recommend a phone.

Amazon’s team uses graph neural networks (GNNs) to essentially give recommender systems more context into the relationships between products, so they can solve for asymmetrical product recommendations.

Another major challenge is delayed feedback. This is a common problem that refers to the fact that the labels on data used to train recommender systems may change over time. Rastogi and his team solve this problem by using novel techniques to predict how likely it is that a label will change in the future, and weigh it accordingly.

Finally, making recommender systems more accurate is a perennial challenge. One way Rastogi and his team are addressing this is by estimating the uncertainty inherent in probability distributions returned by the system. For example, there might be a wide range of predicted probabilities that a customer buys a product; the uncertainty estimate expresses how certain one can be about those predicted probabilities, which enhances overall model accuracy.

But Amazon isn’t the only player dealing with the opportunities and challenges provided by modern recommender systems.

YouTube, owned by Google, relies on content recommendations to fuel its entire business. After all, the longer users stay on the site, the longer they can be served the ads whose revenues comprise the bulk of YouTube’s revenue. The company uses one of the hybrid recommender systems mentioned by Gauri to serve content. It relies on a wide array of signals to serve up content recommendations. Those include individual user interests, past viewing behavior, user engagement with videos, time spent watching videos, and the features of videos themselves to suggest which videos a user might like to watch next.

YouTube also relies on diversified recommendations to improve user satisfaction with its recommender system’s results. A problem many recommender systems, including YouTube’s, run into is that they serve up lots of results that are very similar. These results must be diversified, so users don’t just see a ton of results for things that all look mostly the same. YouTube relies on a type of machine learning model called a determinantal point process that diversifies results at scale, even within a large, mature recommender system.

Tech conglomerate Rakuten also relies on recommender systems to personalize offerings for its customers. Two areas where it leans heavily on recommender systems are ecommerce and content. In its ecommerce offerings, Rakuten employs a content-based recommender system to match users with offers and coupons based on their preferences and shopping history. It also relies on hybrid recommender systems to recommend content across its content platforms.

Predicting the Future

No matter what company you’re talking about, there’s a billion-dollar question lurking behind every single recommender system in consumer life: How accurate is it? Because the more accurate a system is, the more money the company running it makes from product sales, ad sales, or content subscriptions.

You might think the companies with highly accurate recommender systems have some ‘secret sauce’ under the hood that makes them best-in-class. But that’s not actually the case, said McAuley. Instead, the companies doing this right excel at generating great data and have user bases that are highly receptive to recommendations.

“There’s certainly a lot of science and engineering that goes into these things,” said McAuley. “But you can’t make good recommendations without good data. Whereas with great data, you can make great recommendations surprisingly easily.”

According to Gauri, the sites and services we find most addicting are the ones pulling this off. “Amazon, Spotify, Facebook, Netflix, YouTube, Instagram, and TikTok are good examples of companies that are doing a great job.”

TikTok is a perfect example. The company collects vast amounts of user data as users interact with huge numbers of videos over long sessions. This allows the company to build accurate user profiles very quickly. And the vast majority of the content consumption on the platform is recommendation-driven, so TikTok gets very explicit feedback about which recommendations are effective. This helps the company improve future recommendations.

Compare that to a site like eBay, said McAuley. People are unlikely to shop on eBay daily or regularly, and their purchases are largely unrelated to one another based on the nature of the site. As such, eBay is going to provide inferior recommendations to users because it lacks great data.

Lately, the best recommender systems don’t just rely on great data to provide great recommendations. The leading trend heating up this space is the same one heating up the tech world at large:

Generative AI.

Recommender systems are beginning to incorporate conversational interfaces, so you can talk to them just like you’d talk to ChatGPT. These systems are not just more interactive and human-like; they can also explain their predictions to you.

Like all things generative AI, it’s still early, with companies just starting to incorporate these features into product and content recommendations. But it’s no surprise that one of the earliest leaders is one of the original pioneers.

In early 2024, a new generative AI shopping assistant called Rufus was launched that recommends products for you simply through chat.

Its maker? Amazon.