Co-authored by Virgile Carron

Adding tracks to favorites is one of Deezer’s most important features. It’s been here for ten years now. It’s the best way to save the content that you love and above all to help us suggest awesome music according to your tastes. It’s also the essence of our main feature: FLOW. The more tracks you add to your favorites, the more relevant our recommendations will be. Until recently, the number of favorite tracks was capped at 2,000.

In 2020, 140,000 users had more than 1,500 favorite tracks.

Lifting the 2,000 favorite tracks limit was identified as the number one request from our community of users. Many of them had reached the limit and could no longer favorite new tracks. The only way to fix it was to remove old favorite tracks, which didn’t make sense. It was time to work on this.

The challenge

Before starting the project, we had two main concerns: User experience and Technical Infrastructure.

Increasing the number of favorite tracks for each user would inevitably increase not only our storage needs, but also the processing time due to the larger amount of data to handle. It might also have a significant impact on loading times, and on our apps’ overall reactivity.

The whole complexity of this project lies within the fact that we needed to find a solution that would satisfy all our users, even if it was solving a power user problem: improving the product for some users should not degrade the experience of the others.

Plus, favorite tracks are everywhere across the Deezer applications and there are numerous possible actions:

• display the favorite state
• add a track to favorites
• remove a track from favorites
• edit the order of your Favorite tracks’ playlist
• download your Favorite tracks’ playlist for offline listening

As it’s a very popular and visible feature, our changes would have an impact on all our users. We thus needed to be particularly cautious.

We decided that the first step to solve this challenge would be to increase the limit of favorite tracks per user to 10,000. It would give us enough experience to potentially raise the limit even higher in the future.

The solution

How to store more data?

The first problem to tackle to make the project possible was storage. Not being able to handle such a load would have directly nipped the project in the bud…

Thanks to our infrastructure team’s analysis and work, it was stated that we could increase the capacity of our existing storage solution, without any impact on our production. This allowed us to avoid setting up a brand new storage solution that would have implied huge increases in production costs.

How to lighten exchanges?

At the beginning, the project was clearly targeting power users who had reached the 2,000 favorite tracks limit and who were no longer able to add tracks to their favorites.

As we fixed our storage limitation, we could have simply increased the limitation using our current implementation. However, it was not that easy as it wouldn’t have scaled to 10,000 (not even to 5,000). We saw this as an opportunity to modernize our client-side architecture and improve the experience of all our users, even those who were not even close to 2,000 favorite tracks.

This made us dig deep in old code and question the way data was exchanged between each of our applications (Android, iOS, Desktop & Web) and our backend.

Historically, when it came to favorites, we used to synchronize all your favorites hydrated with their metadata (track title, main artist name, album title, duration, etc.) at app startup. If it made sense at some point in the past to send such huge payloads, all these data were lately only used to display the ❤️ next to your favorite tracks everywhere in our apps. The only data we still needed were track IDs and the dates they were added to favorites. Therefore, almost all the payload had become useless.

Reducing this payload would not only lower down the amount of data exchanged over the network, but it would also drastically reduce the processing time and memory consumption on the server side since we wouldn’t need to gather and compute all the previous data anymore. This almost divided the payload’s size by a factor of 30!

As well as allowing us to increase the number of favorites sent in a row, it improved the user experience for everyone: every time you open your Deezer app, data is fetched faster and, as a result, startup time is reduced. It might seem trivial but we need to keep in mind that not all Deezer users have the latest smartphone and the best network conditions.

How to display the same data … for more tracks?

After improving the application startup time by downloading only the favorite IDs, we needed to find a solution to display them in the Favorite tracks’ playlist.

We used to load up to 2,000 tracks in a single API call on mobile apps, but increasing the limit to 10,000 tracks would unsurprisingly lead to longer loading times of the Favorite tracks’ playlist, which was definitely not an option. So, we chose to split the API call into multiple smaller calls (= batches of 500 tracks) and to process them in parallel to drastically reduce the time required to fetch all tracks.

There was also another advantage: in our previous implementation, if the request failed, we had to ask for the whole list again. Now, applications will only retry the failed requests, therefore reducing loading times.

In addition to the parallelization of our API calls, we also used the opportunity of knowing the list of IDs to directly ask the API to return the songs that match these IDs (instead of fetching a subset of the user’s favorites).

It reduced pressure on our servers by calling a generic API endpoint that retrieves songs without processing your whole Favorite tracks’ playlist.

As a result, we considerably improved the way we fetch favorite tracks. It had a great impact on performances on all platforms and the pagination when scrolling on the web app is now smoother.

Conclusion

In the end, lifting the favorite tracks’ limit was just the tip of the iceberg: in addition to fixing one of our main users’ frustrations, this project allowed us to change and optimize the way we load favorite tracks and how we interact with them across the Deezer apps. The result is a more enjoyable experience, for all users.

The journey, from research and conception to release, has been very informative. It was also quite intense, but worth it, as we now have much more control over the code of such an important feature.

This improvement opened a lot of opportunities for future projects, notably applying the same kind of solution to every type of content (favorite artists, albums, playlists, etc.) to benefit from it in terms of loading times and app reactivity.

The past few months have taught us a good lesson: never hesitate to question and challenge existing implementations, we can always do it better and make it faster! 🎵

If you would like to help us make the Deezer experience better and faster for millions of users, take a look at our open positions and join one of our teams!