Optimizing Movie Selections: A Multi-Task, Multi-Modal Framework with Strategies for Missing Modality Challenges

Subham Raj*

Pawan Agrawal*

Sriparna Saha*

Brijraj Singh*

Niranjan Pedanekar*

* External authors

SAC-24

2024

Abstract

Online recommendation systems have become a crucial feature of Over-the-Top (OTT) platforms, which provide streaming media content over the internet. OTT platforms, such as Netflix, Hulu, and Amazon Prime, use recommendation systems to suggest movies, TV shows, and other content to users based on their viewing history and preferences. The accuracy of these recommendations can impact user satisfaction, retention, and revenue for the platform.
In this context, multi-task and multi-modality learning approaches have shown promise in improving the accuracy of movie recommendations. Our proposal is that by simultaneously tackling two correlated tasks, specifically (a) classifying movie genres and (b) identifying user-movie ratings, we can produce high-quality movie embeddings through an end-to-end approach without the use of a rating vector. Additionally, two more correlated tasks, the user's gender and age-group, are also considered for our experiments. We have developed several multitasking models, including fully shared (FS) and shared-private (SP) feature models, to address multiple tasks at once, namely genre classification and user-movie rating prediction. Additionally, we've broadened our method to address situations where certain modalities are missing. Recognizing that real-world situations might not always provide information from every modality, this tackles the challenge of missing modalities. Hence, our proposed model MTM4F which stands for Multi-Task Multi-Modal Missing Modality Framework includes the integration of a meta-learning framework and a multi-task architecture. In experiments, we utilized a multi-modal format of the MovieLens-100K and MMTF-14K datasets. When tested on these datasets using the root mean square error (RMSE) metric, the suggested multitask model, combined with its shared private training approach, surpasses state-of-the-art models currently available.