Modeling Long-Term Drivers of User Engagement

Video engagement is driven by many factors. On the one hand, topics can increase in popularity based on events and social traction in ways that are highly variable, but on the other hand, users also have fairly stable preferences over the kinds of content they engage with in general. At Prizma, in addition to using highly responsive adaptive learning systems to continuously respond to what your users are engaging with right now, we also utilize advanced machine learning algorithms to predict the longer-term, deeper and more abstract drivers of your users' interests.

We track a variety of user interactions with our videos, and use this data over relatively longer periods of time to train models to detect persistent and more general patterns in what your users find interesting and enjoy watching. These models provide us a priori estimates of video performance even before any user data is collected. This enables us to ensure high user engagement as as soon as new content is available

The feature space used in these models consist of a variety of textual features extracted from video metadata, including keywords, sequences of words, closely related words and important people, places and things to understand the topic and substance of a video. We include detailed standardized relationships between key entities which enable us to understand more deeply their more abstract characteristics that are interesting to your users. We also use these features to help go beyond and infer other psychographic dimensions which include "motivations", i.e. the reasons why someone might be watching a video.  When we build our models , these broader, more abstract features are often some of the most important pieces of information for predicting viewer engagement.

Below is a simple visualization of the relative contributions of thousands of video features including keywords, named entities and Prizma’s psychographic features to video engagement on one of our partners over the last six weeks.  As you can see the vast majority of these features are relatively neutral, with a handful of salient features showing real positive or negative predictive value over time.

In this model, among the top features contributing to video engagement were: 1- motivations, including “wanting to laugh”, “wanting to take care of yourself”, and “wanting to know other people's opinions”  and 2 - details about the key celebrities, e.g. whether they are comedians or politicians. With respect to predictive power, these more abstract features often carry more weight than the more specific counterparts in the traditional metadata column, exceeding even those that are highly correlated to these features.

The ability to estimate video’s performance a priori significantly reduces the time and data required to maximize user engagement. This is especially important for environments where the specific popular topics vary rapidly, for example news sites, or where less data is available due to low traffic, or for partners with large and rapidly growing video libraries. This is also helpful when the usual context or personalization based signals are weaker, for example on a site’s home page.

These estimates have resulted in significant improvements in video engagement for our partners. In one A/B test, we compared the performance of our recommendations with and without predicted scores on tens of thousands of users. We found that using these long-term performance predictors increased number of initiated views by about 20%, but they had an even larger impact on the completion rate for those views which increased by >40%. Since these models emphasize the deeper interests of your users, while also improving click rates, we are able to create much larger improvements in further downstream signals of user retention. The results are summarized below.

The success of these algorithms is determined partly by our extensive, in-depth human-centered feature space. Because these dimensions are interpretable and usable, these algorithms can provide deep actionable insights about the wider interests and motivations of your users. We will not only continue to use these models to ensure the best experience for your users, but we also hope to provide insights from these models to help you understand your users better.

Subscribe to Prizma Blog Updates