Predicting Churn to Improve Retention

Case Study — Mobile Gaming Industry

Our client is a leader in the web and mobile gaming industry. As is typical in their industry, it was difficult for them to retain new users because of high early churn rates. They wanted to better understand what user behaviors, demographics, and interactions predicted churn in order to intervene to keep high-risk users active on the platform.

Key challenges

High data volume

Our client has millions of users, each of which performs many actions. Actions are streamed regularly as events into their data warehouse.

Rapid prediction demands

Many users churned just minutes after signing up. It was therefore critical to rapidly predict which users were likely to churn so that they could intervene and keep them engaged.

High stakes precision

Offering a retention incentive to a user has costs. We needed to precise calculate the ROI for incentives and determine the optimal threshold at which to intervene.

Solution: Churn prediction on two timescales

We built, validated, and deployed two predictive models to address their needs. The first was a low-complexity model that was able to quickly ingest critical behavioral features and output a player's churn risk in real-time. This rapid model was used to determine when to intervene with low-cost incentives.

The second was a higher-complexity model that involved substantial feature engineering, run as a daily batch process. This model was able to predict with higher accuracy an individual’s risk of churn and was therefore used to determine when to intervene with higher-cost incentives.


Solution: Prediction threshold ROI optimization

Each model predicts a probability of a player churning. Determining the probability threshold at which to intervene is a simple problem. With a low-cost incentive, it might make sense to intervene even for players with only moderate churn risk probabilities because the downside of intervention is so low; in contrast, with a high-cost incentive, you may need to restrict intervention to higher risk probabilities in order to remain ROI positive.

Ultimately, the right threshold at which to intervene depends on a number of factors including the volume of customers, the impact of intervention on retention and spending, the cost of the incentive, the accuracy of your model, and baseline churn rates.

We analyzed model and intervention data to determine the threshold at which a each particular incentive should be offered to players. If you’re interested in experimenting with how this might work for you, we've made a predictive model ROI calculator publicly available to help you determine when to intervene with your own customers.