Predicting Churn to Improve RetentionCase Study — Mobile Gaming Industry
Our client is a leader in the web and mobile gaming industry. As is typical in their industry, it was difficult for them to retain new users because of high early churn rates. They wanted to better understand what user behaviors, demographics, and interactions predicted churn in order to intervene to keep high-risk users active on the platform.
High data volume
Our client has millions of users, each of which performs many actions. Actions are streamed regularly as events into their data warehouse.
Rapid prediction demands
Many users churned just minutes after signing up. It was therefore critical to rapidly predict which users were likely to churn so that they could intervene and keep them engaged.
High stakes precision
Offering a retention incentive to a user has costs. We needed to precise calculate the ROI for incentives and determine the optimal threshold at which to intervene.
Solution: Churn prediction on two timescales
We built, validated, and deployed two predictive models to address their needs. The first was a low-complexity model that was able to quickly ingest critical behavioral features and output a player's churn risk in real-time. This rapid model was used to determine when to intervene with low-cost incentives.
The second was a higher-complexity model that involved substantial feature engineering, run as a daily batch process. This model was able to predict with higher accuracy an individual’s risk of churn and was therefore used to determine when to intervene with higher-cost incentives.
Solution: Prediction threshold ROI optimization
Each model predicts a probability of a player churning. Determining the probability threshold at which to intervene is a simple problem. With a low-cost incentive, it might make sense to intervene even for players with only moderate churn risk probabilities because the downside of intervention is so low; in contrast, with a high-cost incentive, you may need to restrict intervention to higher risk probabilities in order to remain ROI positive.
Ultimately, the right threshold at which to intervene depends on a number of factors including the volume of customers, the impact of intervention on retention and spending, the cost of the incentive, the accuracy of your model, and baseline churn rates.
We analyzed model and intervention data to determine the threshold at which a each particular incentive should be offered to players. If you’re interested in experimenting with how this might work for you, we've made a predictive model ROI calculator publicly available to help you determine when to intervene with your own customers.