How a machine learning engineer built a churn prediction model that transformed a subscription platform’s retention strategy

Opinions expressed by Digital Journal contributors are their own.

A subscription technology platform with over 100,000 users was losing customers each month despite having access to substantial behavioural data. The retention team could identify who had already churned – but not who was likely to churn next. Namdeo, a machine learning engineer, was brought in to build a predictive solution.

Over 100,000 customer profiles. Session details. Purchase histories. Click rates. Support tickets. NPS feedback. Engagement scores. Subscription behavior. Renewal records.

On paper, it seemed to be an ideal starting point for building a machine learning system.

But the truth was that the firm was losing customers every month, its retention teams were responding reactively, and no one could provide an answer to the one crucial question.

Which customers will churn next and why?

Like many other companies scrambling to integrate AI, the firm had already tried various tactics involving dashboards, automated email campaigns, and analytics. But none of these had actually solved the problem. They could tell who would churn afterwards, but not who would churn ahead of time.

Before building any predictive model, Namdeo focused on constructing the data foundation the model would require.

The data engineering problem behind the machine learning problem

A common misconception in enterprise AI implementations, in Namdeo’s assessment, is that model selection is the primary determinant of project success. Based on her experience, she argues that data infrastructure and problem framing are more frequently the limiting factors.

The moment she started assessing the organization’s data environment, she realized that the issue at hand wasn’t about the lack of machine learning. Rather, it was about the way the organization perceived churn as an issue of reporting rather than behavior.

Data about their customers was available in several different places but it had never been engineered into any kind of signals for which machine learning could be applied.

Before even beginning with her models, she had to build the base.

She began cleaning the data of user behaviors for over 100,000 accounts. She standardized engagement metrics. She correlated data about session frequencies, inactivity periods, content consumption patterns, customer support interactions and purchasing behaviors into features that reflected the actual user behavior.

Building a model the business could actually use

Rather than pursuing complexity for complexity’s sake, Aditi constructed a churn prediction model via logistic regression in Python, pandas, and scikit-learn. This was a deliberate decision.

When the output of a machine learning model is used to inform decisions made by marketers, sellers, and customer success professionals, the interpretability of that model becomes just as important as its accuracy.

Namdeo reports the model achieved 88 percent accuracy in predicting subscription cancellations within a 30-day window, as measured against the platform’s historical data.

However, this was only part of the solution. What made it work was what followed.

Aditi turned predictions into action by creating operational processes, segmenting users based on engagement levels, educating low-engagement subscribers, offering live sessions and premium content to mid-engagement customers, and giving retention incentives. Customer success teams no longer had to guess where to direct their attention.

Retention became proactive.

When AI starts paying for itself

According to Namdeo, measurable retention improvements were observed within months of the system going into production. According to Namdeo, the system contributed to retaining more than 100 high-risk subscribers per month. She estimates this corresponded to over $100,000 in monthly recurring revenue that would otherwise have been at risk of cancellation.

Namdeo says the automation reduced manual customer segmentation time by approximately 75 percent, saving an estimated 420 hours of team time annually.

Namdeo says the system was subsequently integrated into the platform’s standard operational workflow. “As most companies assume that machine learning begins with algorithms, in actuality, it starts with understanding the intricacies of human behavior to the extent of transforming untidy data into meaningful decisions,” says Aditi Namdeo.

Lessons from production: What the churn model revealed

Drawing on Aditi’s experience building production ML systems, Namdeo has identified a pattern she says recurs across enterprise AI implementations. In her view, organisations frequently invest in model development before establishing the data infrastructure that determines whether those models can function reliably in production. 

Organizations spend on AI before spending on data hygiene. They pursue model sophistication before understanding the operational decision. They develop proofs of concept rather than production systems.

The churn model that Aditi built demonstrated something that many companies fail to comprehend. In Namdeo’s view, machine learning’s business value lies not in prediction itself but in how predictions are translated into operational decisions and team behaviours.