When to Target Customers? Retention Management using Constrained Dynamic Off-Policy Policy Learning

Every business that sells things faces a quiet, persistent problem: people stop buying. In the world of customer relationships, the most dangerous moment is right after a first purchase. Many new customers vanish before they ever return, slipping away while the company is still trying to figure out who they are. Keeping these early customers is vital, because a single repeat purchase often signals a much longer and more valuable relationship. For decades, companies have tried to solve this by sending thank-you notes or coupons, but they have often done so blindly, sending the same message to everyone at the same time, regardless of whether the customer is ready to buy again or has already lost interest.

The challenge is that customers are not static; they change from day to day. A person might be eager to return a week after buying a shirt, but indifferent a month later. This is known as the "recency trap," where the longer it has been since a purchase, the less likely a customer is to return. Yet, sending a reward too soon can be a waste of money if the customer was going to return anyway. The ideal solution would be a system that watches each customer's behavior, waits for the right moment, and then decides whether to offer a reward. But there is a catch: companies have limited budgets. They cannot send coupons to everyone, so they must choose who gets one and when, ensuring every dollar spent brings the most value.

This is the precise puzzle a team of researchers set out to solve using data from a major Japanese online fashion retailer. They developed a method to learn the perfect timing for customer retention, creating a strategy that adapts to each individual's changing behavior while strictly respecting a spending limit. Instead of guessing when to send a coupon, their system learns from thousands of real-world experiments to determine the exact moment a specific customer is most likely to respond to an incentive. The result is a dynamic policy that is not only more effective at keeping customers but also significantly cheaper to run than the standard methods used by businesses today.

The researchers worked with a platform that sells clothing to young women, a place where first-time buyers are abundant but often fleeting. The company's original strategy was simple and static: send a generic thank-you email with a coupon exactly two days, ten days, and thirty days after a customer's first purchase. This approach treated every customer the same, ignoring the fact that some might have already bought a second item by day ten, while others might be completely uninterested. The company wanted to do better. They needed a way to personalize these interactions, but they also had a hard constraint: they could not spend unlimited money on coupons. They needed a strategy that would maximize the number of customers who returned, without blowing their budget.

To find the answer, the team designed a massive, real-world experiment. They took thousands of new customers and randomly assigned them to different groups. Some received a coupon two days after their first purchase, others ten days later, and others thirty days later. Some received no coupon at all, only a thank-you message. Crucially, the experiment was designed so that each customer received at most one coupon during the entire period. This mimicked the real-world constraint that a company cannot afford to bombard a single customer with endless discounts. By observing how these different groups behaved over the following months, the researchers gathered a rich dataset showing exactly how the timing of an offer influenced a customer's decision to return.

With this data in hand, the researchers applied a sophisticated learning method to build a "dynamic treatment regime." In plain terms, this is a set of rules that changes over time. Unlike a static plan that decides everything at the start, a dynamic plan waits and watches. It looks at what a customer has done since their first purchase—what they have browsed, what they have clicked, and how much time has passed—and then decides whether to send a coupon right now, wait, or not send one at all. The researchers extended existing mathematical tools to handle the budget limit, ensuring that the learned rules would never spend more money than allowed. They tested two main approaches: one that tried to predict the future value of a customer based on their history, and another that directly learned which actions led to the best results by treating the problem as a series of classification tasks.

The results were striking. When the researchers tested their new dynamic policies on a separate group of customers, they found that the best-performing strategy was one that respected the budget constraint. This policy did not simply send coupons to everyone on day two. Instead, it learned to be selective. For some customers, it sent a coupon immediately because their behavior suggested they were on the verge of leaving. For others, it waited, sending a coupon on day ten or day thirty, or perhaps not at all, because those customers were likely to return on their own. By withholding coupons from customers who would have returned anyway, the policy saved money. It then used those saved funds to target the customers who truly needed an incentive to come back.

The data showed that this budget-conscious, dynamic approach was far more efficient than the old static methods. While the traditional strategy of sending coupons to everyone on day two increased the return rate by a certain amount, it did so at a high cost. The new dynamic policy achieved a similar or better increase in return rates while spending significantly less money. In fact, when measured by how much the return rate improved for every yen spent, the new policy outperformed all the benchmarks. It turned out that the "one-size-fits-all" approach was wasteful. By waiting and watching, the system could identify the precise moment an incentive would make a difference, avoiding the trap of paying people to do what they were already going to do.

The researchers also explored a more complex version of the problem, where the system could choose not only when to send a coupon but also whether to send a simple thank-you email or nothing at all. In this scenario, the system learned to substitute low-cost emails for expensive coupons when appropriate. It would send a free email to a customer who seemed interested but didn't need a financial push, saving the coupon for someone who did. This flexibility further improved the cost-efficiency of the strategy. The company then took these findings to the real world, running large-scale online tests where the new policies were applied to actual customers. The online results confirmed the offline predictions: the budget-constrained dynamic policy generated the highest return on investment, delivering more retained customers per dollar spent than any other strategy the company had tried.

This work demonstrates that the key to effective customer retention is not just having data, but knowing how to use it over time. The researchers proved that by treating customer retention as a sequence of decisions rather than a single event, and by rigorously accounting for budget limits, companies can achieve much better results. The study suggests that the "recency trap" is real, but it can be navigated with the right timing. The most effective strategy is not to shout the loudest or the most often, but to speak at the right moment to the right person. For the Japanese retailer, this meant a policy that was not only smarter but also more profitable, generating millions of dollars in additional value over a single year compared to their previous methods. The lesson extends beyond fashion retail: in any business where customer attention is fleeting and resources are finite, the ability to adapt and wait is just as important as the offer itself.