When and What Will Customers Buy Next? A Dual-Task Deep Learning Framework for Repurchase Prediction in E-Commerce

Technical Summary: A Dual-Task Deep Learning Framework for Repurchase Prediction in E-Commerce

Problem Statement
Current research in e-commerce customer retention predominantly focuses on binary classification: predicting whether a customer will repurchase. However, this approach overlooks two critical dimensions required for effective operational strategy: the specific time frame of the next purchase and the specific product the customer intends to buy. Furthermore, traditional statistical models and standard machine learning algorithms often rely on aggregated features (e.g., RFM metrics) that fail to capture the temporal dynamics and sequential dependencies inherent in transaction data. Additionally, existing deep learning approaches frequently treat timing and product selection as independent tasks or struggle with "zero-inflated" data, where a significant portion of transactions occur on the same day, leading to skewed distributions that challenge standard regression metrics.

Methodology
The study proposes a Multi-Task Deep Learning Framework based on Long Short-Term Memory (LSTM) networks to simultaneously predict the time-to-next-purchase (in days) and the next product (StockCode).

• Data Source & Preprocessing: The model utilizes the UCI Online Retail dataset (541,909 transactions). Preprocessing involved removing cancelled orders, negative quantities, and missing CustomerIDs. To handle customers appearing in multiple countries, the "Mode" method was applied to assign a primary country. The dataset was reduced to 396,337 rows for analysis.
• Feature Engineering: The model operates on sequential transaction chains for each customer. A window size of 5 was used, meaning the model analyzes the five most recent transactions to predict the next event.
  • Input Features (6 per time step): Log of Revenue, Log of Quantity, Time Gap (days between transactions), Rolling Revenue Mean (3 transactions), Rolling Revenue Standard Deviation, and Cumulative Revenue.
  • Embeddings: Product StockCodes are embedded into 32-dimensional dense vectors. Time features are projected into a 32-dimensional space.
• Architecture:
  • Shared Encoder: A stacked LSTM architecture (64 units followed by 32 units) processes the input sequence. The final hidden state serves as a shared representation of the customer's transaction history.
  • Dual Output Heads:
    1. Time Prediction: A dense layer with a single neuron and Softplus activation to ensure positive day predictions.
    2. Item Recommendation: A dense layer with $K=3,655$ units (matching the unique product count) and Softmax activation to output a probability distribution over all products.
• Loss Functions: The model is trained using a weighted sum of two loss functions with equal weights ($\alpha = \beta = 1$):
  1. Mean Absolute Error (MAE): For the time prediction task, chosen for its robustness to outliers and interpretability in days.
  2. Sparse Categorical Cross-Entropy: For the item recommendation task.

Key Results
The model was evaluated on a dataset containing 540,000+ transactions and 3,655 unique products, compared against baselines including Median/Mean predictors, Random Forest, XGBoost, and a single-task LSTM.

• Time Prediction:

  • The proposed Multi-Task LSTM achieved a Mean Absolute Error (MAE) of 1.62 days.
  • The Median Absolute Error (MdAE) was 0.00 days, indicating that for more than 50% of samples (specifically the 96.6% of same-day purchases), the prediction was exact.
  • SMAPE (Symmetric Mean Absolute Percentage Error) was 10.4%, significantly outperforming the single-task LSTM (200%) and other baselines.
  • R² Score: The model reported a negative R² (-0.012). The authors attribute this to the zero-inflated nature of the data (75% of inter-purchase intervals are 0 days) and the presence of long-tail outliers (1.5% of samples with gaps >30 days). The negative R² is driven by the large squared errors of these rare long-interval events, despite the model performing accurately on the vast majority of data.
  • Robustness Check: Using Pinball Loss (quantile loss at $\tau=0.5$), the proposed model showed a 48.2% improvement over the mean baseline, confirming its effectiveness on skewed distributions.

• Item Recommendation:

  • Top-1 Accuracy: 2.35% (vs. random baseline of 0.027%).
  • Top-5 Accuracy: 9.17% (vs. random baseline of 0.122%).
  • The model outperformed random guessing by 86 times for Top-1 and 67 times for Top-5.
  • Mean Reciprocal Rank (MRR): 0.066, a 28-fold improvement over the random baseline.

Significance and Contributions
The paper claims three primary contributions to the literature and practice:

1. Methodological Innovation: It introduces a novel multi-task LSTM architecture that jointly learns the timing and product of the next purchase, leveraging shared representations to capture the correlation between when and what a customer buys.
2. Data Handling: The study demonstrates the efficacy of deep learning on raw sequential transaction data without relying on pre-aggregated features, specifically addressing the challenges of zero-inflated e-commerce data where same-day multi-item purchases are common.
3. Practical Application: The authors propose an AI agent system concept that utilizes these predictions to automate marketing actions (e.g., timing promotions, personalizing recommendations) without human intervention, aiming to reduce operating costs and optimize customer retention.

Limitations and Future Work
The authors modestly acknowledge that the model struggles with predicting long intervals (>30 days) due to data scarcity. They suggest future work should explore two-stage modeling or survival analysis for long-term horizons. Additionally, the current model does not explicitly model product-product relationships (complementary or substitute effects), which is identified as a direction for improving item recommendation accuracy. The proposed AI agent remains a conceptual framework requiring further development for full end-to-end automation.