🤖 machine learning

Contextual Bandits for Maximizing Stimulated Word-of-Mouth Rewards

This paper proposes a novel contextual multi-armed bandit framework that learns individual spillover probabilities to optimize the targeting of connected users in social networks, thereby maximizing stimulated word-of-mouth rewards by accounting for influence heterogeneity.

Original authors: Ahmed Sayeed Faruk, Elena Zheleva

Published 2026-06-16

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ahmed Sayeed Faruk, Elena Zheleva

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a marketing manager trying to spread the word about a new product. You have a budget to give out a limited number of "free samples" or "referral bonuses." Your goal isn't just to give these to random people; you want to give them to the specific people who will not only like the product themselves but will also excitedly tell their friends about it.

This paper is about building a smart computer system that figures out who those friends are, even though it doesn't know them at the start.

Here is the breakdown of the paper's ideas using simple analogies:

1. The Problem: The "Ripple Effect" is Different for Everyone

In the real world, when you tell a friend about a movie, they might love it and tell ten other people. But if you tell a different friend, they might not care at all. This is called spillover.

The tricky part is that this "spillover" isn't the same for everyone.

The Old Way: Most systems assume everyone's friends are equally likely to be influenced. They might guess, "Oh, this person has 100 friends, so they are a good target."
The Reality: Some people are "super-connectors" whose friends are very likely to listen. Others have friends who are very stubborn. The paper argues that we need to learn exactly how likely each specific pair of friends is to influence each other.

2. The Solution: A "Gambler" Who Learns

The authors created a system called SpillCB. To understand how it works, imagine a gambler in a casino with many slot machines (these are called "arms" in the paper).

The Goal: The gambler wants to pull the lever on the machine that pays out the most money.
The Twist: The gambler doesn't know which machine is the best. They have to try them out to learn.
The Context: In this paper, the "machines" are the different friends (neighbors) of a user. The "context" is the information we know about them (like their interests or how close they are).

The system uses a strategy called Contextual Multi-Armed Bandits. Think of it as a two-phase learning process:

Phase 1: Exploration (The "Tasting" Phase):
At the beginning, the system is like a food critic trying new dishes. It randomly picks a few friends to recommend the product to, just to see what happens. It doesn't know who is the best yet, so it has to take some risks to gather data.
Phase 2: Exploitation (The "Ordering" Phase):
Once the system has tasted enough dishes (gathered enough data), it switches to being a smart chef. It looks at the data it collected and says, "Okay, based on what I learned, this specific friend is 90% likely to tell their friends, while that one is only 10% likely." It then focuses all its recommendations on the best friends.

3. How It Works in Practice

The system looks at a network of people (like Facebook or Flickr). When a user gets a reward to share a product, the system has to choose k (a small number) of their friends to share it with.

The Guess: The system looks at the user and their friends. It uses math to guess the "spillover probability" (the chance that Friend A will tell Friend B).
The Test: It picks the top friends based on that guess.
The Feedback: If the friends actually share the product, the system gets a "reward" (a point). If they don't, it gets zero.
The Update: The system updates its math. "Okay, I was right about Friend A, but I was wrong about Friend B. Next time, I'll pick differently."

4. What They Found

The researchers tested this on real social network data (from Flickr and Facebook). They compared their smart "Gambler" system against:

Random: Picking friends by rolling a dice.
Similarity: Picking friends who look exactly like the user (e.g., same age, same interests).
Old Math Models: Using standard statistics to guess connections.

The Result:
The SpillCB system (the smart gambler) was much better at finding the right friends.

It learned faster as time went on.
It made fewer mistakes in guessing who would share the product.
Crucially, it found that exploring (trying new, risky friends) for a little while at the start helped it make much better choices later on.

Summary

The paper presents a new way to use computer learning to figure out who influences whom in a social network. Instead of guessing or using one-size-fits-all rules, the system acts like a smart learner: it tries different people out, learns who is the best at spreading the word, and then focuses its efforts on those specific people to get the most "word-of-mouth" rewards.

The authors conclude that this method works better than current standard methods, but they note this is preliminary work and they plan to test it on even more data in the future.

Technical Summary: Contextual Bandits for Maximizing Stimulated Word-of-Mouth Rewards

Problem Definition
The paper addresses the challenge of optimizing stimulated word-of-mouth (WOM) in social networks. Unlike organic WOM, stimulated WOM relies on incentives (e.g., rewards, discounts) to encourage users to share information. The core problem is heterogeneity in network spillover: the probability that a recommendation made by a system-activated user $v_i$ will influence a neighbor $v_j$ to adopt the product varies significantly across different user pairs and network positions.

Existing literature often assumes spillover probabilities are known or focuses on general influence maximization without learning individual edge probabilities. This paper identifies a gap where current stimulated WOM strategies overlook these varying spillover effects, leading to sub-optimal targeting. The specific problem formulated is Spillover Probability Prediction for Top- $k$ Neighbor Recommendation: given an attributed network $G=(V, E)$ , the system must learn the spillover probability $e_{ij}.p$ for edges over $T$ rounds to recommend the top- $k$ neighbors most likely to activate, thereby maximizing cumulative rewards.

Methodology: SpillCB Framework
The authors propose SpillCB, a novel Contextual Multi-Armed Bandit (CMAB) framework designed to learn individual spillover probabilities in an online setting.

Data Model: The network is represented as a bidirected attributed graph where nodes have attribute vectors ( $v.X$ ). The context for an edge $e_{ij}$ is defined as the concatenation of the attributes of the source node $v_i$ , the target neighbor $v_j$ , and relevant product attributes.
Bandit Formulation: The problem is modeled as a stochastic 2-armed bandit for each edge:
- Arm $a_0$ : Do not recommend the neighbor.
- Arm $a_1$ : Recommend the neighbor $v_j$ to the system-activated node $v_i$ .
- Reward: A binary reward (1 or 0) is generated if the recommended neighbor $v_j$ becomes activated due to spillover. The reward is a function of the context vector and the action, plus noise.
Learning Strategy: The framework operates in two phases over $T$ $T$ rounds:
1. Exploration: For an initial fraction ( $z\%$ ) of rounds, $k$ neighbors are selected randomly to diversify edge selections and gather data.
2. Exploitation: In subsequent rounds, the algorithm selects the top- $k$ neighbors based on the estimated expected rewards of arm $a_1$ , predicted by the underlying CMAB algorithm.
Algorithms: The framework is agnostic to the specific CMAB solver but is evaluated using LinUCB and EENet. The goal is to minimize cumulative regret, defined as the difference between the rewards of the optimal top- $k$ set and the selected set.

Experimental Setup and Results
The authors evaluated SpillCB on two real-world attributed network datasets: Flickr and Facebook. Spillover probabilities were synthetically generated based on a weighted function of cosine similarity, level similarity, topological overlap, diffusion importance, and degree product.

Baselines: The proposed method was compared against:
- Random: Random selection of $k$ neighbors.
- Similarity-based: Selection based on Euclidean distance of attribute vectors.
- Ridge Regression: Link weight prediction using ridge regression on labeled edges.
- GraphSage: Link weight prediction using node embeddings and a neural network scoring function.
Metrics: Performance was measured using Recall@k (ability to identify true top- $k$ neighbors) and Root Mean Squared Error (RMSE) of the learned spillover probabilities.
Key Findings:
- SpillCB variants (SpillCBLinUCB and SpillCBEENet) consistently outperformed all baseline methods in Recall@k across varying values of $k$ (5 to 25) on both datasets.
- Increasing $k$ generally reduced RMSE, as it allowed for the exploration of more edges, accelerating the learning process.
- A trade-off was observed between the exploration ratio ( $z$ ) and performance: increasing exploration from 0% to 20% reduced RMSE (improving probability estimation accuracy) but slightly decreased Recall@10 in the final round.
- As the number of rounds ( $T$ ) increased, Recall@10 improved and RMSE decreased, demonstrating the effectiveness of the online learning approach.

Significance and Claims
The paper claims to present preliminary work that leverages contextual multi-armed bandits to estimate heterogeneous spillover effects, a factor often overlooked in stimulated WOM optimization. The primary contribution is a framework that learns individual spillover probabilities rather than assuming them to be known.

The authors assert that accounting for spillover heterogeneity enhances the targeting precision of top- $k$ connected users, boosting rewards and outperforming methods that do not learn individual spillover effects. The work aims to improve the effectiveness of recommendation systems by delivering more personalized user experiences and maximizing WOM rewards. The authors remain modest regarding their claims, noting that the results are preliminary and that future work will involve testing on more datasets and baselines to further understand the approach's strengths and limitations.