🤖 AI

Mitigating Matthew Effect: Multi-Hypergraph Boosted Multi-Interest Self-Supervised Learning for Conversational Recommendation

This paper proposes HiCore, a novel conversational recommendation framework that leverages multi-level hypergraphs and self-supervised learning to effectively mitigate the Matthew effect by capturing dynamic multi-interest user preferences.

Original authors: Yongsen Zheng, Ruilin Xu, Guohua Wang, Liang Lin, Kwok-Yan Lam

Published 2026-07-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yongsen Zheng, Ruilin Xu, Guohua Wang, Liang Lin, Kwok-Yan Lam

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are walking through a massive, endless library where a helpful librarian is trying to guess your next favorite book. In a perfect world, this librarian would know your unique taste, from obscure indie novels to classic sci-fi. But in the real world, many recommendation systems suffer from a problem called the "Matthew Effect." It's a fancy way of saying "the rich get richer." If a book is already popular, the librarian keeps shoving it in your face, while the hidden gems in the back corner get completely ignored. Over time, this creates a boring loop where you only see what everyone else sees, and the system forgets to explore your deeper, more specific interests. This happens not just in libraries, but in any system that suggests movies, music, or products based on what you've clicked before. The challenge for scientists is to build a system that doesn't just follow the crowd but actually listens to the unique, shifting conversation between you and the machine, ensuring that the "underdogs" get a fair shot at your attention.

Enter HiCore, a new, clever framework proposed by researchers to fix this exact problem in Conversational Recommender Systems (CRS). Think of a CRS as a chatbot that doesn't just hand you a list of links but actually talks to you, asking questions and learning from your replies to suggest the perfect item. The researchers noticed that as you chat with these bots over time, the "Matthew Effect" gets worse; the bot gets stuck in a rut, only suggesting the same popular things. To break this cycle, HiCore acts like a super-observant detective that builds a complex, multi-layered map of your interests instead of just a simple list.

Instead of looking at you as a single person who likes "Action Movies," HiCore builds three different types of maps, or hypergraphs, to understand you on three levels:

The Item Level: It looks at the specific things you've talked about (like a specific movie title).
The Entity Level: It looks at the "world" around those things (like the director, the genre, or the actors, pulling from a giant knowledge base).
The Word Level: It analyzes the actual words you use in conversation (like "thrilling" or "heartwarming") to catch the vibe of what you're looking for.

But it doesn't stop there. For each of these levels, HiCore draws three different kinds of connections, like different colored strings on a spiderweb:

The Group Channel: It sees how you fit into a crowd. If you and your friends all liked the same obscure band, that's a strong signal.
The Joint Channel: It looks at shared behaviors. Did you and a friend buy the same item at the same time? That suggests a deep, unspoken connection in taste.
The Purchase Channel: It connects dots even between strangers. If you and a total stranger both bought a weird, niche item, HiCore realizes you share a hidden interest, even if you've never met.

By weaving all these threads together, HiCore creates a "multi-interest" profile. It's like having a team of three experts (one for items, one for facts, one for words) who all talk to each other to figure out what you really want, rather than just what's trending. The system uses a special self-learning trick (called self-supervised learning) to practice this on its own data, constantly refining its understanding of your unique "taste fingerprint."

The researchers tested HiCore on four different datasets involving real conversations about movies, music, and more. The results were impressive. In the recommendation game, HiCore beat all the other top methods, including giants like BERT and GPT-3. For example, on the REDIAL dataset, it achieved a Recall@10 of 0.2192, which was a statistically significant improvement over the next best model. In the conversation game, where the bot has to generate a natural reply, HiCore also crushed the competition, showing a 123.83% improvement in diversity (measured by Dist-2) compared to some strong baselines.

Most importantly, HiCore actually solved the "Matthew Effect." When the researchers checked how many different types of items the system suggested, HiCore was much better at bringing up the "long tail"—those less popular, niche items that usually get ignored. It lowered the "Average Popularity" of recommended items to 0.0017 on the OpenDialKG dataset, proving it wasn't just pushing the same famous stuff. It showed that by understanding the complex, multi-layered web of a user's interests, the system could break the echo chamber and offer a truly diverse, personalized experience.

However, the authors are careful to note that this isn't a magic wand without limits. Building these complex, multi-layered maps is computationally heavy, meaning it takes a lot of computer power. They also warn that if the data is too sparse (not enough conversations), the system might struggle to find those hidden patterns. But for now, HiCore stands as a powerful new way to make sure that in the conversation between you and your digital assistant, the quiet, interesting voices get heard just as loud as the popular ones.

Technical Summary: HiCore – Mitigating Matthew Effect in Conversational Recommendation

1. Problem Statement

The paper addresses the Matthew Effect within Conversational Recommender Systems (CRSs). This phenomenon, summarized by the adage "the privileged gain more privilege," describes a scenario where popular items receive increasing attention while less popular ones are marginalized. In CRSs, this issue is exacerbated by the dynamic user-system feedback loop: as users interact with the system over time, the reinforcement of popular items intensifies, potentially creating echo chambers and filter bubbles.

Existing methods face two primary limitations in addressing this:

Static Focus: Most approaches target offline or quasi-static recommendation scenarios, overlooking the adverse effects of the dynamic feedback loop inherent in conversational settings.
Limited Interest Exploration: Current methods often rely on single-channel hypergraphs or traditional Knowledge Graphs (KGs). These fail to capture diverse, high-order user relation patterns due to data sparsity and the restriction to specific relationship types, thereby failing to adequately explore multi-level user interests necessary to break the cycle of popularity bias.

2. Methodology: HiCore Framework

The authors propose HiCore (Multi-Hypergraph Boosted Multi-Interest Self-Supervised Learning for Conversational Recommendation), an end-to-end framework designed to mitigate the Matthew effect by learning multi-level user interests through a dynamic feedback loop. The framework consists of two main components:

A. Multi-Hypergraph Boosted Multi-Interest Self-Supervised Learning

This module constructs a set of triple-channel hypergraphs to model user interests at three distinct levels: Item-level, Entity-level, and Word-level. Each level utilizes three specific channels to capture different facets of user behavior:

Group Channel (g): Analyzes social relations and shared preferences among users (e.g., group motifs).
Joint Channel (j): Captures shared behaviors among friends (e.g., friends purchasing the same items).
Purchase Channel (p): Identifies implicit higher-order relationships among users who lack explicit social ties but share purchase histories.

Construction Details:

Item-Oriented Hypergraphs: Built using triangular motifs (e.g., $M^g_1$ to $M^g_7$ ) derived from historical conversations to represent group, joint, and purchase dynamics.
Entity-Oriented Hypergraphs: Leverages the DBpedia KG to map items to entities and their $k$ -hop neighbors, capturing semantic nuances.
Word-Oriented Hypergraphs: Utilizes the ConceptNet KG to analyze keywords in conversations, capturing semantic associations like synonymy and co-occurrence.

Learning Mechanism:
The framework employs a Hypergraph Convolution Network to propagate information across these multi-channel structures. It uses Self-Gating Units (SGUs) to control information flow from base user embeddings to different channels. Finally, an attention mechanism and graph convolution aggregate the interests from all channels and levels into a unified multi-interest representation ( $X_m$ ). A Self-Supervised Learning objective (using InfoNCE loss) is employed to align the learned representations with ground truth data, enhancing robustness against data sparsity.

B. Interest-Boosted CRS

The learned multi-interest representation ( $X_m$ ) is integrated into the CRS to enhance two tasks:

Recommendation Module: Uses $X_m$ to select target items, aiming to improve diversity and accuracy in predicting user preferences.
Conversation Module: Feeds $X_m$ into a Transformer-based architecture (specifically Multi-Head Attention layers) alongside current and historical conversation contexts. This allows the system to generate responses that are not only contextually relevant but also aligned with the user's diverse, multi-level interests, thereby reducing the tendency to suggest only popular items.

3. Key Contributions

Novel Framework for CRS: HiCore is the first work to build multi-hypergraphs from triple-channel settings (group, joint, purchase) specifically to learn multi-interest representations for mitigating the Matthew effect in conversational settings.
End-to-End Architecture: The proposed framework jointly optimizes conversation generation and item recommendation, utilizing multi-level interests (item, entity, word) to enhance both tasks simultaneously.
Empirical Validation: The study provides extensive quantitative and qualitative evidence demonstrating the framework's superiority in mitigating the Matthew effect compared to state-of-the-art baselines.

4. Experimental Results

The authors evaluated HiCore on four benchmark datasets: REDIAL, TG-REDIAL, OpenDialKG, and DuRecDial.

Recommendation Performance: HiCore achieved new state-of-the-art results across all datasets. On REDIAL, it improved Recall@10 (R@10) to 0.2192 and NDCG@10 to 0.1107, outperforming strong baselines like MHIM and KGConvRec.
Conversation Performance: In terms of response diversity (measured by Distinct-n), HiCore showed significant improvements. For instance, on REDIAL, it achieved a Dist-2 score of 0.5871, representing a substantial increase over baselines like C2-CRS and DialoGPT.
Mitigation of Matthew Effect:
- Coverage: HiCore demonstrated the highest Coverage@k (C@k) across all datasets, indicating it recommends a broader spectrum of items.
- Popularity Bias: The model achieved the lowest Average Popularity (A@K) scores, suggesting a reduced tendency to recommend mainstream items.
- Long Tail: HiCore showed favorable Long Tail Recommendation Ratios (L@K), confirming its ability to surface less popular but relevant items.

5. Significance and Claims

The paper claims that HiCore effectively addresses the dynamic nature of the Matthew effect in conversational systems. By moving beyond static, single-channel models to a multi-hypergraph, multi-interest approach, the system can adapt to evolving user preferences over time. The authors argue that this approach not only improves recommendation accuracy and conversation quality but, crucially, alleviates the disparity where popular items dominate recommendations, thereby fostering a more diverse and equitable recommendation ecosystem.

The authors acknowledge limitations, noting the computational complexity of triple-channel hypergraphs and potential scalability challenges with larger datasets, but maintain that the proposed method represents a significant step forward in balancing recommendation performance with diversity.