🤖 machine learning

Hyperparameter Learning for Latent Factorization of Tensors for Representation Learning to Large-scale Dynamic Weighted Directed Network

This paper proposes DE-LFT, an automated hyperparameter optimization framework based on Differential Evolution that dynamically learns optimal regularization parameters for Latent Factorization of Tensors, thereby improving prediction accuracy and reducing manual tuning efforts in large-scale dynamic weighted directed networks.

Original authors: Yaqian Zhan, Jialan He, Tianzhu Chen

Published 2026-06-10

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Yaqian Zhan, Jialan He, Tianzhu Chen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the future of a massive, bustling city. In this city, people (nodes) are constantly interacting with each other in complex ways: some give money, some give advice, and some ignore each other. These interactions change every second, and they have different "weights" (some interactions are very strong, others are weak).

This is what the paper calls a Large-scale Dynamic Weighted Directed Network. It's like a giant, ever-shifting web of relationships.

The Problem: The "Goldilocks" Dilemma

To make sense of this chaotic city, the researchers use a tool called Latent Factorization of Tensors (LFT). Think of LFT as a super-smart translator that tries to compress this massive, messy web of interactions into a simple, low-dimensional "map" (a tensor) that captures the hidden patterns.

However, this translator has a tricky setting: Hyperparameters.
Imagine you are baking a cake. You need to get the amount of sugar, flour, and baking powder just right. If you use too much sugar, it's cloying; too little, and it's bland. In the world of this AI model, these "ingredients" are numbers called $\lambda_1$ , $\lambda_2$ , and $\lambda_3$ . They control how much the model "trusts" the data versus how much it tries to keep things simple.

The Catch: Until now, finding the perfect amounts of these "ingredients" was like guessing. Researchers had to:

Manually guess (like a chef tasting and adjusting).
Grid Search (like baking 1,000 cakes with every possible combination of ingredients to find the best one).

Both methods are slow, expensive, and require a lot of human effort.

The Solution: The "Evolutionary Chef"

The paper introduces a new method called DE-LFT. Instead of a human chef guessing or baking thousands of cakes, they use a Differential Evolution (DE) algorithm.

Think of DE as a survival-of-the-fittest cooking competition:

The Contestants: The algorithm creates a "population" of different recipe combinations (different values for $\lambda_1, \lambda_2, \lambda_3$ ).
The Taste Test: It tests each recipe on the data. The ones that predict the city's interactions most accurately (with the lowest error) are the "winners."
The Evolution: The winners don't just stop; they "mate" (crossover) and "mutate" (make small random changes) to create new, potentially better recipes for the next round.
The Result: Over many rounds, the population naturally evolves toward the perfect recipe without anyone needing to manually tweak the knobs.

The Results: A Better Cake

The researchers tested this "Evolutionary Chef" on four real-world datasets (like Yelp reviews, music CDs, and two network datasets). They compared it against:

Manual Tuning: The human guessers.
Grid Search: The method that bakes everything.

The Outcome:
The DE-LFT method consistently baked the "best cake." It achieved the lowest MAE (Mean Absolute Error) and RMSE (Root Mean Square Error). In plain English, its predictions were closer to reality than the other methods.

Summary

In short, this paper says: "We built a system that automatically finds the perfect settings for our complex network model using an evolutionary algorithm. Instead of wasting time and money guessing the settings, our system 'evolves' the best settings on its own, resulting in more accurate predictions of how things interact in large, changing networks."

Key Takeaway: It's about automating the "tuning" process so the model can learn better, faster, and with less human help.

Technical Summary: Hyperparameter Learning for Latent Factorization of Tensors

Problem Statement
Large-scale dynamic weighted directed networks (DWDNs) are essential for modeling time-varying interactions in domains such as personalized recommendation, intelligent transportation, and power grid infrastructures. The Latent Factorization of Tensors (LFT) method is widely used to extract latent representations from these networks by modeling them as high-dimensional and incomplete (HDI) tensors and performing low-rank approximations. However, the performance of LFT is heavily dependent on the selection of hyperparameters, specifically the regularization coefficients ( $\lambda_1, \lambda_2, \lambda_3$ ) associated with the latent feature matrices. In current practice, these parameters are typically determined through manual tuning or grid search. These traditional methods are computationally expensive, inefficient, and require significant human effort, often failing to identify optimal parameter combinations that minimize prediction errors.

Methodology
To address the inefficiency of manual hyperparameter tuning, this paper proposes DE-LFT, an automated hyperparameter optimization framework based on Differential Evolution (DE). The core methodology involves integrating the DE algorithm directly into the training process of the LFT model.

Model Formulation: The LFT model approximates observed entries in a third-order tensor (representing the DWDN) using three low-rank latent factor matrices ( $U, S, T$ ). The prediction for an entry $\hat{x}_{u,s,t}$ is calculated as the sum of products of latent factors across dimension $K$ .
Optimization Strategy: Instead of fixed or manually searched regularization parameters, the DE algorithm acts as a population-based heuristic. It iteratively evolves candidate solutions through mutation, crossover, and selection operations.
Objective Function: The DE algorithm searches the hyperparameter space to find the optimal set of regularization coefficients ( $\lambda_1, \lambda_2, \lambda_3$ ) that minimize the prediction error, measured by Mean Absolute Error (MAE) and Root Mean Square Error (RMSE).
Training Process: The LFT model is trained using Stochastic Gradient Descent (SGD), while the DE algorithm simultaneously optimizes the regularization parameters to adaptively improve model accuracy.

Key Contributions

Automated Framework: The paper introduces a novel framework that automates the selection of regularization parameters for LFT, eliminating the reliance on manual tuning or exhaustive grid searches.
Integration of DE: It successfully integrates Differential Evolution into the tensor factorization training loop, allowing the model to adaptively explore the hyperparameter space.
Performance Improvement: The approach is designed to enhance prediction accuracy while reducing the computational costs associated with extensive parameter tuning.

Experimental Results
The proposed DE-LFT method was evaluated on four real-world benchmark DWDN datasets. The datasets were split into 80% training and 20% testing sets. The performance was compared against standard LFT with manually tuned parameters and LFT with Grid Search optimization.

The results, measured by RMSE and MAE, indicate that:

Grid Search provided slight improvements over manual tuning by exploring a broader parameter space.
DE-LFT consistently achieved the best performance across all four datasets.
Specifically, DE-LFT yielded the lowest error rates. For example, on the Yelp dataset, DE-LFT achieved an RMSE of 0.9724 and MAE of 0.7668, outperforming both manual tuning (RMSE: 1.0186) and Grid Search (RMSE: 1.0012). Similar trends were observed across the CDs, Network7a, and Network6a datasets.

Significance and Claims
The paper claims that the proposed DE-LFT framework effectively addresses the bottleneck of hyperparameter selection in tensor factorization for dynamic networks. By automatically learning optimal regularization coefficients, the method improves the prediction accuracy of LFT models without requiring extensive human intervention. The authors conclude that Differential Evolution is a more effective tool for searching the hyperparameter space in this context compared to traditional manual or grid-based approaches, consistently yielding lower MAE and RMSE values on real-world data.

The Problem: The "Goldilocks" Dilemma

The Solution: The "Evolutionary Chef"

The Results: A Better Cake

Summary

More like this