🤖 machine learning

Routing by Reaching: Composition of Pre-trained GFlowNets for Multi-Objective Generation

This paper proposes a framework that composes pre-trained GFlowNets at inference time to efficiently handle multi-objective generation tasks without retraining, offering exact recovery for linear scalarization and bounded approximation for nonlinear operators while achieving performance comparable to jointly trained baselines.

Original authors: Seokwon Yoon, Youngbin Choi, Seunghyuk Cho, Seungbeom Lee, MoonJeong Park, Dongwoo Kim

Published 2026-07-21

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Seokwon Yoon, Youngbin Choi, Seunghyuk Cho, Seungbeom Lee, MoonJeong Park, Dongwoo Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a master chef trying to invent the perfect new dish. You have a list of ingredients, but you also have a list of rules: it must be spicy, but not too hot; it must be sweet, but not cloying; and it must be cheap to make, but still look fancy. In the world of artificial intelligence, scientists use special tools called "Generative Flow Networks" (or GFlowNets) to act like these chefs. Instead of cooking, they "cook up" new things like molecules for medicine or complex graphs. These tools are great because they don't just find one "best" answer; they explore a whole kitchen full of diverse, promising options, giving you a variety of dishes that fit different tastes.

However, real life is messy. Usually, you don't just want one thing; you want to balance several things at once, like making a molecule that is both strong and safe. The problem is, the old way of doing this was like hiring a new chef for every single combination of rules. If you wanted to change the recipe from "spicy and cheap" to "sweet and safe," you had to fire the old chef and train a brand new one from scratch. This is slow, expensive, and wasteful. The big question scientists have been asking is: Can we take a team of chefs who are already experts in their own specific dishes and have them work together instantly to create a new, complex flavor without needing to retrain them?

This paper introduces a clever new method called "Routing by Reaching" that answers "yes" to that question. The authors propose a framework that acts like a smart conductor, taking a group of pre-trained AI chefs (GFlowNets) and mixing their instructions on the fly to create new, multi-objective recipes. Instead of training a new model for every new combination of goals, the system simply looks at how likely each existing model is to reach a certain step in the process and uses that as a weight to blend their decisions together.

The researchers found that for simple combinations, like adding up different goals with weights (a process called linear scalarization), this mixing method is mathematically perfect—it creates the exact distribution of results you would get if you had trained a new model from scratch. For more complex, non-linear combinations (like using logic to say "must be A but NOT B"), the method isn't mathematically perfect in every single case, but the authors show through simulations that it is incredibly accurate in the most important areas: the high-quality, high-reward regions where you actually want to find your solutions.

To test this, the team ran experiments on two very different stages. First, they used a simple 2D grid game, like a video game map, where they could see exactly how well the AI navigated to reward zones. Here, their method was significantly more accurate than previous approaches, especially as they added more objectives to the mix. Then, they moved to the real world: generating new molecules for drug discovery. They tested this on two types of molecule building: one where they snap together pre-made chunks (fragments) and another where they build atom-by-atom. In these real-world tests, their method matched or even beat the performance of models that required massive retraining for every new goal. Perhaps most impressively, when they used "logical" rules (like "high binding affinity but low toxicity"), their method was roughly 30 to 65 times faster at generating answers than the previous best method, which had to do heavy calculations at every single step.

The paper suggests that this approach turns pre-trained AI models into reusable building blocks. Just as a musician can mix different pre-recorded tracks to make a new song without re-recording the instruments, this method allows scientists to compose new AI behaviors instantly. While the method is not a magic bullet that solves every possible problem (it works best when the underlying models are well-trained and the goals aren't too contradictory), it offers a flexible, fast, and highly accurate way to explore complex scientific problems without the heavy cost of constant retraining.

Technical Summary: Routing by Reaching

Problem Statement

Generative Flow Networks (GFlowNets) are effective for sampling diverse candidates proportional to a reward function, a capability crucial for scientific discovery tasks like molecule generation. However, real-world applications often require balancing multiple, frequently conflicting objectives (e.g., binding affinity, synthetic accessibility, and drug-likeness). Existing approaches to multi-objective GFlowNets generally fall into two categories: scalarization (training a single model conditioned on preference vectors) and logical operators (combining distributions via classifiers).

These existing methods suffer from two critical limitations:

Lack of Reusability: They require joint training for every specific combination of objectives. Introducing a new objective or a new trade-off preference necessitates retraining the model from scratch or training a new auxiliary classifier, incurring substantial computational overhead.
Fragmented Frameworks: Current methods typically address either scalarization or logical composition in isolation, lacking a unified framework capable of handling diverse composition schemes (from linear sums to complex nonlinear operators) without retraining.

Methodology

The authors propose a framework titled "Routing by Reaching," which composes pre-trained GFlowNets at inference time by mixing their forward policies. The core insight is that the reaching probability $u(s)$ —the probability of visiting a state $s$ under a specific forward policy—provides natural weights for combining generation processes. Intuitively, a model that routes more probability mass through a current state is considered more relevant to that state's objective.

The Mixing Rule

Given $k$ pre-trained GFlowNets with forward policies $p_{i,F}$ , reaching probabilities $u_i$ , and terminating distributions $p_i$ , the proposed mixing policy $p_{M,F}$ for a transition from state $s$ to $s'$ is defined as:

$p_{M,F}(s'|s) = \frac{G(u_1(s)p_{1,F}(s'|s), \dots, u_k(s)p_{k,F}(s'|s))}{N_M(s)}$

where:

$G$ is the composition function (e.g., weighted sum for scalarization, harmonic mean for conjunction).
$N_M(s)$ is a local normalization constant ensuring the policy sums to 1.
The reaching probability $u_i(s)$ acts as a weight, scaling the transition probability of the $i$ -th model.

Computing Reaching Probabilities

To implement this without retraining, the authors provide two routes to obtain $u_i(s)$ from pre-trained models:

Model F: For models trained with Flow Matching, Detailed Balance, or Sub-Trajectory Balance, the state flow $F_i(s)$ is explicitly parameterized. $u_i(s)$ is derived directly as $F_i(s)/Z_i$ .
DB F: For models trained with Trajectory Balance (where state flow is not explicit), $u_i(s)$ is reconstructed on-the-fly during trajectory generation using the detailed balance identity: $u_i(s_t) = \prod_{j=1}^t \frac{p_{i,F}(s_j|s_{j-1})}{p_{i,B}(s_{j-1}|s_j)}$ .

Theoretical Analysis

Linear Scalarization ( $\beta=1$ ): The authors prove that for linear weighted sums of rewards, the mixing policy exactly recovers the target distribution. The distortion factor between the induced and target distributions becomes constant.
Nonlinear Operators: For nonlinear compositions (e.g., $\beta \neq 1$ , harmonic mean, contrast), the distortion factor is not guaranteed to be constant. However, empirical analysis shows the distortion remains approximately constant in high-density regions (where the target reward is high), ensuring accurate approximations where they matter most for sampling.

Key Contributions

Training-Free Composition: A method to compose pre-trained GFlowNets for arbitrary objective sets and diverse composition schemes (linear and nonlinear) without any additional training or fine-tuning.
Theoretical Guarantees: Proof of exact recovery for linear scalarization and a quantification of approximation quality for nonlinear operators via a distortion factor.
Unified Framework: A single approach that handles both preference-conditioned scalarization and logical operators (conjunction/subtraction), overcoming the fragmentation of previous literature.
Efficiency: Significant reduction in inference time for logical operators compared to classifier-guided baselines, as it avoids the need to enumerate and evaluate all successor states for a guidance term.

Experimental Results

The framework was evaluated on a synthetic 2D grid and real-world molecule generation tasks (fragment-based and atom-based/QM9).

Synthetic 2D Grid:
- Scalarization: The method achieved substantially lower L1 error than preference-conditioned baselines (MOGFN, HN-GFN) across 2 to 5 objectives. Unlike baselines, performance remained stable as the number of objectives increased.
- Logical Operators: The method performed comparably to classifier-guided baselines for harmonic mean and contrast operations but required no additional training. It significantly outperformed an "ensemble" baseline that mixed policies without reaching probability weighting.
Molecule Generation:
- Scalarization: The approach matched or exceeded the sample quality (average reward and diversity) and hypervolume of baselines that require joint training for each objective combination.
- Logical Operators: The method achieved high target bin percentages for harmonic mean and contrast operations, often outperforming classifier guidance. Crucially, it maintained molecular validity comparable to base models.
- Inference Cost: For scalarization, the method incurred minimal overhead (8–40ms per sample). For logical operators, it was 30–65 times faster than classifier guidance because it avoids the exhaustive state enumeration required by the latter.

Significance and Claims

The paper claims that "Routing by Reaching" enables pre-trained GFlowNets to serve as reusable building blocks. By composing these models at inference time, practitioners can rapidly adapt to diverse objective combinations and explore the Pareto front without the prohibitive cost of retraining.

The authors emphasize that while the method is theoretically exact for linear scalarization, it provides accurate approximations for complex nonlinear operators in high-reward regions. The primary significance lies in decoupling the training of individual objectives from the composition stage, thereby facilitating flexible, multi-objective exploration in scientific discovery domains like drug design, where requirements often shift dynamically. The work does not claim to introduce fundamentally new generative capabilities but rather a mechanism to efficiently leverage existing trained models for complex, multi-objective tasks.