📊 statistics

Hybrid Partial Least Squares Regression with Multiple Functional and Scalar Predictors

This paper proposes Hybrid Partial Least Squares (Hybrid PLS), a novel regression framework that unifies functional and scalar predictors within a hybrid Hilbert space to perform supervised dimension reduction and regression, featuring an efficient NIPALS-based algorithm with roughness penalties and validated by both theoretical geometric properties and applications to renal imaging data.

Original authors: Jongmin Mun, Jeong Hoon Jang

Published 2026-01-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jongmin Mun, Jeong Hoon Jang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a doctor trying to diagnose a kidney problem. You have two types of information about each patient:

The "Movie" (Functional Data): A video recording of how a radioactive tracer moves through the kidney over time. This is a smooth, flowing curve with thousands of data points.
The "Stats Sheet" (Scalar Data): A list of 14 summary numbers derived from that movie, like "peak height," "time to peak," and "total area under the curve," plus the patient's age.

The Problem:
Traditionally, doctors might throw away the "movie" and just use the "stats sheet" because it's easier to handle. But the paper argues this is like throwing away the plot of a movie and only keeping the box office numbers. You lose subtle details that might be crucial for the diagnosis.

However, if you try to use both the movie and the stats sheet at the same time, you run into a mess. The movie and the stats are deeply connected (the stats are literally calculated from the movie). This creates a "tangled web" of correlations. If you try to analyze them with standard math tools, the computer gets confused, the results become unstable, and you might end up with a diagnosis that looks good on paper but fails in the real world.

The Solution: Hybrid PLS
The authors, Jongmin Mun and Jeong Hoon Jang, invented a new mathematical tool called Hybrid Partial Least Squares (Hybrid PLS).

Think of this tool as a smart translator and filter that does three things:

Builds a Unified Language: It creates a special "Hilbert Space" (a fancy math room) where the flowing movie curves and the static numbers can sit side-by-side and talk to each other without getting confused. It treats them as one single, hybrid object rather than two separate, clashing groups.
Finds the "Golden Threads": Instead of just looking for the most common patterns (like a standard camera might), this tool looks specifically for patterns that predict the diagnosis. It ignores the "noise" (the parts of the movie that vary wildly but don't tell you if the kidney is blocked) and focuses only on the "signal" (the specific shape changes that indicate a blockage).
Keeps it Smooth: Because the data comes from a smooth biological process, the tool forces the results to be smooth, too. It prevents the math from getting "jittery" or over-interpreting tiny, random wiggles in the data.

How It Works (The "NIPALS" Dance)
The paper describes an algorithm called NIPALS (Nonlinear Iterative Partial Least Squares). Imagine a dance where the computer takes turns:

Step 1: It looks at the data and finds the single best "direction" (a combination of the movie and the stats) that explains the most about the kidney blockage.
Step 2: It subtracts that explanation out of the data, leaving behind only the "leftovers" (the parts it hasn't explained yet).
Step 3: It repeats the process on the leftovers to find the next best direction.

It keeps doing this until it has built a small, efficient "summary" of the data that captures everything important.

Why It's Better (The Results)
The authors tested this on fake data and real kidney scans from Emory University.

The "Movie vs. Stats" Test: When they tried to predict kidney blockages, their new method found the answer using just one or two of these "golden threads."
The Competition: Other methods (like standard Principal Component Regression) needed three times as many threads to get the same accuracy, and even then, they were often confused by the noise.
The "Overfitting" Trap: Standard methods often try to memorize the specific quirks of the training data (overfitting), which makes them fail on new patients. The Hybrid PLS method stayed stable and accurate, proving it learned the actual rules of kidney blockages rather than just memorizing the examples.

In a Nutshell
This paper presents a way to combine "movies" (curves) and "numbers" (stats) into a single, powerful diagnostic tool. It cuts through the confusion caused by their natural connection, strips away the noise, and delivers a simple, smooth, and highly accurate way to predict kidney health. It's like having a translator that can read a complex novel and a spreadsheet simultaneously to tell you exactly what the story is about, without getting lost in the details.

Technical Summary: Hybrid Partial Least Squares Regression with Multiple Functional and Scalar Predictors

1. Problem Statement

The paper addresses the challenge of supervised dimension reduction and regression in settings where the response variable is influenced by a mixture of scalar covariates and multiple functional predictors (curves). This hybrid setting, formalized as a Partially Functional Linear Model (PFLM), presents two primary difficulties:

Parsimony: The need for interpretable, low-dimensional representations, particularly when clinical decision rules are required.
Multicollinearity: The presence of complex dependencies at four levels: (i) within-function correlations, (ii) between-function correlations, (iii) correlations among scalar predictors, and (iv) intermodal correlations between functional and scalar predictors.

Existing methods often fail to address these issues simultaneously. High-dimensional Euclidean methods (e.g., Lasso, SCAD) assume homogeneous data structures. Functional regression methods typically focus on single functional predictors or use two-stage procedures (e.g., converting functional data to finite-dimensional representations via unsupervised PCA, followed by regression) that ignore cross-modality correlations and treat functional representations as response-agnostic.

2. Methodology: Hybrid PLS

The authors propose Hybrid Partial Least Squares (Hybrid PLS), a framework that unifies functional and scalar predictors into a single mathematical structure to perform supervised dimension reduction.

A. Unified Hybrid Hilbert Space
The method constructs a product space $\mathcal{H} := (L^2[0, 1])^K \times \mathbb{R}^p$ to treat functional and scalar predictors jointly. An element $h \in \mathcal{H}$ is an ordered tuple $(f_1, \dots, f_K, u)$ . The inner product is defined as:
$\langle h_1, h_2 \rangle_{\mathcal{H}} := \sum_{k=1}^K \int_0^1 f_{1k}(t)f_{2k}(t) dt + \omega u_1^\top u_2$
where $\omega$ is a weight parameter balancing the scales of functional and scalar components. This allows the extension of the Nonlinear Iterative Partial Least Squares (NIPALS) algorithm to the hybrid setting.

B. Population-Level Algorithm
At the population level, the method seeks PLS directions $\xi$ that maximize the squared covariance between the predictor $W$ and the response $Y$ , subject to orthogonality constraints. The authors establish that the optimization problem is equivalent to finding the eigenfunctions of a composite operator $U = \Sigma_{YW} \otimes \Sigma_{YW}$ , ensuring the problem is well-posed under mild moment conditions. The algorithm guarantees orthogonality of latent scores and uncorrelated residuals.

C. Sample-Level Implementation
The proposed sample algorithm (Algorithm 3) alternates between two steps:

Regularized Estimation of PLS Directions: To prevent overfitting and ensure smoothness (crucial for functional data), the PLS direction $\hat{\xi}$ $\hat{ξ}$ is estimated by maximizing empirical covariance subject to a roughness penalty on the second derivatives of the functional components. This leads to a generalized Rayleigh quotient problem.
- Key Computational Insight: Due to the rank-one structure of the covariance matrix, the authors derive a closed-form solution that requires solving only linear systems. This avoids the computational instability often associated with generalized eigenproblems.
Residualization: Both predictors and the response are residualized against the estimated scores, mirroring the standard NIPALS procedure but within the hybrid Hilbert space.

D. Final Estimation
The regression coefficient $\hat{\beta}$ is reconstructed as a linear combination of the estimated PLS directions, adjusted for the residualization steps to ensure consistency with the original model.

3. Key Contributions

Theoretical Extension: Rigorously extends the NIPALS algorithm to a hybrid setting involving multiple functional and scalar predictors by defining a unified Hilbert space and proving the existence of a complete orthonormal system of eigenfunctions for the associated operator.
Algorithmic Efficiency: Derives a computationally efficient closed-form solution for the regularized iterative steps, making the method scalable for dense or irregular functional data.
Geometric Properties: Establishes that the proposed framework preserves the core geometric properties of PLS, specifically the orthogonality of PLS directions (under a roughness-sensitive inner product) and the orthogonality of latent scores.
Handling Intermodal Correlation: Unlike two-stage PFLM approaches, Hybrid PLS explicitly models the joint dependencies between functional and scalar predictors, preventing the loss of predictive information contained in cross-modality correlations.

4. Results

Numerical Studies:

Geometric Validation: Simulations confirmed that the orthogonality properties (between directions, scores, and residuals) hold robustly across different regularization strengths, with deviations on the order of $10^{-11}$ to $10^{-16}$ .
Predictive Power vs. Parsimony: In synthetic scenarios involving dominant non-predictive variance (nuisance factors) and strong cross-modality correlations, Hybrid PLS significantly outperformed Principal Component Regression (PCR). Hybrid PLS identified the predictive signal in the first component, whereas PCR was misled by high-variance nuisance factors, resulting in higher error and redundant representations.
Coefficient Estimation: Sensitivity analyses showed that roughness penalties effectively control estimation error for both functional and scalar coefficients, with regularization parameters selected via cross-validation.

Real Data Application (Emory University Renal Study):

The method was applied to a dataset of 216 kidneys with two functional renogram curves (baseline and post-furosemide) and 14 scalar pharmacokinetic covariates.
Performance: Hybrid PLS achieved lower test Root Mean Squared Error (RMSE) and greater stability than Penalized Functional Regression (PFR) and scalar-only OLS.
Efficiency: It matched the predictive power of PCR but with significantly greater parsimony. While PCR required $3L$ components (extracting $L$ from each of the three sources) to reach similar accuracy, Hybrid PLS achieved comparable performance with only $L$ components.
Interpretability: The first latent component extracted by Hybrid PLS aligned with clinical knowledge of kidney obstruction (e.g., distinguishing accumulation vs. drainage patterns), whereas PCR exhibited strong, potentially redundant correlations between its component scores across modalities.

5. Significance and Claims

The paper claims that Hybrid PLS provides a parsimonious and robust framework for regression in the presence of mixed data modalities. By embedding functional and scalar predictors into a unified space, the method:

Mitigates Multicollinearity: It effectively handles the complex intermodal correlations that degrade the performance of standard two-stage approaches.
Preserves Predictive Structure: It extracts latent components that maximize covariance with the response, ensuring that low-dimensional representations retain predictive power even when the primary variance in the data is non-predictive.
Ensures Computational Feasibility: The closed-form solution allows for efficient computation without sacrificing the theoretical guarantees of orthogonality and smoothness.

The authors conclude that this unified approach is particularly valuable in biomedical contexts, such as renal imaging, where combining full functional trajectories with scalar summaries yields superior predictive models compared to methods that treat these data types separately or sequentially. Future work suggested includes deriving theoretical convergence rates and minimax optimality bounds.