FunctionalCalibration: an R package for estimation in aggregated functional data model

The paper introduces the R package `FunctionalCalibration`, which provides tools for estimating individual constituent curves from aggregated functional data observations using spline or wavelet basis expansions within additive error models, such as those found in chemometrics.

The Gist

Imagine you are a chef standing in front of a giant, delicious smoothie. You can taste the final drink, but you don't know exactly how much strawberry, banana, or mango went into it. You know the recipe (the ratio of ingredients), but you've lost the individual fruit profiles.

Now, imagine that instead of fruit, you are dealing with light waves passing through a mixture of chemicals. This is the real-world problem this paper solves, specifically in a field called chemometrics (using math to analyze chemicals).

Here is the story of the FunctionalCalibration package, explained simply.

The Problem: The "Blended Smoothie" Mystery

In many scientific labs, scientists measure a mixture of substances. According to a famous rule called the Beer-Lambert Law, the light absorption of a mixture is just the sum of the light absorption of its individual parts, weighted by how much of each part is there.

• The Mixture: The "Aggregated Curve" (the smoothie you can see).
• The Ingredients: The "Constituent Curves" (the hidden strawberry, banana, and mango profiles).
• The Weights: The concentration (how much of each ingredient is in the mix).

Usually, scientists know the weights (they know they mixed 50% water and 50% alcohol). But they want to know: "What does the pure alcohol curve look like? What does the pure water curve look like?"

This is hard because the data is noisy (like static on an old radio) and the ingredients are blended together perfectly.

The Solution: A New Digital Tool

The authors, Alex and Vitor, built a free tool called FunctionalCalibration (an R package) to act as a "digital blender" that can un-mix the smoothie. It uses two different mathematical "lenses" to separate the ingredients: Splines and Wavelets.

Think of these two lenses as different ways to look at a picture:

1. The Spline Lens: The "Smooth Painter"

Imagine you are trying to draw a perfect, rolling hill. You use a flexible ruler (a spline) to connect dots smoothly.

• How it works: This method assumes the ingredients are smooth and gentle, like a calm river. It connects the dots with smooth curves.
• When to use it: If your chemical ingredient changes slowly and steadily (like the smooth curve of a sine wave), Splines are great.
• The Limitation: If your ingredient has a sharp edge, a sudden spike, or a jagged cliff (like a step function), the "Smooth Painter" gets confused. It tries to smooth out the sharp edge, making the result look blurry and inaccurate.

2. The Wavelet Lens: The "Microscope & Flashlight"

Now, imagine you have a flashlight that can zoom in and out.

• How it works: Wavelets are like a set of tiny flashlights. Some are wide and catch the big picture (the overall shape of the curve). Others are narrow and zoom in to catch tiny details, sharp spikes, or sudden jumps.
• The Magic Trick: The package uses a technique called Shrinkage. Imagine you are looking at a noisy photo. The Wavelet method says, "That tiny speck of dust is probably just noise, let's shrink it to zero. But that big, sharp spike? That's a real feature! Keep it."
• When to use it: This is perfect for ingredients that have sudden changes, jagged edges, or weird spikes. It doesn't try to force a jagged edge into a smooth curve; it respects the sharpness.

How the Tool Works in Practice

The paper demonstrates this with a "Simulated Dataset" (a fake smoothie created by the computer).

1. The Setup: They created two fake ingredients:
   • Ingredient A: A smooth, wavy curve (like a gentle hill).
   • Ingredient B: A jagged, step-like curve (flat, then a sudden jump, then flat again).
2. The Mix: They blended them together with random amounts to create a noisy, messy "Aggregated Curve."
3. The Test:
   • When they used the Spline tool, it did a great job recreating the smooth Ingredient A. But it failed miserably at Ingredient B, turning the sharp step into a blurry ramp.
   • When they used the Wavelet tool, it nailed both! It captured the smooth waves of A and the sharp, sudden jumps of B perfectly.

Why Does This Matter?

In the real world (like analyzing meat quality or chemical mixtures), scientists often need to know the pure properties of a substance to calculate how much of it is in a new sample.

• Before this tool: Scientists might have to use expensive, slow lab tests to figure out concentrations.
• With this tool: They can use a quick, cheap light scan, run it through the FunctionalCalibration package, and instantly know the concentration of every ingredient, even if the data is messy or the ingredients have sharp, weird shapes.

The Bottom Line

The FunctionalCalibration package is like a smart kitchen gadget that can take a blended smoothie, look at it through a Smooth Painter's eye (Splines) or a Detail-Oriented Microscope (Wavelets), and tell you exactly what the original fruits looked like.

If your ingredients are smooth, use the painter. If they are jagged and full of surprises, use the microscope. This package lets you choose the right tool for the job, saving time and money in the lab.

Technical Summary

1. Problem Statement

The paper addresses the statistical inverse problem of estimating constituent curves from aggregated observations. This scenario arises frequently in fields like chemometrics, governed by the Beer-Lambert law, where the observed absorbance curve of a mixture is a weighted sum of the absorbance curves of its individual chemical constituents.

• The Model: The aggregated function $A(t)$ is modeled as:
  $$A(t) = \sum_{l=1}^{L} y_l \alpha_l(t) + \epsilon(t)$$
  Where:
  • $A(t)$ is the observed aggregated curve.
  • $\alpha_l(t)$ are the unknown component (constituent) curves to be estimated.
  • $y_l$ are known weights (concentrations).
  • $\epsilon(t)$ is additive Gaussian noise ($\mathcal{N}(0, \sigma^2)$).
• The Challenge: While traditional methods often rely on multivariate techniques (e.g., PCR, PLS) or Bayesian approaches, there is a need for robust methods within the Functional Data Analysis (FDA) framework that can handle both smooth functions and those with local features (discontinuities, spikes).

2. Methodology

The authors propose and implement two distinct estimation procedures within the FunctionalCalibration R package, utilizing basis expansion methods to solve the inverse problem.

A. Wavelet-Based Approach

• Basis: Utilizes an orthonormal wavelet basis ($\psi_{jk}$) to expand the component functions $\alpha_l(t)$.
• Transformation: The discrete model is transformed using the Discrete Wavelet Transform (DWT), represented by an orthogonal matrix $W$. This converts the problem into the wavelet domain: $D = \Gamma y + \epsilon^*$.
• Shrinkage: To estimate the wavelet coefficients $\Gamma$, a shrinkage rule is applied to the empirical coefficients. The paper specifically highlights a Bayesian shrinkage rule (Sousa, 2022) based on a mixture prior (point mass at zero and a logistic distribution).
  • The estimator is the posterior expectation of the coefficients.
  • Hyperparameters ($p, \tau, \sigma$) are estimated from the data (e.g., using the median absolute deviation for $\sigma$).
• Reconstruction: The final estimates of $\alpha_l(t)$ are obtained by applying the Inverse Discrete Wavelet Transform (IDWT) to the estimated coefficients.
• Advantage: Wavelets are particularly effective for functions with local features, such as discontinuities, spikes, or rapid oscillations.

B. Spline-Based Approach

• Basis: Utilizes B-spline basis functions ($B_k(t)$) to expand the component functions.
• Estimation: The model is formulated in matrix form $A(t) = D(t)\Theta + \epsilon$. The unknown coefficient matrix $\Theta$ is estimated using Ordinary Least Squares (OLS):
  $$\hat{\Theta} = (D(t)^T D(t))^{-1} D(t)^T A(t)$$
• Advantage: Splines are highly effective for estimating smooth functions.
• Limitation: As noted in the paper, spline expansions struggle to accurately reconstruct functions with sharp discontinuities.

3. Key Contributions

The primary contribution of this work is the development and release of the FunctionalCalibration R package (available on CRAN and GitHub), which operationalizes these FDA methods.

Key Features of the Package:

1. Dual Estimation Capabilities: Implements both the wavelet-based method (Sousa, 2024) and the spline-based method (Saraiva and Dias, 2009).
2. Simulation Tools: Includes a simulated_data() function generating 100 samples with known ground-truth component functions (one smooth, one discontinuous) and additive Gaussian noise for validation.
3. Core Functions:
   • functional_calibration_wavelets(): Estimates components using wavelets with options for various shrinkage methods (Bayesian, Universal, SURE, etc.) and wavelet families (Haar, Daubechies).
   • functional_calibration_splines(): Estimates components using B-splines with user-defined basis size.
   • plot_aggregated_curve(): Visualizes the weighted sum of constituent curves.
   • weight_estimation(): Solves the inverse problem of estimating weights ($y_l$) given the aggregated curve and known component curves.
4. Flexibility: Supports various wavelet families and shrinkage rules, allowing users to tailor the method to the specific characteristics of their data.

4. Results and Validation

The authors validated the package using a simulated dataset containing two component functions:

• $\alpha_1(x)$: A smooth function ($\sin(5x)e^{-x^2}$).
• $\alpha_2(x)$: A piecewise constant function with a sharp discontinuity.

Findings:

• Spline Performance: The spline-based method successfully recovered the smooth function $\alpha_1(x)$ but failed to accurately capture the discontinuity in $\alpha_2(x)$, resulting in smoothing artifacts (Gibbs phenomenon-like behavior).
• Wavelet Performance: The wavelet-based method successfully recovered both functions. It accurately captured the smooth behavior of $\alpha_1(x)$ and, crucially, preserved the sharp discontinuities in $\alpha_2(x)$.
• Conclusion: The results demonstrate that while splines are suitable for smooth data, wavelets are superior for aggregated data containing local features or non-smooth constituents.

5. Significance

• Practical Application: The package provides a vital tool for chemometricians and statisticians working with spectroscopic data (e.g., NIR), where determining the concentration and spectral signature of individual constituents from a mixture is critical.
• Methodological Advancement: By integrating wavelet-based shrinkage into the FDA framework for aggregated data, the package offers a robust solution for "ill-posed" inverse problems where data may be noisy or contain non-smooth features.
• Accessibility: Prior to this package, these advanced FDA methods were likely implemented in custom scripts. FunctionalCalibration standardizes these procedures, making them accessible to the broader R community for research and industrial application.

In summary, the paper presents a comprehensive software solution for decomposing aggregated functional data, demonstrating that the choice of basis (Wavelets vs. Splines) is critical depending on the smoothness of the underlying constituent curves.