A Functional Data Framework For Analyzing Shapes and Textures in Images

Technical Summary: A Functional Data Framework For Analyzing Shapes and Textures in Images

Problem Statement
The paper addresses the challenges of statistical image analysis, specifically the limitations of pixel-based representations. While images are increasingly available for applications ranging from medical diagnosis to agronomy, treating them as high-dimensional matrices with complex correlation structures renders classical statistical methods unsuitable. Furthermore, dominant approaches like Convolutional Neural Networks (CNNs), while effective, suffer from high computational costs and a lack of interpretability. The authors argue that as acquisition technologies produce more detailed images, the computational burden of pixel-based models will increase, necessitating the development of "frugal" statistical methods that respect the continuous nature of object contours and textures.

Methodology
The proposed framework shifts from discrete pixel grids to a Functional Data Analysis (FDA) perspective, modeling images as observations of continuous random functions. The methodology proceeds through the following stages:

1. Image Segmentation and Contour Extraction:
   The process begins by identifying a single object within an image $I$ to obtain its silhouette $I'$. The object's boundary is represented as a closed planar curve $C: [0, 1] \to \mathbb{R}^2$. Using algorithms like Otsu's thresholding and the marching squares method, the discrete silhouette is converted into a functional contour.

2. Shape Analysis (Deformation Alignment):
   Following the work of Moindjié et al. (2025), the contour $C$ is decomposed into a shape component $\tilde{C}$ (in the sense of Kendall) and deformation variables: uniform scaling ($\rho$), translation ($c$), rotation ($O$), and reparametrization ($\gamma$). The authors estimate these variables to align and center the curves, resulting in a shape variable $S$ that is invariant to these transformations. This allows for the statistical analysis of the object's form independent of its position or size.

3. Texture Representation via Star-Shaped Domains:
   A core innovation is the treatment of texture (interior color variation). The framework assumes the object's interior domain $D$ is star-shaped with respect to a center point $c$. Under this hypothesis, the authors define a mapping $\phi$ that transforms the unit disk $\mathbb{D}$ to the object's interior $D$.

   • This mapping allows the texture function $T^*(u) = I(u)$, originally defined on the irregular domain $D$, to be re-parameterized onto the unit disk $\mathbb{D}$ (or a square $[0,1]^2$ via a bijective transformation $g$).
   • The resulting texture $T(v)$ is a random surface defined on a common domain, enabling the application of standard functional data techniques.

4. Joint Statistical Modeling:
   The framework constructs a joint functional variable $Z$ combining the texture $T \circ g$ and the aligned shape $\tilde{C}$. For supervised classification, the authors employ linear models where the response $Y$ is regressed on $Z$.

   • Since the shape space is a non-linear manifold, the shape component is projected onto the tangent space at the Fréchet mean $\mu$.
   • To handle the ill-posed nature of regression with functional covariates, the authors utilize Multivariate Functional Principal Component Analysis (MFPCA) and Multivariate Functional Partial Least Squares (MFPLS) to estimate regression coefficients.

Key Contributions

• Frugal Representation: The paper introduces a sparse representation of images that explicitly separates shape (contour) and texture (interior color), avoiding the high dimensionality of pixel-based matrices.
• Star-Shaped Mapping: It provides a theoretical and practical mapping for texture analysis under the star-shaped domain assumption, allowing for the integration of interior color variations without requiring annotated landmarks.
• Joint Functional Framework: It extends previous contour-only analysis (Moindjié et al., 2025) to a joint analysis of contours and textures, treating both as continuous functional variables.
• Interpretability: By using linear models on functional data, the approach offers a more interpretable alternative to deep learning black boxes.

Results
The framework was evaluated on a real-world dataset of 2,148 pistachio images (two varieties: Kirmizi and Siirt) from Singh et al. (2022). The study involved:

• Data Augmentation: Random rotations and zooming were applied to test robustness against deformations.
• Comparison: The proposed method ("OUR") was compared against a baseline competitor ("COMP") that treated images as random surfaces (scalar-on-image regression) without explicit shape/texture separation.
• Performance: Using 10-fold cross-validation, the proposed framework achieved significantly higher accuracy and F1 scores.
  • OUR (MFPCA): 77% Accuracy, 74% F1.
  • OUR (MFPLS): 76% Accuracy, 73% F1.
  • COMP (MFPCA): 63% Accuracy, 42% F1.
  • COMP (MFPLS): 57% Accuracy, 54% F1.
    The baseline competitor performed near random chance levels, whereas the proposed framework demonstrated robust discrimination capabilities.

Significance and Claims
The authors claim that this work establishes a new paradigm for statistical image analysis by explicitly modeling the primary aspects of images (shape and texture) as continuous functions rather than discrete pixels. The significance lies in:

1. Efficiency: It offers a computationally lighter ("frugal") alternative to CNNs and high-dimensional pixel models.
2. Robustness: The explicit handling of deformation variables (rotation, scaling) allows the model to perform well even when objects are not perfectly aligned.
3. Applicability: The star-shaped assumption is presented as a weak condition satisfied by many objects in image analysis (e.g., biological cells, leaves, organs), making the method broadly applicable.

The paper modestly notes that the reliance on the star-shaped domain assumption is a limitation for non-star-shaped objects (like a horseshoe) and suggests future work could explore convex-hull extensions or the incorporation of background colors to handle more complex geometries.