Technical Summary: Identity-Consistent Expression Fields (ICEF)

Problem Statement

The paper addresses a critical limitation in few-shot dynamic Neural Radiance Fields (NeRF) for facial expression synthesis. While existing methods (e.g., FDNeRF) successfully reconstruct 3D faces from a small number of images and enable expression editing, they typically rely on warping a single learned feature volume conditioned on target expression parameters.

The authors observe that this approach lacks explicit constraints to preserve identity-specific details (such as skin texture and fine geometric structure) when the model is driven toward expressions that are far from the few-shot input set (i.e., during extrapolation). Consequently, the warped feature volume tends to "drift," inadvertently altering identity-specific attributes alongside the intended expression changes. This results in artifacts and a loss of identity consistency, particularly when extrapolating to novel expressions.

Methodology: ICEF Framework

The proposed Identity-Consistent Expression Fields (ICEF) framework tackles this issue through three core mechanisms: explicit disentanglement, a targeted regularizer, and a confidence-weighted warping strategy.

1. Static-Dynamic Disentanglement

Unlike prior methods that warp a single entangled feature volume, ICEF decomposes the radiance field into two distinct components:

• Static Canonical Field ($F_{static}$): Encodes identity-specific appearance and geometry in a neutral (canonical) space. Crucially, this field receives only deformed coordinates and viewing directions, never raw expression parameters, ensuring identity details are not directly modulated by expression.
• Dynamic Deformation Field ($D_\phi$): Maps observation-space points to the canonical space based on expression parameters ($\beta_{exp}$). All expression-dependent variation is forced to flow through this explicit deformation field.

The rendering equation is defined as:
$$c(x, d), \sigma(x) = F_{static}(D_\phi(x, \beta_{exp}), d)$$

2. Identity-Preservation Regularizer

To prevent the deformation field from altering identity-specific regions, ICEF introduces a regularization term ($L_{id}$) based on a coarse facial region mask ($M(x)$).

• Mechanism: The mask, derived from projected facial landmarks, identifies high-expression-relevance regions (mouth, eyes, brows) and low-expression-relevance regions (forehead skin away from brows, cheeks, nose bridge).
• Constraint: The regularizer penalizes deformation magnitude in low-relevance regions:
  $$L_{id} = \mathbb{E}_x [(1 - M(x)) \cdot \|D_\phi(x, \beta_{exp}) - x\|^2]$$
  This encourages the network to leave identity-specific canonical geometry and appearance in non-expression regions effectively unchanged.

3. Confidence-Weighted Warping

To mitigate artifacts when extrapolating to expressions far from the training distribution, ICEF employs a confidence-weighted blending mechanism:

• Extrapolation Distance ($\delta$): Calculated as the minimum distance between the target expression and the observed few-shot expressions.
• Confidence Weight ($\kappa(\delta)$): Defined as $\exp(-\delta/\lambda)$, this weight decreases as the target expression moves further from the training data.
• Blending: The final deformation is a blend of the full-capacity learned network ($D_\phi$) and a lower-capacity, smooth linear baseline ($D_{lin}$, a first-order Taylor approximation):
  $$D_{final} = \kappa(\delta) \cdot D_\phi + (1 - \kappa(\delta)) \cdot D_{lin}$$
  This ensures that for far-extrapolated expressions, the model relies more on the smoother, safer linear deformation to avoid aggressive, low-confidence warps.

Key Contributions

The paper outlines four primary contributions:

1. Explicit Disentanglement: A static-identity/dynamic-expression decomposition for few-shot dynamic NeRF, contrasting with prior single-feature-volume warping approaches.
2. Identity-Preservation Regularizer: A constraint that limits expression-conditioned deformation to expression-relevant facial regions, directly addressing identity drift during extrapolation.
3. Confidence-Weighted Warping: An adaptive mechanism that scales deformation capacity based on the proximity of the target expression to the observed few-shot distribution.
4. Evaluation Protocol: A novel protocol that stratifies evaluation by extrapolation distance and measures identity-consistency metrics (facial recognition embedding similarity) alongside standard rendering quality metrics (PSNR, SSIM, LPIPS).

Results and Evaluation Claims

The paper proposes an evaluation framework rather than presenting a full empirical study with numerical results in this text. However, it articulates specific expectations regarding performance:

• Far-Extrapolation Tier: The authors expect ICEF's advantages to be most pronounced here. The confidence-weighted mechanism is anticipated to reduce reliance on the learned network in favor of the smoother linear fallback, while the regularizer strictly constrains deformation, thereby preserving identity better than baselines.
• Near-Extrapolation Tier: The performance gap is expected to be more modest, as baseline methods are already well-supported by nearby training expressions.
• Metrics: The evaluation is designed to quantify identity drift as a function of extrapolation distance, hypothesizing that explicit disentanglement improves identity preservation specifically in the challenging few-shot, far-extrapolation regime.

Significance and Limitations

Significance:
ICEF aims to solve the "identity-drift" failure mode observed in prior few-shot dynamic NeRF methods. By explicitly separating identity appearance from expression deformation and introducing safeguards for extrapolation, the framework targets the specific challenge of rendering novel expressions while maintaining a consistent identity from limited data.

Limitations:
The authors acknowledge that the facial region mask ($M(x)$) used for regularization is a coarse, fixed prior derived from landmarks, rather than a learned or identity-specific segmentation. Consequently, it may not perfectly capture expression-relevant regions for all facial structures. The paper suggests that learning $M(x)$ jointly with the model or conditioning it on identity-specific geometry is a natural direction for future refinement.