Technical Summary: Enhancing Language Models for Robust Greenwashing Detection

1. Problem Definition

Sustainability reports are fundamental to Environmental, Social, and Governance (ESG) assessment, yet their reliability is frequently compromised by "greenwashing"—the use of vague, inflated, or strategically ambiguous language to obscure a lack of substantive action. While Natural Language Processing (NLP) systems are widely employed to analyze these disclosures, current models exhibit significant fragility. They typically rely on surface-level lexical and stylistic patterns, resulting in strong performance on in-distribution data but poor generalization to unseen ESG categories or shifted linguistic formulations.

A critical limitation identified is that ESG analysis is often framed as a flat prediction problem, optimizing a single objective without structuring the latent representation space. Consequently, models fail to explicitly encode critical distinctions such as degrees of actionability, commitment, or evidential grounding. Furthermore, simply increasing model capacity (scaling) does not consistently improve robustness in this setting, as companies can alter reporting focus without changing underlying practices, requiring models to look beyond surface cues.

2. Methodology

The authors propose a parameter-efficient framework that structures the latent spaces of Large Language Models (LLMs) by integrating contrastive learning with an ordinal ranking objective. The framework is applied directly to Low-Rank Adaptation (LoRA) adapters to ensure continuity of the learned latent space.

Core Components:

• Structured Representation Learning: The framework moves beyond flat classification by organizing claims along an ordinal continuum of actionability: indeterminate (aspirational), planning (future-oriented), and implemented (verifiable).
• Contrastive Learning: A supervised contrastive objective clusters semantically related claims while separating unrelated ones, enforcing similarity structure in the latent space.
• Ordinal Ranking Objective: To address the limitation of standard contrastive learning, an ordinal margin loss is introduced. This enforces that claims with higher actionability (e.g., implemented) remain closer to the anchor than those with lower actionability (e.g., indeterminate) by a fixed margin, aligning the representation geometry with the action scale.
• Gated Feature Modulation: A per-sample gating mechanism dynamically emphasizes task-relevant dimensions to filter out noise and boilerplate language common in sustainability disclosures.
• MetaGradNorm for Loss Balancing: To stabilize the multi-objective optimization (balancing contrastive, ordinal, and base losses), the authors introduce MetaGradNorm. This strategy adaptively balances competing gradients and loss contributions based on task difficulty, ensuring robust convergence without manual tuning of loss weights.

Training Procedure:

The training occurs in two stages using the same LoRA adapters:

1. Structured Representation Learning: Optimizing contrastive and ordinal objectives with gated representations and MetaGradNorm to shape the latent space.
2. Task-Specific Fine-Tuning: Adapting the same adapters to downstream supervision without reinitialization, preserving the learned structure.

3. Key Contributions

The paper outlines four primary contributions:

1. Structured PEFT Framework: A novel integration of contrastive and ordinal representation learning directly into low-rank adapters for robust ESG claim analysis.
2. Gated Feature Modulation: A mechanism to dynamically filter noise and strategic language in sustainability disclosures by emphasizing task-relevant dimensions.
3. MetaGradNorm: An optimization strategy that adaptively balances competing objectives to stabilize multi-loss training in structured representation settings.
4. Comprehensive Empirical Evaluation: A rigorous assessment of robustness and generalization trade-offs across multiple open-source LLMs (T5, LLaMA-3-8B, Mistral-7B, Gemma-7B, DeepSeek-V3-7B, Qwen2.5-7B) and ESG categories, identifying key scaling and structural limitations.

4. Experimental Results

The framework was evaluated on the A3CG dataset using a cross-category generalization protocol (disjoint training and test categories).

• Robustness Gains: The proposed framework (referred to as COGLM in the results) consistently improved robustness across all tested LLMs. Notably, it improved unseen category performance by 4 to 5 F1 points compared to standard baselines.
• Model Scale vs. Structure: Structured fine-tuning on medium-scale models (7–8B parameters) yielded higher robustness than 70B-parameter variants and proprietary systems (e.g., GPT-4o, Claude 3.5 Sonnet). This suggests that representation structure is more critical than model scale for this task.
• Trade-offs: While T5 achieved the strongest overall performance, the study revealed a trade-off between representational rigidity and generalization. The most effective configuration was not always the most complex; optimal performance emerged from the balanced combination of components rather than the maximal application of all objectives.
• Latent Space Analysis: Visualization (PCA/UMAP) confirmed that the structured framework organizes the latent space more effectively, creating tighter clusters for operational categories (e.g., emissions control) and better separating actionable from non-actionable content compared to LoRA-only baselines.

5. Significance and Limitations

Significance:
The paper argues that for ESG analysis, representation structure is more critical than model scale. By explicitly encoding ordinal relationships and actionability during fine-tuning, models can better capture underlying intent and distinguish rhetorical patterns from credible actions. The work demonstrates that parameter-efficient adaptation with structured objectives provides a superior inductive bias for transferable action-level representations.

Limitations:
The authors maintain a modest stance regarding the scope of their findings:

• Benchmark Constraints: The study is constrained by the A3CG benchmark, which is English-only and limited to a specific taxonomy of ESG aspects and three ordinal action levels.
• Non-Monotonic Gains: Improvements are not monotonic across all folds and models; additional components increase the hyperparameter surface and can reduce generalization if miscalibrated.
• Analysis Depth: The representation analysis relies on low-dimensional projections and clustering proxies, which do not fully characterize decision boundary transfer.
• Scope: The focus is on PEFT with a fixed training protocol; alternative objectives or data augmentation strategies were not explored.

The authors conclude that robust structured fine-tuning requires careful, staged hyperparameter control, and that gains emerge from the precise joint calibration of structural objectives and their optimization landscapes rather than methodological complexity alone.