Greener Than Humans? Environmental Attitudes in Large Language Models

Technical Summary: Greener Than Humans? Environmental Attitudes in Large Language Models

Problem Statement
Large Language Models (LLMs) are increasingly deployed in sustainability-related decision support, reporting, and public communication. However, there is a lack of systematic evidence regarding the environmental attitudes—specifically cognition, affect, and behavioral recommendations—embedded within these models. While LLMs are framed as tools for advancing Sustainable Development Goals (SDGs), concerns persist regarding their potential to reproduce biases, hallucinate facts, exhibit "stochastic parrot" behaviors, and display sycophancy (adapting answers to user beliefs rather than facts). The opacity of training data and alignment procedures complicates the evaluation of the value judgments reflected in model outputs. This study addresses the critical gap in understanding what LLMs "know" about environmental issues, how they evaluate them, and what actions they implicitly or explicitly recommend.

Methodology
The authors developed a structured benchmark to evaluate environmental attitudes across 31 widely used proprietary and open-weight LLMs (including models from the US, EU, and China). The framework draws on instruments from the German Federal Environment Agency's biennial "Environmental Awareness in Germany" (UBS) survey, which assesses the German adult population.

The study operationalized three interrelated dimensions:

1. Environmental Cognition: 17 items measuring factual knowledge and assessment of environmental problems (excluding narrow fact-based trivia).
2. Environmental Affect: 7 items measuring emotional involvement and concern regarding environmental conditions.
3. Behavioral Recommendations: 17 items reframed as recommendations (since LLMs do not perform physical actions) and 7 quantitative questions based on the German Environmental Agency's CO2 calculator to estimate potential CO2 footprint reductions.
4. Willingness to Pay (WTP): 11 questions assigning monetary values to the mitigation of environmental impacts (CO2, nitrogen, phosphorus, water conservation) to evaluate trade-offs between financial and environmental goals.

Experimental Design and Robustness:

• Baseline: Models were queried via APIs or self-hosted instances with a temperature of 0 to minimize stochastic variation. No prompt engineering was applied to preserve comparability with human survey data.
• Contextual Sensitivity: The study tested robustness through persona-based prompting (assigning roles like "CFO" or "NGO advocate") and sycophancy testing (users specifying their own identity, e.g., "I am a...").
• Metrics: The authors computed Euclidean distances in a two-dimensional Affect-Cognition space to quantify steerability and sycophancy. They also constructed normalized indices (0–10 scale) for comparison against the 2024 UBS population survey data.
• Exclusions: Five models were excluded for failing factual tests, and three were excluded for order-dependent inconsistencies.

Key Results

1. "Greener" Baseline: 19 out of 31 models exhibited higher levels of environmental affect and cognition than the average German survey respondent. Most models aligned with the "committed," "individually sustainable," and "ambivalent" attitude clusters, showing a strong preference for progressive environmental views over "populist" or "neoliberal opposition" clusters.
2. Behavioral Impact: Models recommended behaviors with the potential to reduce individual CO2 emissions by approximately 4 to 5 tons of CO2e per year (top performers), though this varies based on the user's starting point. Grok models ranked in the lower third for recommended CO2 reductions.
3. Lack of Systematic Correlation with Metadata: No systematic relationship was found between sustainability-oriented responses and model origin (country), size (Small to XL), or release context. This suggests that post-training alignment and instruction tuning play a more dominant role than base model architecture or origin.
4. Steerability and Sycophancy: Models demonstrated significant contextual sensitivity. Assigning specific roles (e.g., "economic liberal") shifted affect and cognition scores. Models exhibited sycophantic shifts, mirroring user-specified ideological positions. Grok 4.1 showed the highest sensitivity to role prompting, while GPT 5.2 was the least sensitive.
5. Willingness to Pay: Models tended to assign higher economic costs to emissions than the "should cost" valuation, though most estimates were lower than the German Federal Environment Agency's calculated climate damage costs (300 Euro/ton). WTP showed weak correlation with other attitude categories.

Key Contributions

• Evaluation Framework: The paper introduces a reusable benchmark and testing pipeline for assessing sustainability-related value alignment in LLMs, adapting established social science survey instruments for AI evaluation.
• Empirical Assessment: It provides the first systematic, large-scale empirical assessment of environmental cognition, affect, and behavioral recommendations across a diverse set of 31 LLMs.
• Governance Implications: The study derives implications for policymakers, developers, and users, highlighting the need for governance, transparency, and critical oversight as AI systems become embedded in sustainability transformations.

Significance and Claims
The authors conclude that while LLMs currently align more closely with environmentally progressive attitudes than the average human respondent, this "greenness" does not guarantee reliability or transformative capacity. The study emphasizes that:

• Manipulability: Model outputs are susceptible to manipulation via persona prompting and user context, raising concerns about normative reliability in real-world deployments.
• Limitations of "Paper" Greenness: LLMs may appear environmentally friendly in text but lack the capacity to implement transformative strategies or account for structural barriers (e.g., institutional decisions) that individual actions cannot overcome. They may also inadvertently shift responsibility onto individuals rather than corporate actors.
• Data Gaps: LLMs encode incomplete information, often missing indigenous knowledge or non-digitized data, and rely on static training data that may not reflect current scientific consensus or future scenarios.

The paper asserts that LLMs are useful tools for factual sustainability questions but require critical evaluation of their recommendations. It calls for improved data curation (including diverse perspectives), value-sensitive design in development, and the integration of environmental risks into digital regulations (e.g., EU AI Act). The authors caution that without critical oversight, the "will-to-please" and sycophantic tendencies of LLMs could undermine the objective assessment of sustainability challenges.