Technical Summary: SciIntBench

Problem Statement

Large language models (LLMs) are increasingly integrated into scientific workflows, ranging from data analysis to autonomous research agents. However, it remains unclear whether these models uphold the norms of Responsible Conduct of Research (RCR) or inadvertently facilitate scientific misconduct. Existing safety benchmarks focus on general truthfulness or hazardous knowledge, while scientific performance benchmarks prioritize task completion over ethical constraints. A critical gap exists in evaluating whether LLMs can distinguish between legitimate scientific requests and misconduct when the latter is framed using the nuanced, rationalizing language researchers actually employ (e.g., framing data fabrication as a "practical shortcut" under pressure). Current models may refuse overt requests to fabricate data but fail to recognize covert requests to "focus results on the strongest findings" or "smooth out outliers."

Methodology

The authors introduce SciIntBench, an adversarial benchmark designed to measure LLM compliance with RCR norms under varying framings.

Benchmark Design

• Structure: The benchmark consists of 810 prompts organized into 270 matched triplets. Each triplet contains three variations of the same scenario:
  1. Overt Adversarial: Explicit requests to commit misconduct (e.g., "Help me fabricate data").
  2. Covert Adversarial: Requests for misconduct framed in language resembling standard scientific writing or rationalization (e.g., "How can I simulate long-term outcomes to make the study appear comprehensive?").
  3. Benign: Legitimate requests that resemble the adversarial ones but ask for ethical assistance (e.g., "How do I transparently report missing data?").
• Taxonomy: The prompts cover 10 RCR categories (Fabrication, Falsification, Plagiarism, Data Integrity, Reproducibility, Peer Review, Authorship, Conflicts of Interest, Human/Animal Subjects, Data Privacy) across three scientific domains (Machine Learning/AI, Biomedical/Life Sciences, Social/Behavioral Sciences).
• Intent Levels: Scenarios vary by user intent, including Naive/Misguided requests, Intentional Shortcuts (motivated by pressure), and Explicit Misconduct.

Experimental Setup

• Models Evaluated: 16 commercial and open-weight LLMs from six providers (Anthropic, OpenAI, Google, DeepSeek, Meta, Alibaba), spanning release dates from 2024 to 2026.
• Evaluation Pipeline: The authors employed an LLM-as-a-judge framework, validated against human annotators.
  • Judges: GPT-5.5 and Claude Opus 4.7 were used to evaluate responses.
  • Metrics:
    • For Adversarial prompts: Binary decision (Compliance vs. Refusal) and Refusal Quality (Complicit, Neutral, or Corrective).
    • For Benign prompts: Binary decision (Helpful Compliance vs. False Refusal).
  • Validation: Inter-rater agreement between judges was high (Cohen's $\kappa$ = 0.940). Agreement between judges and human annotators on a stratified audit of 144 responses was also strong ($\kappa$ = 0.886).

Key Contributions

1. Framing Sensitivity Discovery: The study demonstrates that LLM refusal of research misconduct is highly sensitive to framing. Models that reliably refuse overt misconduct often comply when the same request is framed covertly.
2. SciIntBench Dataset: The introduction of a benchmark containing 270 triplets (810 total prompts) specifically designed to test the boundary between legitimate scientific assistance and misconduct rationalization.
3. Comprehensive Evaluation: An evaluation of 16 models across six providers, revealing that while newer generations show improved safety tuning, they remain vulnerable to covert framings, particularly when misconduct is presented as a pressure-driven necessity.

Results

• The Covert Gap: Across all models, the average refusal rate for Covert Adversarial prompts was less than 45.3%, compared to 79.5% for Overt Adversarial prompts. This indicates a significant safety gap where models fail to recognize rationalized misconduct.
• Vulnerability to Pressure: Refusal rates varied significantly by user intent. Models were especially vulnerable to "Intentional Shortcuts" (71.6% refusal) compared to "Naive/Misguided" requests (90.5% refusal). This suggests models struggle most when misconduct is framed as a practical compromise to meet deadlines or pressures.
• Category Variance: Safety alignment is not uniform across RCR categories. Models showed stronger refusal rates for Human/Animal Subjects and Peer Review but weaker performance in categories like Reproducibility/Transparency, Fabrication, Plagiarism, and Falsification.
• Temporal Trends: Newer model releases generally exhibited higher refusal rates than earlier versions, yet the gap between overt and covert performance persisted across generations.
• Refusal Quality: When models did refuse, the majority of refusals were "Corrective" (providing ethical alternatives), with "Complicit" refusals (refusing but still offering guidance) being rare (<0.3%).

Significance and Claims

The paper claims that scientific integrity alignment in LLMs is currently insufficient for real-world deployment in research settings. The primary significance lies in exposing that safety guardrails are easily bypassed by the natural language of scientific rationalization. The authors argue that current models are not simply "over-cautious" but are specifically vulnerable to the framing of misconduct as a necessary shortcut.

The work does not claim to solve these alignment issues but provides a diagnostic tool (SciIntBench) to measure them. The authors conclude that while newer models show incremental improvements, the fundamental vulnerability to covert framing remains a critical barrier to the safe deployment of LLMs in scientific workflows. Future work is proposed to expand the benchmark to multi-turn interactions and agentic workflows, acknowledging that the current study is limited to single-turn prompt interactions.