TSAssistant: A Human-in-the-Loop Agentic Framework for Automated Target Safety Assessment

1. Problem Statement

Target Safety Assessment (TSA) is a critical, early-stage process in drug discovery used to evaluate the potential adverse effects of modulating a biological target in humans. It involves synthesizing heterogeneous evidence from five distinct domains:

1. Genetic: Human variants linking target perturbation to phenotypes.
2. Transcriptomic: Tissue-specific expression profiles.
3. Target Homology: Sequence/structural similarity to other proteins (off-target risks).
4. Pharmacological: Effects of known drugs modulating the same target.
5. Clinical: Adverse event signals from clinical registries.

Current Challenges:

• Manual & Non-Standardized: The process is largely manual, leading to inconsistent content composition, varying levels of depth, and poor reproducibility.
• Scalability Issues: As evidence bases grow, synthesizing data across large target portfolios becomes time-consuming and prone to knowledge gaps.
• Limitations of Current AI: General Large Language Models (LLMs) lack consistency and reproducibility. While tool-augmented LLMs and multi-agent systems exist, they often lack the traceability, human oversight, and fine-grained control required for high-stakes pharmaceutical safety decisions. Fully autonomous systems risk hallucination and error propagation, which are unacceptable in regulated environments.

2. Methodology: TSASSISTANT Framework

TSASSISTANT is a multi-agent, human-in-the-loop (HITL) framework designed to automate TSA report drafting while retaining expert authority. It employs a hybrid architecture combining soft constraints (prompts) and hard constraints (programmatic enforcement).

A. Hierarchical Instruction Architecture

To manage complexity and ensure stability, agent behavior is governed by a three-layer instruction hierarchy:

1. System Prompts (Layer 1): Define stable coordination logic, agent roles, and output structures. These are target-agnostic and rarely modified.
2. Domain Skill Modules (Layer 2): Encapsulate section-specific expertise (e.g., retrieval strategies, database weighting, writing guidelines) as reusable, versioned modules. This allows toxicologists to update specific domain knowledge without breaking the overall system logic.
3. Runtime User Instructions (Layer 3): Inject project-specific constraints (e.g., specific disease indications, proprietary data) at the highest priority, overriding defaults as needed.

B. Agent Pipeline & Decomposition

The system decomposes report generation into a coordinated pipeline of specialized subagents:

• 5 Research Subagents: Each is assigned exactly one TSA evidence domain (Genetic, Transcriptomic, Homology, Pharmacological, Clinical). They operate in context isolation to prevent error propagation between sections.
• 3 Synthesis Subagents: Integrate findings across sections to produce the Executive Summary, Conclusion, and Integrated Risk Assessment.

C. Programmatic Enforcement Layer

To address the "softness" of LLM prompts, TSASSISTANT implements hard constraints:

• Structured Tool Interfaces (MCP): Subagents do not access raw APIs. Instead, they use standardized tools connected to curated databases (e.g., Ensembl, UniProt, GWAS Catalog, ClinicalTrials.gov). These tools perform multi-step data engineering (filtering, de-duplication, aggregation) before returning structured, analysis-ready data to the LLM.
• Execution Hooks:
  • Pre-execution: Security checks, memory injection, and input validation.
  • Post-execution: Citation validation via natural language inference, output compression, and integrity checks.
  • Runtime: Continuous monitoring for evidence verification and cross-section consistency.
• Persistent Memory:
  • Tool Memory: A durable, queryable store of all retrieved evidence with provenance metadata, decoupling knowledge from the LLM's ephemeral context window.
  • Agent Memory: Compressed factual representations of completed sections injected into downstream agents based on a dependency graph.

D. Interactive Refinement Loop (Human-in-the-Loop)

The system operates in an iterative cycle where human experts intervene at the section level:

• Users can manually edit content, append new information, upload additional sources/figures, or re-invoke specific subagents for targeted revision.
• The system maintains conversational memory across iterations to adapt retrieval strategies and prompt templates based on user feedback.
• Philosophy: LLMs handle evidence synthesis and drafting; human experts retain final decision authority and validation.

3. Key Contributions

1. Section-Based Multi-Agent Pipeline: A novel architecture where specialized subagents independently ground and cite each evidence domain, enabling modular task distribution and targeted revision without regenerating the entire report.
2. Three-Layer Instruction Architecture: A design that separates coordination logic, domain skills, and user intent, allowing for independent updates to domain knowledge without destabilizing the system.
3. Programmatic Enforcement Layer: A robust framework using execution hooks and persistent memory stores to enforce output validity, traceability, and controlled information sharing, moving beyond simple API wrappers to domain-engineered workflows.
4. Interactive Refinement Loop: A HITL design that integrates human oversight at the section level, allowing for early error detection and progressive personalization of the workflow.

4. Results & Evaluation

The paper outlines a systematic evaluation framework applied to a set of 50+ therapeutic targets. While comprehensive quantitative results are noted as pending for future revisions, the evaluation focuses on four dimensions:

• Factual Consistency: Verifying that claims match cited source data (categorizing claims as supported, unsupported, contradicted, or hallucinated).
• Evidence Completeness: Ensuring all key safety-relevant sub-topics within the five evidence domains are covered.
• Structural Alignment: Checking conformity to internal TSA templates, writing styles, and evidence prioritization.
• Evidence Traceability: Measuring the proportion of claims with verifiable inline citations to primary sources (critical for regulatory compliance).

The framework is designed to reduce drafting time and the number of refinement iterations required per section compared to manual processes.

5. Significance

• Regulatory Compliance & Safety: TSASSISTANT addresses the critical need for traceability and human accountability in pharmaceutical safety. It ensures that AI augments rather than replaces expert judgment, meeting regulatory requirements for safety decisions.
• Scalability: By automating the mechanical burden of evidence synthesis and report drafting, the system allows toxicologists to scale assessments across large target portfolios without sacrificing depth or consistency.
• Scientific Rigor: The use of "hard constraints" (programmatic hooks, structured tools) mitigates the risk of LLM hallucinations and error amplification, which are common pitfalls in autonomous scientific agents.
• Future of AI in Pharma: TSASSISTANT serves as a blueprint for applying agentic AI in high-stakes, regulated domains, demonstrating how to balance automation efficiency with the necessity of expert oversight and auditability.

In summary, TSASSISTANT represents a significant step toward AI-augmented pharmaceutical safety science, providing a structured, reproducible, and human-centric framework for one of the most critical steps in drug discovery.