Large Language Model Agent in Financial Trading: A Survey

1. Problem Statement

Financial trading is a high-stakes domain requiring the synthesis of vast amounts of heterogeneous data (news, financial reports, market metrics) and rapid decision-making. While Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) and demonstrated potential as autonomous agents in other fields (healthcare, education), their application in financial trading agents remains an emerging and under-explored area.

The core challenges addressed by this survey include:

• Architectural Diversity: There is no standardized framework for how LLMs should be integrated into trading systems (e.g., direct decision-making vs. signal generation).
• Data Integration: How to effectively combine numerical, textual, and visual data within an LLM context.
• Performance Validation: Determining if LLM agents can genuinely outperform professional traders and traditional quantitative models in backtesting.
• Limitations: Identifying current bottlenecks such as latency, reliance on closed-source models, and the lack of rigorous evaluation across diverse market conditions.

2. Methodology

The authors conducted a systematic review of 27 academic papers focusing on LLMs in financial trading. Seven of these explicitly used the term "agent" in their titles. The selection process involved multiple Google Scholar searches using keywords like "LLM for trading" and "GPT stock agent," followed by manual assessment to confirm relevance.

The survey categorizes the research into three primary dimensions:

1. Architecture: How the agent is structured.
2. Data: The types of inputs utilized.
3. Evaluation: Metrics and baselines used to measure performance.

3. Key Contributions & Technical Findings

A. Architectural Taxonomy

The survey classifies LLM trading agents into two main categories, further subdivided by mechanism:

1. LLM as a Trader (Direct Decision Making)
The LLM directly generates trading signals (BUY, HOLD, SELL).

• News-Driven: The most fundamental type. LLMs analyze news and macroeconomic updates to predict price movements. Some use zero-shot prompting, while others (e.g., FinGPT) use fine-tuning. Advanced versions employ memory modules to summarize news and reason about sentiment.
• Reflection-Driven: Inspired by cognitive science, these agents use Memory (raw observations) and Reflection (high-level insights aggregated from memory). Agents like FinMem and FinAgent retrieve relevant memories to refine decisions, mitigating hallucinations and improving context awareness.
• Debate-Driven: Utilizes a multi-agent framework where LLMs with different roles (e.g., mood agent, rhetoric agent) debate to enhance reasoning validity and robustness (e.g., TradingGPT, HAD).
• Reinforcement Learning (RL) Driven: Uses backtesting results as a reward signal to align LLM outputs with profitable behaviors. Frameworks like SEP and RL-based LG models use PPO (Proximal Policy Optimization) to refine predictions based on historical market trajectories.

2. LLM as an Alpha Miner (Indirect Decision Making)
The LLM does not trade directly but generates Alpha Factors (predictive signals) for downstream trading systems.

• Human-in-the-Loop: Frameworks like QuantAgent and AlphaGPT use an "inner-loop" (writer agent generates code/scripts, judge agent refines them) and an "outer-loop" (real-world testing provides feedback to improve the judge). This automates the resource-intensive process of alpha mining.

B. Data Modalities

The survey identifies four critical data types used by these agents:

1. Numerical Data: Stock prices, volumes, and technical indicators (e.g., MACD, RSI). LLMs must convert these to text strings. High-performing agents (e.g., FinAgent) integrate these to capture market dynamics.
2. Textual Data:
   • Fundamental Data: Financial reports (10-K, 10-Q) and analyst reports.
   • Alternative Data: News from major outlets (Bloomberg, WSJ) and Social Media (Twitter, Reddit). While social media is underutilized, it is crucial for events like the GameStop squeeze.
3. Visual Data: K-line charts and trading graphs. While historically difficult for LLMs, multimodal models (e.g., GPT-4v) are beginning to show promise in interpreting visual technical analysis.
4. Simulated Data: Synthetic environments used to test agent behavior under extreme pressure, bias, and ethical constraints (e.g., testing for insider trading tendencies).

C. Evaluation and Performance

• Strategies: Agents typically use Long-Short strategies, ranking stocks based on sentiment scores or predicted returns.
• Metrics: Standard financial metrics are used, including Cumulative Return, Annualized Return, Sharpe Ratio, and Maximum Drawdown. Signal-specific metrics like F1 Score, Accuracy, and Information Coefficient (IC) are also employed.
• Backtesting Results:
  • Most studies focus on the US and Chinese stock markets (2020–2024).
  • Performance: LLM agents demonstrated strong potential, achieving annualized returns of 15% to 30% over the strongest baselines (Rule-based, ML, and RL models) in backtesting.
  • Baselines: Comparisons were made against "Buy and Hold," Mean Reversion, Random Forest, LSTM, and Deep RL (PPO/DQN).

4. Limitations Identified

The authors highlight several critical gaps in current research:

• Architecture: Heavy reliance on closed-source models (GPT-3.5/4) raises data privacy concerns and limits customization. Most agents use in-context learning without fine-tuning, and inference latency makes them unsuitable for high-frequency trading (HFT).
• Data: Underutilization of social media data and visual data.
• Evaluation:
  • Backtesting periods are often short (median 1.3 years) and arbitrary.
  • Trading costs (slippage, fees) are rarely factored into performance metrics.
  • Lack of testing in diverse markets (derivatives, bonds, commodities).
• Interpretability: Few studies perform ablation studies to understand the specific reasoning processes or "personalities" of the agents.

5. Significance and Future Directions

This survey is the first comprehensive review of LLM agents specifically for financial trading. Its significance lies in:

• Establishing a Taxonomy: Providing a clear structural framework (Trader vs. Alpha Miner) for future researchers.
• Validating Potential: Demonstrating that LLMs can effectively process unstructured data to generate profitable trading signals, outperforming traditional baselines in backtests.
• Roadmap for Research: The authors propose future directions including:
  • Developing open-source, fine-tuned models for trading.
  • Integrating visual data and social media more robustly.
  • Conducting rigorous long-term backtesting that includes transaction costs.
  • Exploring simulated environments to study agent ethics, bias, and decision-making patterns under stress.

In conclusion, while LLM trading agents show immense promise in extracting insights from massive textual datasets, the field requires maturation in terms of model transparency, latency optimization, and rigorous, cost-aware evaluation before widespread deployment in production financial systems.