🤖 AI

Dynamic Objective Selection with Safeguards and LLM Oversight for Financial Decision-Making

This paper introduces DOSS, a learning-based framework that dynamically selects interpretable financial optimization objectives from recent market data while employing confidence-aware gating and LLM oversight to ensure stability, prevent excessive switching, and enforce safety constraints.

Original authors: Keigo Sakurai, Takahiro Ogawa, Miki Haseyama, Anjyu Anan, Kei Nakagawa

Published 2026-06-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Keigo Sakurai, Takahiro Ogawa, Miki Haseyama, Anjyu Anan, Kei Nakagawa

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a ship navigating a vast, ever-changing ocean. Your goal is to reach a destination (making money) as efficiently as possible. However, the ocean isn't static; sometimes it's calm and sunny, other times it's stormy and dangerous.

In the world of finance, "captains" (algorithms) usually pick one rulebook for the whole journey. They might decide, "We will always sail as fast as possible," or "We will always avoid the roughest waves." The problem is that a rulebook perfect for sunny days might sink the ship in a storm, and a rulebook for storms might be too slow when the weather is nice.

This paper introduces a new system called DOSS (Dynamic Objective Selection with Safeguards). Think of DOSS as a smart, adaptive co-pilot that doesn't just steer the ship, but constantly rewrites the rulebook based on what the water looks like right now.

Here is how it works, broken down into simple concepts:

1. The Menu of Options (The "What")

Instead of trying to invent a brand-new rulebook on the fly, DOSS picks from a small, pre-approved menu of three strategies:

The Speedster: Focuses purely on going fast (maximizing returns). Good for calm, rising markets.
The Safety First: Focuses on avoiding big drops (minimizing downside risk). Good for stormy, falling markets.
The Balanced Captain: Tries to get a good speed while keeping the ship steady (risk-adjusted).

2. The "No Crystal Ball" Rule (The "How")

Many systems try to guess the future or detect hidden "regimes" (like predicting a storm before it happens). The authors say this is often noisy and unreliable.
Instead, DOSS looks only at what just happened. It takes a snapshot of the recent past (like the last few weeks of waves) and asks: "Given this specific pattern, which of our three rulebooks worked best in the past?"
It uses a "rolling window," meaning it constantly updates its training data with the most recent history, ensuring it never cheats by looking at the future.

3. The "Confidence Meter" and the Safety Net

This is the most critical part. Sometimes, the ocean looks confusing, and the co-pilot isn't sure which rulebook to pick.

The Confidence Meter: DOSS calculates a score. If it's 100% sure, it picks the best option. If it's only 50% sure, it gets nervous.
The Safety Net (Fail-Safe): If the confidence meter is too low, DOSS refuses to make a risky guess. Instead, it automatically switches to the "Safety First" rulebook (the conservative default). It's like saying, "I don't know if we should speed up or slow down, so let's just steer steady to avoid crashing."

4. The "Human-in-the-Loop" Supervisor (The LLM)

The paper also suggests adding a Large Language Model (LLM), but with a very strict job description.

What it is NOT: It is not allowed to invent new strategies or change the menu.
What it IS: It acts as a supervisor. It looks at the co-pilot's choice and the confidence score. It can either say, "Approved, go ahead," or "Override! Switch to the Safety First mode."
The Guardrails: If the co-pilot is switching too often (which causes instability and high costs), the system has a hard rule to stop the switching and stick to the safe default. This keeps the ship from jerking around uncontrollably.

5. The Results: Why It Matters

The authors tested this on a public financial benchmark (a simulated ocean).

Static Captains: Ships that stuck to one rulebook (always fast or always safe) did okay, but missed opportunities.
The "Oracle": A magical ship that knew the future and picked the perfect rulebook every time. It did the best, but it switched rules constantly, which is impossible in real life.
DOSS: The DOSS ship didn't do quite as well as the magical Oracle, but it did significantly better than the static ships. Crucially, it didn't switch rules as frantically as the Oracle, keeping the journey stable.
The LLM Test: When they tried letting an AI just "guess" the rulebook directly without the safeguards, it performed worse and switched more often. The DOSS system with its safety nets was more reliable.

The Bottom Line

The paper argues that in financial decision-making, adaptability is good, but chaos is bad.
DOSS solves this by being a system that knows when to change tactics and, more importantly, knows when to stop guessing and play it safe. It combines a learning algorithm that picks the best strategy from a small menu with strict "guardrails" (confidence checks and a supervisor) to prevent the system from making wild, costly mistakes when it's unsure.

Technical Summary: Dynamic Objective Selection with Safeguards and LLM Oversight for Financial Decision-Making

Problem Statement
Financial decision-making tasks, such as stock recommendation and portfolio allocation, rely on optimizing an objective function (e.g., maximizing return, minimizing downside risk, or optimizing risk-adjusted metrics) to select trades. While the choice of this objective significantly impacts realized performance, market conditions are non-stationary; a fixed objective is often suboptimal across different regimes. Existing approaches that attempt to switch objectives based on latent regime estimates often suffer from noisy or delayed signals, leading to mistimed switches. Furthermore, frequent switching can induce operational instability and excessive turnover. The core challenge is to select the most appropriate decision-relevant objective function at each time point directly from observable market features, without relying on unobservable regime variables, while ensuring the selection process is robust, forward-looking, and operationally stable.

Methodology: DOSS
The authors propose DOSS (Dynamic Objective Selection with Safeguards), a learning-based framework that treats objective selection as a policy problem. The method operates on a bounded, predefined set of candidate objectives $\mathcal{O}$ (e.g., Return-seeking, Loss-averse, Risk-adjusted) and avoids introducing intermediate latent regime variables.

The framework consists of four key components:

Rolling-Window Objective Selector: The system formulates objective selection as a classification problem. It uses interpretable statistical summaries of recent returns as input features ( $x_t$ ) to predict the best objective from the candidate set. To prevent temporal leakage and adapt to concept drift, the selector is trained on a rolling window of "matured" data. A label for a past time $i$ is only available at time $t$ if the evaluation horizon $H$ has fully realized ( $i + H \leq t$ ). The model is a linear softmax classifier trained via regularized cross-entropy on this rolling dataset.
Confidence-Aware Safeguards: To mitigate misselection and excessive switching, DOSS employs a confidence gating mechanism. It calculates a confidence score ( $c_t$ ) as the probability gap between the top-1 and top-2 predicted objectives. If $c_t$ falls below a threshold $\tau$ , the system overrides the raw proposal with a conservative default objective ( $o_{def}$ ). Additionally, an optional switching-frequency control limits the number of objective changes within a trailing window to prevent operational churn.
LLM Oversight (Governance): Rather than using a Large Language Model (LLM) to generate new objectives, DOSS positions the LLM as a constrained auditor. The LLM receives the proposed objective, input features, and confidence score, and is restricted to either ACCEPT the proposal or OVERRIDE it to the predefined safe default. This ensures the decision space remains bounded and provides an auditable trace of the decision-making process.
Downstream Execution: Once the objective is selected (and potentially gated/audited), a fixed downstream decision module executes the optimization (e.g., calculating portfolio weights) based on the chosen objective.

Key Contributions

Policy Formulation: The paper reframes objective-function switching in finance as a direct policy problem, selecting objectives from observable features at each time step rather than inferring latent regimes.
DOSS Framework: The introduction of a learning-based selector that utilizes rolling-window updates for forward-looking selection without temporal leakage, operating on interpretable statistical features.
Safeguards and Governance: The integration of confidence-aware gating to a conservative default and an optional, constrained LLM auditor. These mechanisms are designed to control misselection and excessive switching in deployment-oriented settings, ensuring accountability and stability.

Experimental Results
The authors evaluated DOSS on the FAR-Trans benchmark, a public dataset for financial asset recommendation with a rolling evaluation protocol over 61 decision time points. The candidate set included three objectives: RETURN, DOWNSIDERISK, and SHARPELIKE.

Performance: DOSS with confidence gating (threshold $\tau=0.2$ ) achieved a Hit Percentage Accuracy (HPA) of 0.5626, outperforming the strongest static baseline (Static(DOWNSIDERISK), HPA 0.5235) by approximately 7.5% relative improvement. It also outperformed a direct LLM-based selector (GPT-4.1-mini) which achieved an HPA of 0.5173.
Stability: While the hindsight oracle (upper bound) achieved the highest HPA (0.6150), it exhibited a high switch rate (0.8197). DOSS achieved a significantly lower switch rate (0.4590) compared to the oracle and the direct LLM selector (0.5902), demonstrating a balance between adaptivity and operational stability.
Sensitivity Analysis: A sweep of the confidence threshold $\tau$ revealed a "U-shaped" trade-off. Disabling gating ( $\tau=0$ ) reduced performance, while overly conservative gating ( $\tau=0.4$ ) collapsed the system toward a static strategy. A moderate threshold ( $\tau=0.2$ ) yielded optimal results, confirming that selective rejection of low-confidence switches improves overall accuracy.
Statistical Reliability: Bootstrap analysis over time points confirmed that the improvement of DOSS over the static baseline is robust, with a 95% confidence interval of $[0.0118, 0.0706]$ and a probability of positive improvement of 0.9986.

Significance and Claims
The paper claims that DOSS provides a practical middle ground for financial deployment. It captures a meaningful portion of the performance headroom available from an ideal hindsight oracle while explicitly controlling operational instability through conservative fallback mechanisms. By restricting the LLM to an accept/override role rather than a generative one, the framework maintains a bounded decision space, ensuring traceability and alignment with governance requirements. The results suggest that dynamic objective selection, when coupled with confidence-aware safeguards, can outperform static strategies and direct LLM approaches without incurring the costs of excessive switching.