Self-Supervised Auxiliary Task Discovery for Stable Reinforcement Learning in Stock Trading

The world of stock trading has long been a realm where human intuition battles against chaotic, shifting markets. For decades, traders have relied on algorithms to sift through mountains of data, hoping to find patterns that predict whether a price will rise or fall. In recent years, a powerful type of artificial intelligence known as reinforcement learning has emerged as a leading tool for this task. Imagine a digital agent learning to trade not by being told the right answer, but by trying actions, making mistakes, and slowly learning from the profits and losses it experiences. This approach is promising because it can adapt to changing conditions, unlike older methods that assume the market behaves the same way today as it did yesterday. However, teaching these agents to be both profitable and stable is incredibly difficult. The financial market is noisy and unpredictable, often giving the agent very little feedback about whether a specific decision was good or bad until much later. Without clear guidance, the agent can struggle to learn, becoming erratic or failing to generalize its skills to new market conditions.

To help these agents learn faster and more reliably, researchers often give them extra, smaller tasks to solve alongside the main goal of making money. These are called auxiliary tasks. Traditionally, experts have to manually design these extra tasks, guessing what might help the agent understand the market better, such as predicting the next price change or estimating how volatile the market might be. But the problem with this manual approach is that the market changes constantly. A task that works well during a calm period might be useless or even harmful during a turbulent one. If the helper tasks are fixed in stone, they cannot adapt to the shifting landscape of the stock market, leaving the trading agent with outdated or irrelevant information.

A team of researchers at the Indian Institute of Technology Mandi has developed a new way to solve this problem. Instead of asking a human to guess which extra tasks might help, they created a system that discovers these tasks automatically. They call their new framework QUESTrader. The core idea is to let the artificial intelligence figure out what it needs to learn on its own. The system uses a dual-network design. One network, the main trader, learns how to buy and sell stocks to maximize profit. The second network acts as a question generator. It constantly proposes new, simple questions for the main trader to answer, such as "what will the price do in the next few days?" or "how much will the market move?" based on the daily closing price data and technical indicators used in the study. These questions are not fixed; they are generated dynamically based on what the system is currently experiencing. The system then trains the main trader to answer these questions while it trades.

The brilliance of this approach lies in how the system decides which questions are worth asking. The researchers use a sophisticated learning mechanism that looks at the long-term results. It asks: "Did answering this specific question help the trader make better decisions later on?" If a question leads to better trading performance, the system adjusts its parameters to keep asking it. If a question turns out to be a distraction, the system refines its parameters to stop generating it. This process ensures that the auxiliary tasks are always relevant to the current market conditions. The researchers tested this method on four major global stock markets: the Dow Jones Industrial Average in the United States, the FTSE 100 in the United Kingdom, the Sensex in India, and the TAIEX in Taiwan. They compared their new system against a wide range of existing strategies, including traditional investment methods, standard artificial intelligence models, and other systems that use manually designed helper tasks.

The results were striking. Across all four markets, the QUESTrader system consistently outperformed the other methods. In the US market, it achieved an annual return of 21.785 percent, significantly higher than the next best method, which managed 18.176 percent. More importantly, the system was not just making more money; it was doing so with greater stability. It achieved the highest risk-adjusted scores, meaning it generated more profit for every unit of risk it took on. When the market dropped, the QUESTrader agent lost less money than its competitors. In the Indian market, for instance, it maintained a maximum loss of 10.584 percent, which was much lower than the 17.783 percent loss seen in other advanced models. The system also proved effective in the highly volatile Taiwanese market, where it delivered an annual return of 30.279 percent, far surpassing all other approaches.

The researchers found that the key to this success was the ability of the system to adapt its helper tasks in real-time. By automatically discovering the right questions to ask, the trading agent built a richer and more accurate understanding of the market. It learned to recognize patterns that human-designed tasks might have missed. The study also explored how many questions the system should ask at once and how far into the future it should look to judge the value of a question. They found that a moderate number of questions, around sixteen to sixty-four, worked best, and that looking ahead about ten steps in the learning process allowed the system to properly credit the right questions for future success. This balance prevented the system from being overwhelmed by too much information or confused by looking too far ahead.

This work suggests that the future of automated trading may not lie in smarter humans designing better rules, but in smarter machines that can teach themselves what to pay attention to. By letting the system discover its own learning goals, the researchers created a trading agent that is more robust, more adaptable, and ultimately more profitable than those relying on static, human-made instructions. The findings indicate that in the complex, shifting world of finance, the ability to learn what to learn is just as important as the ability to learn how to trade.