DS@GT ARC at eRisk 2026: Hybrid Multi-Agent LLM System with Structured Algorithmic Guidance for Conversational Depression Screening

Imagine a world where a friendly, chatty robot could sit down with someone and gently figure out if they are feeling down, without ever asking the scary, direct questions like "Are you depressed?" This is the dream of conversational depression screening. Scientists have long known that tools like the Beck Depression Inventory (BDI-II) are great at measuring sadness and other symptoms, but they usually need a trained human doctor to ask the questions. That's expensive and hard to scale. Enter Large Language Models (LLMs): super-smart computer programs that can talk like humans. The big question researchers are asking is: Can we build a robot interviewer that is smart enough to spot the signs of depression just by chatting, without needing a human in the loop? The challenge is tricky because these robots can't just "know" the answer; they have to guess based on a conversation, and they can't be told the right answer while they are learning.

In this paper, a team from Georgia Tech (DS@GT) tries to solve this puzzle for a competition called eRisk 2026. They built a system to interview computer characters (called "personas") that are pretending to have different levels of depression. The goal was to get a score and list the top four symptoms, all while keeping the conversation natural. The team started with a simple idea: just use one super-expensive, powerful AI to do the whole job. But that was costly and sometimes inconsistent. So, they evolved their system into a "team" of AIs: one to chat, one to grade, and a manager to keep things on track. Their biggest discovery? They swapped the expensive "star player" interviewer for a cheaper, open-source model (Gemma 27B) and used clever math tricks to help it stay on task. Surprisingly, this cheaper team performed almost as well as the expensive one, proving you don't always need the most powerful (and pricey) brain to get the job done, as long as you have a good playbook.

The Story of the Talking Robot Team

The Georgia Tech team, known as DS@GT, entered a contest where they had to build a system to interview 20 different computer characters. Each character was a "persona" simulating a person with a specific level of depression. The rules were strict: the system had to act like a natural conversation, never asking direct questions like "Are you sad?" but instead figuring out the answer by listening to the chat. At the end, the system had to guess a total score (from 0 to 63) and name up to four key symptoms.

The team tried three different approaches, like upgrading a car engine step-by-step.

1. The Single-Engine Prototype
First, they tried a "monolithic" system. Imagine one single robot doing everything: asking the questions, listening to the answers, and grading the mood all at once. It worked, but it was a bit shaky. Sometimes the robot would get confused, give different scores for the same person on different days, or run out of time before asking about all the important feelings. It was like a student trying to take a test, write an essay, and grade the essay all in one breath—too much to handle at once.

2. The Multi-Agent Team (The Baseline)
To fix the confusion, they split the job. They built a Multi-Agent System. Think of this as a small team of specialists:

• The Interviewer: A robot whose only job is to chat naturally and ask the right follow-up questions.
• The Scorer: A different robot that watches the whole chat and constantly updates the score and confidence level for every possible symptom.
• The Orchestrator: A manager robot that tells the Interviewer, "Hey, we haven't asked about sleep yet, go ask that next!" and decides when to stop the conversation.

This team approach was much more reliable. However, it had a downside: it was expensive. Since they were using a paid, high-end AI (GPT-5-nano) for the Interviewer, and they had to run the conversation many times to get a good average, the cost added up fast.

3. The Hybrid Configuration (The Big Experiment)
The team asked a bold question: Can we replace the expensive Interviewer with a cheaper, open-source model (Gemma 27B) if we give it better instructions?

They swapped the paid Interviewer for the open-source Gemma 27B. But they knew this model was a bit "weaker"—it might not follow instructions as perfectly or dig as deep into emotions. To fix this, they didn't just hope for the best; they built three "algorithmic safety nets" to guide the weaker model:

• The Dialogue Tree: Instead of letting the robot make up questions on the fly, they gave it a pre-written map (a tree). It started with specific questions about sadness and followed a branching path based on the answers. This stopped the robot from going off-topic.
• Reliability-Weighted Aggregation: Since the weaker model might make mistakes, they ran the interview 20 times instead of 10. Then, instead of just picking the middle answer, they used a smart math method (inspired by the "Weaver" framework) to weigh the answers. If most runs agreed on a symptom, that answer got more weight. It's like asking 20 students a question and trusting the answer that most of them agree on, rather than just picking one.
• Cluster Imputation: Sometimes the robot would miss a symptom entirely because it ran out of time. The system had a backup plan: if it missed a symptom but saw strong signs of related symptoms (like seeing "fatigue" and "sleep trouble" when it missed "loss of energy"), it would guess the missing score based on those clues. This prevented the final score from being too low just because the robot forgot to ask one question.

The Results: Cheaper and Just as Good

The team ran their system on all 20 personas and submitted three different versions of their results.

• Run 1 used the expensive, paid team (Baseline).
• Runs 2 and 3 used the cheaper, hybrid team with the open-source Interviewer.

The results were surprising. The hybrid system, using the cheaper Gemma 27B model, actually performed better than the expensive paid system in the main scoring metric (ADODL).

• Run 3 (Hybrid) scored 0.9063, placing 3rd out of all the runs from every team in the competition.
• Run 1 (Paid Baseline) scored 0.8841.

Even more impressive, the hybrid system cost about $2 per persona, while the paid system cost about $8. That's a 75% cost reduction for better results!

The authors suggest that this proves a weaker, open-source model can compete with a stronger, paid model if you give it enough "algorithmic supervision" (the map, the math, and the backup plans). However, they also noted a small flaw: the hybrid system was really good at spotting physical symptoms (like tiredness or sleep issues) but sometimes missed the deeper emotional ones (like guilt or sadness). This led to slightly lower scores for identifying specific symptoms, even though the overall depression score was accurate.

What This Means

The paper concludes that you don't necessarily need the most expensive AI to build a good mental health screening tool. By using a smart team structure and giving the AI a clear map to follow, a cheaper, open-source model can do a fantastic job. This is a big deal because it means such tools could be used in places where money is tight or where privacy is a huge concern (since the open-source model can run on local computers without sending data to big tech companies).

The team admits they didn't do a perfect "head-to-head" test where they kept everything exactly the same except for the model, because the competition rules changed week by week. But the evidence they have strongly suggests that with the right guidance, a smaller AI can punch way above its weight class.