💻 computer science

Learning Selective LLM Autonomy from Copilot Feedback in Enterprise Customer Support Workflows

This paper presents a deployed system that leverages copilot feedback and UI interaction traces to train a selective LLM agent for automating enterprise customer support workflows, achieving a 45% automation rate and 39% reduction in handling time while maintaining quality through a confidence-based abstention mechanism.

Original authors: Nikita Borovkov, Elisei Rykov, Olga Tsymboi, Sergei Filimonov, Nikita Surnachev, Dmitry Bitman, Anatolii Potapov

Published 2026-04-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Nikita Borovkov, Elisei Rykov, Olga Tsymboi, Sergei Filimonov, Nikita Surnachev, Dmitry Bitman, Anatolii Potapov

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a busy customer support center as a giant, complex control room. Inside, human operators are constantly juggling computers, clicking buttons, filling out forms, and typing messages to solve customer problems. It's a high-stakes job where one wrong click can cause a headache for a customer.

This paper describes a new "smart assistant" system built for this control room. Instead of trying to replace the humans entirely, the system acts like a super-observant co-pilot that learns how to drive the car, but only takes the wheel when it's 100% sure it won't crash.

Here is how it works, broken down into simple steps:

1. The "Shadow" Phase (Learning by Watching)

First, the system doesn't do anything. It just watches. It records every single click, keystroke, and screen change the human operators make while solving thousands of customer cases. Think of this like a new driver sitting in the passenger seat, watching a pro driver navigate a city for weeks, memorizing every turn and stop sign.

2. The "Co-Pilot" Phase (Suggesting, Not Deciding)

Next, the system starts offering suggestions. It says, "Hey, based on what I've seen, you should probably click this button next."

The Human's Role: The operator can either say, "Yes, good idea," and let the system do it, or they can say, "No, do it this way instead."
The Learning: Every time the human corrects the system, the system learns. It's like a student getting a red pen correction on a homework assignment. Over just two weeks, the system gets really good at guessing the right next move.

3. The "Selective Autopilot" Phase (Taking the Wheel Safely)

This is the most important part. The system doesn't just blindly take over. It has a built-in "Safety Judge" (called a critic).

High Confidence: If the system is very sure (e.g., "Click the 'Next' button"), it does it automatically in the background.
Low Confidence: If the system is unsure (e.g., "Should I send this specific message to the customer?"), it stops and asks the human for help.
The Result: The human operator can now watch multiple conversations at once. They only have to step in when the system pauses to ask a question. They aren't doing the typing anymore; they are just the safety net.

How It Handles Mistakes

The paper notes that the system is built to be careful.

The "Stop" Sign: If the computer screen changes in a weird way the system hasn't seen before, or if the system feels nervous about a decision, it immediately hands control back to the human.
The "Reset" Button: If the system makes a mistake, it doesn't spiral out of control. It stops, lets the human fix the screen, and then picks up exactly where it left off.

The Real-World Results

The team tested this in a real business environment with about 15,000 customers.

Speed: The system handled 45% of the entire conversation from start to finish without human help.
Time Saved: The time an operator had to spend actively working on a single customer dropped by 39%.
Quality: The quality of the support didn't go down. The customers were just as happy as before.

The Big Takeaway

The paper argues that you don't need a robot that tries to do everything perfectly to be useful. Instead, you need a robot that is selectively brave. It does the boring, repetitive, low-risk tasks (like clicking buttons or filling forms) automatically, but it knows exactly when to say, "I'm not sure, you handle this one."

This approach is faster to build than traditional computer programs (which require humans to write rigid rules for every possible scenario) and is more reliable than a "wild" AI that tries to do everything on its own. It turns the human operator from a "driver" into a "fleet manager," supervising many conversations at once with much less stress.

1. Problem Statement

Deploying Large Language Model (LLM) agents for end-to-end automation in enterprise customer support faces significant hurdles:

Error Propagation: Small errors in multi-step UI interactions compound over long horizons, leading to workflow failures.
Cost and Complexity: Traditional multi-step control loops are expensive at scale, and frequent UI/process changes make brittle Robotic Process Automation (RPA) or manually scripted flows difficult to maintain.
Safety and Trust: Fully autonomous agents risk degrading support quality or causing irreversible errors in sensitive business processes (e.g., account changes).
Data Scarcity: Unlike web browsing benchmarks, enterprise workflows lack large, public datasets for training, and the domain logic is highly specialized.

The authors propose a system that achieves selective automation: automating high-confidence steps while deferring uncertain decisions to human operators, thereby balancing efficiency with safety.

2. Methodology

The system employs a staged deployment pipeline inspired by imitation learning (specifically DAgger), progressing through three phases to train a policy and a critic.

A. Data Representation and Serialization

Schema-Driven View: Instead of relying on raw DOM or screenshots (which are fragile to UI changes), the system extracts a schema-driven JSON representation of the Business Process Management (BPM) interface.
State-Action Traces: Operator sessions are logged as trajectories $\{s_t, a_t\}$ . The state $s_t$ includes chat history, UI context (customer profile, active scenario), and global announcements. Actions $a_t$ are categorized into four types: message composition, navigation, selection/form-filling, and session management.
Input Format: UI snapshots and actions are merged into a chronological dialog interleaving customer messages, operator actions, and updated UI states for LLM input.

B. Staged Deployment Pipeline

Logging Phase:
- Collects full operator interaction traces.
- Trains an initial Policy Model ( $\pi_\theta$ ) via Supervised Fine-Tuning (SFT) on behavior cloning (predicting the next action given the state).
Copilot Phase:
- Deploys the policy to suggest actions to operators in real-time.
- Feedback Loop: Operators either accept the suggestion or correct it.
- Critical vs. Non-Critical: Non-critical actions (e.g., UI navigation) may be auto-executed; critical actions (e.g., sending messages, closing chats) require human approval.
- This phase generates labeled data (accept/reject) specifically for critical steps.
Selective Automation Phase:
- Trains a Critic Model ( $c_\phi$ ) using the copilot feedback to estimate the probability that a proposed critical action is correct.
- Gated Execution: The system executes an action $\hat{a}$ only if the critic's score $c_\phi(s, \hat{a}) \geq \tau$ (where $\tau$ is a calibrated threshold).
- Abstention: If the score is below $\tau$ , control is handed to the human operator, who resumes from the updated UI state.
- Calibration: The threshold $\tau$ is tuned to meet business precision requirements (e.g., ensuring 90% of automated actions would be accepted by humans).

C. Safety and Adaptability

Out-of-Distribution (OOD) Handling: The system uses daily validation rules to detect new UI elements. If drift is detected, automation is disabled for the affected slice, reverting to copilot mode to gather new supervision.
Anonymization: Training data undergoes strict masking (entity replacement, session-level distinctiveness) to protect PII.
Continuous Improvement: Models are retrained on rolling windows of recent data, with slice-specific policy-critic pairs for different issue clusters.

3. Key Contributions

Production-Scalable Staged Deployment: A novel pipeline that transitions from logging to copilot to selective automation, leveraging naturally occurring supervision (accept/reject) rather than requiring manual rule authoring.
Schema-Driven State Serialization: Moving away from visual/screenshot-based agents to a structured JSON representation of the BPM interface, enabling robust monitoring and reliable tool calling without image processing.
Critic-Gated Autonomy: A mechanism combining an SFT-trained policy with a critic trained on trajectory feedback to manage the error-coverage trade-off, allowing safe background execution of high-confidence steps.
Engineering Framework: Comprehensive documentation of safeguards (abstention, data drift handling, fallbacks) and an online evaluation protocol with ablation studies.

4. Experimental Results

The system was evaluated in a live production environment with approximately 15,000 customers per A/B test group.

Offline Results

SFT Necessity: Zero-shot prompting (even with large models like GPT-5 or Gemini 3) achieved only ~47% tool accuracy. Domain-specific SFT on the Qwen2.5-7B model achieved 79.15% tool accuracy, proving that specialized fine-tuning is essential for enterprise logic.
Data Mixing: Mixing historical logs with copilot rejection corrections improved generalization on hard cases.
Critic Performance: An SFT-trained critic outperformed a confidence-based baseline by +3.84% Precision and +3.03% F1, validating that learning from explicit human feedback is superior to raw model uncertainty.

Online Results (A/B Tests)

Efficiency: Compared to a human-only baseline, the selective automation system reduced Average Agent Time (AAT) by 39% (from 227s to 139s).
Quality: Resolution correctness scores remained statistically unchanged compared to the baseline, confirming no degradation in support quality.
Automation Rate: The system fully automated 45% of sessions end-to-end without human intervention.
Comparison to Classic Bots: Outperformed a rule-based chatbot baseline by reducing AAT by 16%.
Incremental Gain: Moving from "mandatory confirmation" to "automation-enabled execution" further reduced AAT by 25%.

5. Significance and Conclusion

This paper demonstrates a practical, scalable path to deploying LLM agents in high-stakes enterprise environments.

Paradigm Shift: It moves away from "full autonomy" (which is risky) and "static RPA" (which is brittle) toward incremental, selective autonomy.
Human-in-the-Loop: The system treats operators as "AI editors," supervising multiple concurrent sessions and intervening only when the system is uncertain.
Generalizability: The approach is applicable to any structured interface workflow where interaction logs are available, offering a blueprint for safe, efficient automation in regulated industries.

The work highlights that safety is achieved not by restricting the model's capabilities, but by gating its execution through a learned critic and a staged deployment process that continuously adapts to changing business logic.