Collaborating with AI Agents: Field Experiments on Teamwork, Productivity, and Performance
Through a large-scale field experiment involving 2,234 participants creating ads, this study demonstrates that human-AI teams outperform human-human teams in productivity and text quality by fostering task-oriented communication and delegation, though this collaboration leads to more homogeneous outputs and reveals a "jagged frontier" where human partners still excel in image quality.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are running a small advertising agency. You need to create thousands of digital ads for a think tank. To see how Artificial Intelligence (AI) changes the game, the researchers built a special digital playground called Pairit.
In this playground, they set up a massive experiment with over 2,000 people. They split them into two types of teams:
- Human-Human Teams: Two people working together.
- Human-AI Teams: One person working with a super-smart AI robot.
Both teams had 40 minutes to make as many ads as possible. These ads weren't just sitting in a lab; the researchers actually ran them on the social media platform X (formerly Twitter) to see how real people reacted. They tracked everything: every keystroke, every message sent, every image edited, and every click on the ad.
Here is what they found, explained simply:
1. The "Speed vs. Style" Trade-off
The Human-AI teams were 50% more productive. They made way more ads per person than the human-only teams.
- The Text: The Human-AI teams wrote much better ad copy (the words). It was clearer and more engaging.
- The Images: However, the Human-Human teams created better-looking images.
The Analogy: Think of the AI as a brilliant but clumsy writer. It can type a perfect essay in seconds, but if you ask it to pick the perfect photo to go with it, it might grab a blurry or weird picture. The human partner is the artistic eye who knows which photo looks good but might take longer to write the words.
2. The "Jagged Frontier"
The researchers call this the "Jagged Frontier." Imagine a mountain range where some peaks are incredibly high (AI is amazing at writing text) and some valleys are deep (AI is not great at picking images).
- Because the Human-AI teams wrote better text, people clicked on their ads more often and read them longer.
- Because the Human-Human teams had better images, the ads cost less to run (better "cost-per-click").
- The Result: In the real world, both types of teams performed about the same overall. The AI's strength in words balanced out its weakness in pictures.
3. How the Teams Talked (The "Manager vs. Friend" Dynamic)
This is where the teamwork changed the most.
- Human-Human Teams: They acted like friends. They spent time chatting, saying "No worries," "Great idea," or "Sorry I'm late." They built rapport. This social chatter actually helped them work well together, but it took up time.
- Human-AI Teams: They acted like a manager and a robot. The human stopped chatting about feelings and started giving direct orders like, "Write a headline about X" or "Generate an image of a desk."
- The Human-AI teams sent 25% more task-focused messages and 18% fewer friendly messages.
- Because they didn't waste time being polite to the robot, they got more work done.
4. The Art of Delegation
When working with a human, you usually split the work 50/50 because you don't want to be rude or overwork your partner.
- With the AI, the humans delegated 17% more work to the robot.
- Instead of typing the words themselves, they told the AI to do it.
- The Catch: When humans delegated too much to the AI, the ads started to look and sound the same. The "diversity" of the ads collapsed. They were all high-quality, but they all felt a bit generic, like they came from the same factory.
5. Knowing Who You Are Working With
The researchers found that if a person knew they were working with an AI, they behaved differently. They became more like a strict manager: giving clear instructions and letting the AI do the heavy lifting. If they thought they were working with a human (but weren't), they tried to be too friendly, which actually made the work slightly worse.
The Big Picture
The study shows that AI isn't just a tool like a calculator; it's a teammate that changes how we work.
- Productivity: It makes us faster.
- Quality: It makes our words better but our pictures worse (for now).
- Process: It turns us from "collaborators" into "directors." We stop doing the work ourselves and start telling the AI what to do.
The researchers conclude that to get the best results, we need to treat AI like a specialized worker: let it handle the writing and the heavy lifting, but keep humans in charge of the visual style and the creative spark.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.