Campus AI vs Commercial AI: A Late-Breaking Study on How LLM As-A-Service Customizations Shape Trust and Usage Patterns
This late-breaking study serves as a prequel to a larger field investigation, aiming to stimulate discussion and refine methods for examining how user-salient customizations of LLM-as-a-Service, such as interface and branding changes, influence trust and usage patterns among university students and staff compared to commercial alternatives like ChatGPT.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine the internet as a giant, bustling library where the most popular books are written by a super-smart, all-knowing robot that can chat with you about anything. This robot is called a Large Language Model (LLM), and it's like a magical encyclopedia that never sleeps. But here's the catch: this robot is owned by a big, private company, and when you talk to it, you're borrowing its tools. Sometimes, schools and offices worry that borrowing these tools might be risky, like letting a stranger into your house to help you study. So, they ask, "Can we build our own version of this robot, or at least paint our own logo on it, to make it feel safer and more like 'us'?" This is where the science of "Human-Computer Interaction" comes in. It's the study of how people feel and act when they use technology. The big question researchers are asking is: Does putting a school's logo on a robot chatbot actually make people trust it more, use it differently, or worry less about their secrets? It's a bit like asking if wearing a team jersey makes you play better, or if a friendly face on a vending machine makes you trust the snacks inside.
This paper is a "late-breaking study," which is a fancy way of saying the researchers are sharing their game plan before the big game is even played. They are setting up a massive experiment at a German university to see how students and staff react to their school's customized robot chatbot compared to the famous commercial one, ChatGPT. The researchers aren't just looking at the robot's brain (the code); they are looking at the robot's "skin" and "clothes"—things like the school's logo, the colors on the screen, and the little warnings that pop up. They suspect that even if the robot's brain is exactly the same, the way it looks might trick our brains into feeling different. For instance, they wonder if seeing the university's logo makes us trust the robot more, or if it makes us less careful when the robot makes up facts (which researchers call "hallucinations"). They also want to know if showing a "fuel gauge" for how much computer power the robot uses makes people think more about saving energy.
The team has designed a survey to ask about 250 people—students, teachers, and office workers—how they feel about the two different chatbots. They are checking six main ideas, or "hypotheses." First, they guess that people will trust the school's version more because it feels like it comes from a friend (the university) rather than a stranger (a big company). Second, they think that if you already trust your school, you'll trust the school's robot even more. Third, they worry that because the school's robot looks so official, people might stop double-checking its answers, even when it makes things up. Fourth, they suspect people might actually notice fewer made-up facts in the school version because they are too busy trusting the logo. Fifth, they believe people will feel their data is safer with the school's robot. Finally, they hope that by showing a "token meter" (like a fuel gauge for the AI), the school's version will make people use the robot more carefully and think about the environment.
Right now, this paper doesn't have the final answers yet because the big experiment hasn't finished. The researchers are currently in the "prequel" phase, which means they are sharing their map and asking for feedback to make sure they don't get lost. They have set up a customized version of the chatbot using Microsoft Azure, where they turned off some "randomness" to make the answers more predictable, added the university logo, and included a visual bar showing how many "tokens" (chunks of text) are left. They are also making sure the data stays within the school's network so it's not shared with the outside world.
The researchers are careful to say that they don't know for sure yet if their ideas are right. They are suggesting that these visual changes might change how people behave, but they need the data from their spring 2025 survey to prove it. They aren't claiming that painting a logo fixes all the problems with AI, but they do think it's a powerful tool that organizations shouldn't ignore. If their guesses are correct, it could mean that schools and companies need to think very carefully about how they dress up their AI tools, because the outfit might change how people use the machine just as much as the machine's brain does. Until the survey results are in, this story is still being written, but it promises to tell us a lot about how we trust the digital helpers in our daily lives.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.