APrivacy-Preserving Architecture for On-Device Intelligent Computing Using a Small Language Model with Input and Output Guardrails: Prototype Implementation and Preliminary Evaluation

In the modern world, we often ask computers to think for us, using powerful artificial intelligence to write emails, solve problems, or answer questions. Most of these smart systems live in the cloud, meaning your words travel across the internet to a distant server to be processed. While this works well, it raises a quiet worry: every time you type a personal detail like an email address or a phone number, that information leaves your device and enters someone else's system. To solve this, researchers have developed smaller, more compact versions of these thinking machines that can run directly on your own computer or phone. This keeps your data right where it belongs, on your device. However, just because a computer is local does not mean it is safe. A clever trick known as a "prompt injection" can still fool the machine into ignoring its rules or revealing secrets, and the machine might accidentally spit out private information it was supposed to keep hidden. The challenge, then, is to build a system that stays local but also acts as a vigilant guard, checking what goes in and what comes out.

A researcher named Harshal Gadekar from Sanjivani University in India has built a working prototype to test exactly this idea. He created a privacy-focused assistant that runs a small language model called Phi-3 Mini entirely on a local computer, using a tool named Ollama to manage the process. The system is designed with a double layer of security, like a security checkpoint that scans passengers before they board and again after they arrive. Before the computer even thinks about an answer, an "input guardrail" scans the user's message. It looks for signs that someone is trying to trick the system, such as commands to ignore previous instructions or to reveal the machine's hidden rules. If it finds such a trick, it stops the request immediately. It also scans for sensitive personal details, like email addresses, and hides them before the computer reads them. Once the local computer generates a response, a second "output guardrail" scans the answer to make sure no private information accidentally slipped through.

The researcher tested this system with a small set of carefully chosen questions to see how it behaved in the real world. When asked a simple, harmless question like "What is artificial intelligence?", the system worked perfectly. It processed the request locally, passed all security checks, and returned an answer in just over three seconds. The screen showed that the input was clean, no tricks were detected, and the final answer was safe to display. The system then faced a series of four different attempts to trick it, where users tried to command the machine to reveal its secret instructions or ignore its safety rules. In every single one of these cases, the input guardrail caught the attempt. The system identified the trick, blocked the request, and showed a warning message instead of letting the computer generate a response. This proved that even when the computer is running on your own desk, it still needs a guard to stop it from being manipulated.

The test also included a question containing fake personal data, specifically a made-up email address and a phone number. Here, the system showed both its strength and a clear weakness. When the user typed in the email address, the system spotted it, replaced the text with a placeholder that said "email redacted," and passed the message to the computer. The final answer showed the redacted text, meaning the email was successfully protected. However, the system failed to catch the eight-digit phone number. It let the number pass through unchanged, and the final output displayed the full phone number as if nothing had happened. The output guardrail also missed it, reporting that no sensitive data was found in the answer. This result highlights that while the idea of local, secure computing is viable, the current rules for spotting private information are not yet perfect. The system successfully blocked the trickery and caught the email, but it left the phone number exposed.

The study concludes that it is possible to combine local computing with security checks in a single, lightweight pipeline. The system demonstrated that it can run on a personal device, block common attempts to trick it, and hide some types of private data. Yet, the researcher is careful to note that these findings are preliminary. The test set was very small, consisting of only a handful of questions, and the results were observed on a single machine without measuring how often the system might make mistakes or how fast it would be under heavy load. The failure to catch the phone number suggests that the detection rules need to be expanded to cover more formats of personal data. Before this approach can be considered fully reliable for everyday use, the system needs to be tested on a much larger variety of inputs, including more complex tricks and different types of personal information. For now, the work stands as a promising proof of concept, showing that a secure, private AI assistant is within reach, provided the security guards are constantly improved.