Hate Speech Classification In Roman Urdu: A Comparative Study On Parameter Efficient Fine-Tuning And Prompt Engineering

The internet has become a vast, global town square where people from every culture share their thoughts, but this open space also allows harmful words to spread with alarming speed. Detecting these harmful messages, known as hate speech, is a difficult task for computers because it requires understanding not just the words themselves, but the intent and context behind them. This is especially true for languages that do not follow strict rules or have large libraries of pre-written examples for computers to learn from. Roman Urdu, a way of writing the Urdu language using the Latin alphabet, is a prime example of such a language. It is spoken by millions in Pakistan and among Urdu speakers worldwide, yet it is often informal, with inconsistent spelling and grammar that changes depending on who is typing. Because standard computer programs struggle with this lack of structure, researchers are looking for new, lighter ways to teach machines to spot hate speech in these messy, real-world texts without needing massive amounts of data or expensive supercomputers.

A researcher named Toneema Zubair tackled this challenge by testing three different methods to see which one could best teach a computer to distinguish between a harmless comment and a hateful one in Roman Urdu. The study focused on a specific dataset containing over 72,000 comments collected from social media, where the vast majority were harmless, but a smaller portion contained toxic language. The first method tested was simply asking a large, pre-trained computer model to guess the answer without any extra training, a technique known as zero-shot prompting. The second method involved giving the model a few examples of what to look for before asking it to guess, called few-shot prompting. The third method was a more technical approach known as parameter-efficient fine-tuning, where the researchers made very small, targeted adjustments to the model's internal settings to help it learn the specific patterns of Roman Urdu hate speech, rather than trying to retrain the entire model from scratch.

The results showed that simply asking the computer to guess without any help was often unreliable. When the models were left to their own devices, they frequently mislabeled harmless comments as toxic or missed actual hate speech entirely, largely because the language was so informal and the dataset was heavily skewed toward non-toxic comments. However, when the researchers provided the models with just a handful of examples to guide them, the performance improved significantly. The method that yielded the most consistent and reliable results, though, was the one that made small, precise adjustments to the model. By fine-tuning the model with these lightweight updates, the computer learned to recognize the subtle nuances of Roman Urdu much better than it did with just examples or no help at all. This approach allowed the model to achieve a high level of accuracy, correctly identifying both toxic and non-toxic comments with a balance that the other methods could not match.

The study also explored different ways of framing the questions asked to the computer, such as using specific sentence structures or adding invisible, learnable tokens to the prompts. While these variations showed promise and sometimes performed well with small amounts of data, they did not consistently outperform the method that adjusted the model's internal settings. The research suggests that while asking a powerful computer model to guess based on a few examples is a fast and resource-friendly option, it can be unpredictable. In contrast, making small, efficient changes to the model itself provides a more stable and robust solution for detecting hate speech in low-resource languages like Roman Urdu. This finding is important because it offers a practical path forward for moderating online content in regions where data is scarce and computing power is limited, ensuring that harmful speech can be identified without requiring massive, energy-intensive training sessions.