Leakage-aware multimodal GeoAI for residential rent valuation: evidence from Arabic property listings in four Saudi cities

Imagine trying to guess the price of a house you have never seen, based only on a short advertisement. You know the number of rooms, the size of the land, and perhaps a few sentences describing the neighborhood. In the world of real estate, this is the daily challenge of an automated valuation model. For decades, experts have understood that a home's value is not just about its bricks and mortar, but also about where it sits and what surrounds it. This idea, known as hedonic theory, suggests that every feature of a property, from the number of bathrooms to the distance to the city center, adds a specific price tag. Today, computers are being asked to do this math at a massive scale, combining hard numbers with the messy, unstructured text of online listings to predict rents and sales. But there is a catch: if the computer is allowed to use information from the test set during its training, it will appear brilliant while actually knowing nothing.

A new study from Saudi Arabia tackles this problem by looking at how we teach computers to value homes, specifically focusing on the rental market for villas in four major cities: Riyadh, Jeddah, Dammam, and Al-Khobar. The researchers wanted to know if adding location data and Arabic text descriptions to the mix would actually help, or if the high accuracy numbers reported in previous studies were simply an illusion caused by bad testing methods. They discovered that the way you split the data for testing matters more than the fancy computer algorithm you use. In fact, they found that a common way of testing these models was so flawed that it made a computer look like it was guessing correctly within a 20% margin for over 83% of cases, when in reality, it was just memorizing the answers.

The researchers started with a public dataset of nearly 3,700 rental advertisements scraped from a popular Saudi real estate website. These ads contained structured details like the number of bedrooms and bathrooms, as well as free-text descriptions written in Arabic. Before building any models, the team performed a rigorous audit of the data and found a startling problem: more than half of the listings were exact duplicates. In some cities, like Dammam and Al-Khobar, the situation was even worse, with over 90% of the records being repeated copies of the same ad. This meant that the dataset looked much larger and more balanced than it truly was. When the researchers removed these duplicates, the sample shrank significantly, leaving them with a much smaller, more realistic set of unique properties to work with.

To test how well these models could actually predict rent, the team compared four different ways of splitting the data for training and testing. The first method was the standard approach used in many studies: randomly shuffling all the listings and splitting them into training and testing groups. When they used this method on the data with duplicates still included, the computer model achieved a stunningly low error rate, appearing to predict rents with almost perfect accuracy. However, as soon as the researchers removed the duplicate listings, the performance of the model collapsed. The error rate jumped from a tiny fraction of a percent to 15.4%, revealing that the model had not learned to value homes; it had simply memorized the repeated entries.

The study then moved to a more honest testing method called spatial cross-validation. Instead of shuffling the listings randomly, the researchers grouped them by neighborhood. They trained the model on some neighborhoods and tested it on completely different ones that the model had never seen before. This simulates a real-world scenario where a computer tries to value a house in a district it has no prior data for. Under these stricter conditions, the model's accuracy dropped further, with the error rate rising to nearly 20%. This confirmed that the earlier high scores were indeed a result of data leakage, where the model was given the answer key during its training phase.

With a fair testing method in place, the researchers investigated what actually helped the model make better predictions. They tested three types of information: the basic structured facts about the house, the free-text Arabic descriptions, and geospatial data about the neighborhood. The geospatial data included the size of the district, its distance from the city center, and the average rent of neighboring districts. The results showed that location was the most powerful tool. Adding neighborhood context to the basic house features reduced the prediction error by about 20%. This suggests that knowing where a house is located, and what the neighbors are paying, is far more valuable than the specific details of the house itself when trying to predict rent in a new area.

Surprisingly, the Arabic text descriptions added very little value once the location was already known. While the text did help slightly when the model was tested on random data, its benefit vanished when the model was tested on unseen neighborhoods. The researchers suspect this is because the text often repeats information already present in the structured fields or mentions the neighborhood name, which overlaps with the geospatial data. Furthermore, the descriptions were often cut off at a short length, limiting the amount of useful information they could provide. This finding challenges the idea that adding more text to a model always makes it smarter; in this case, the location data did the heavy lifting.

The study also examined whether a model trained in one city could be used to predict rents in another. The results were sobering: the models did not transfer well. A model trained on data from Riyadh and Jeddah performed poorly when asked to predict rents in Dammam, often getting the prices completely wrong. This highlights a fundamental limitation of these systems: they learn the specific patterns of the cities they are trained on, and those patterns do not necessarily hold true in other markets. To make the predictions useful, the researchers found they needed to adjust the uncertainty estimates for each city individually, ensuring that the confidence intervals for the predictions were accurate for that specific location.

Ultimately, this research serves as a crucial warning for anyone using artificial intelligence to value real estate. It demonstrates that without careful data cleaning and honest testing methods, a model can appear incredibly accurate while being completely useless in practice. The study proves that removing duplicate records and testing on unseen neighborhoods are not just optional steps, but essential requirements for credible results. While the combination of structured data and location information can significantly improve rent estimates, the promise of text analysis remains limited in this context, and the models cannot simply be moved from one city to another without recalibration. The path to reliable automated valuation lies not in more complex algorithms, but in cleaner data and a deeper respect for the unique geography of each neighborhood.