Customer Shopping Behaviour Analytics: An Integrated Data Pipeline and Decision-Support System
This paper presents an AI-driven, end-to-end data analytics framework integrating Python ETL, MySQL modeling, and interactive dashboards to analyze retail transactions, revealing that while gender spending parity exists despite volume skew, current promotional discounts and subscription models fail to significantly drive basket size or per-visit spend, thereby highlighting critical gaps in acquisition and monetization strategies.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the bustling world of modern retail, every transaction leaves a digital footprint. When a customer buys a jacket, pays with a credit card, or leaves a star rating, they generate a piece of data. For decades, store managers and corporate planners have tried to make sense of these millions of scattered notes to understand what drives people to buy, what keeps them coming back, and how to price items for maximum profit. The challenge has always been one of volume and clarity; the data exists, but it is often buried in messy spreadsheets, riddled with missing numbers, and locked away in formats that require specialized skills to unlock. The goal of this research is to transform that chaotic pile of receipts and reviews into a clear, living picture of customer behavior, allowing business leaders to see exactly what is happening in their stores without needing to be computer experts.
To achieve this, the researchers built a new kind of system that acts as an automated translator between raw sales records and human decision-making. They started with a massive collection of 3,900 real transactions from a retail environment, covering everything from clothing and footwear to accessories. In their original form, these records were imperfect; some customer satisfaction scores were missing, and the data was scattered across different columns that didn't always line up. The team first cleaned this data using a computer program that automatically filled in the missing satisfaction scores by looking at the median rating for that specific type of product, ensuring no valuable sales information was thrown away. They then organized the cleaned data into a structured database, a digital filing system that allows for instant searching and sorting. Finally, they built a simple, interactive website where a store manager can click a few buttons to ask questions like "How much do men spend compared to women?" or "Do discounts actually make people buy more?" and get an immediate answer.
The results of this investigation revealed several surprising truths that challenge common assumptions about how shoppers behave. One of the most striking discoveries concerns gender. While men made up two-thirds of all the purchases in the dataset, the amount of money spent per visit was virtually identical for both men and women. Men spent an average of $59.57 per order, while women spent $59.52. This tiny difference of five cents proves that the lower total revenue from female shoppers is not because they spend less when they do shop, but simply because fewer of them are walking through the door. The gap is entirely in how many customers are being reached, not in how much they are willing to pay.
Another finding questions the effectiveness of standard sales tactics. The researchers looked closely at what happened when customers used discount coupons. They found that while nearly 42% of all orders used a discount, the average size of the shopping basket barely grew. Customers who used a coupon spent only $0.74 more on average than those who paid full price. This suggests that for many shoppers, a discount does not encourage them to buy more items or spend more money; it simply lowers the price of the items they were already planning to buy. Consequently, the store loses a significant portion of its potential profit on those sales without gaining any extra volume to compensate.
The study also uncovered a paradox regarding loyalty programs and subscriptions. Many businesses assume that customers who pay for a membership or subscription are more valuable and spend more per visit. However, the data showed that active subscribers spent an average of $59.47 per visit, while non-subscribers spent $59.59. The difference is negligible, indicating that these memberships currently function more as a convenient way to pay or a perk for frequent shoppers rather than a tool that drives them to spend more. The real drivers of high spending were found to be the specific types of products and the frequency of visits. For instance, customers who bought footwear spent the most per order at $60.75, and the most loyal customers, those who had made more than ten purchases in the past, accounted for nearly half of all transactions.
By combining these automated data cleaning steps with a user-friendly website, the researchers demonstrated that complex business questions can be answered in less than a second, a task that previously took hours of manual spreadsheet work. The system allows managers to instantly filter data by age, gender, or product type, seeing exactly how different groups behave. For example, they found that customers who purchased footwear had the highest average order value, while the system also confirmed that shipping preferences and payment methods are evenly distributed, with no single option dominating the others, suggesting that offering a variety of choices is essential for keeping customers happy.
The ultimate value of this work lies in its ability to turn raw numbers into actionable strategy without requiring a degree in computer science. The researchers showed that the path to better profits does not necessarily lie in more aggressive discounting or assuming that members will spend more. Instead, the data points toward a need to attract more female shoppers, since they spend just as much as men once they arrive, and to rethink how loyalty programs are structured to truly encourage higher spending. The system they built serves as a reliable, low-cost tool that can be used by any retail business to audit its performance, spot these hidden patterns, and make decisions based on what the customers are actually doing, rather than what the business assumes they are doing.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.