A Comparative Performance Evaluation of SQL and NoSQL Databases Under Different Indexing and Cache Conditions
This study benchmarks MySQL 8.0.46 and MongoDB 8.0.32 across varying dataset sizes, indexing strategies, and cache conditions, revealing that MongoDB excels in bulk writes while both systems benefit significantly from indexing for lookups, ultimately demonstrating that database performance is highly context-dependent rather than universally superior for either model.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the digital world, almost every action leaves a trace. A purchase, a login, a message sent—each event is stored in a database, a vast electronic filing cabinet that holds the memory of modern life. To keep these systems running smoothly, engineers must choose between two main types of filing systems. One type, known as relational, organizes data into rigid tables with strict rows and columns, much like a traditional spreadsheet. The other, called NoSQL, stores information as flexible, self-contained documents that can vary in shape and size. For years, a debate has simmered over which system is faster or better. The answer, however, is rarely simple, because speed depends entirely on what you are asking the system to do and how much data it must search through.
A recent study set out to settle this question not with theory, but with a controlled experiment. Researchers Amir Khan, Sawaira Gull, and Arooj Shahzadi built a testing ground on a single computer to see how two popular systems, MySQL and MongoDB, performed under identical conditions. They did not just ask which was faster; they asked how the speed changed when the amount of data grew from a small pile to a massive mountain, and how the use of indexes—specialized shortcuts that help the system find information quickly—altered the results. They tested three specific tasks: dumping a large amount of new data into the system, finding a single record based on a specific name, and grouping many records together to calculate a total.
The researchers began by generating synthetic transaction records, creating three separate datasets containing ten thousand, one hundred thousand, and one million entries. They loaded these records into both database systems and measured how long it took to complete each task. In the first test, which involved writing all the data into the system at once, the document-based system, MongoDB, proved consistently faster. When handling one million records, it finished the task in roughly twelve and a half seconds, while the relational system, MySQL, took nearly thirty seconds. This difference held true regardless of whether the dataset was small or large, suggesting that for pure data ingestion, the document-based approach had a clear advantage in this specific setup.
The second test focused on finding a single record, similar to looking up a specific customer by their ID number. Here, the researchers measured the time it took to find a record both without any help and with the aid of an index. Without an index, the system had to scan through every single record to find the match. As the data grew to one million entries, this search became painfully slow, taking over half a second in both systems. However, once the researchers added a single-field index—a simple list that points directly to the location of each ID—the results changed dramatically. The time to find the record plummeted to just over one millisecond for both systems. This reduction, which exceeded ninety-nine percent, showed that for simple lookups, the type of database mattered far less than the presence of a good map to guide the search.
The third test was more complex, involving a filtered aggregation. The researchers asked the system to find all transactions in a specific category, group them by their status, and sum up the total amounts. This task is harder because it requires not just finding data, but also sorting and calculating it. In this scenario, the results were less uniform. For the smaller datasets, the relational system was sometimes faster, but as the data grew to one million records, the document-based system pulled ahead significantly. The document system completed the task in about three hundred milliseconds, while the relational system took over a second.
Perhaps the most surprising finding emerged during this third test with the largest dataset. When the researchers used an index on the category field to speed up the relational system, the task actually became slower than when they had no index at all. The system took over a second with the index, compared to roughly seven hundred milliseconds without it. This counterintuitive result was not a mistake; the researchers repeated the test independently and confirmed that the index was indeed causing a delay in this specific configuration. It appears that for this particular type of complex calculation, the overhead of using the index outweighed its benefits, a nuance that a simple "one is faster than the other" conclusion would miss.
The study also examined how the systems behaved when they were already "warm," meaning they had recently processed similar requests and had data ready in their memory. In these conditions, both systems became faster, but the degree of improvement varied. The document system saw a significant boost in its aggregation speed when warm, while the relational system showed a more modest gain. These findings reinforce that performance is not a fixed property of the software itself, but a fluid result of the interaction between the data size, the type of query, the presence of indexes, and the current state of the system's memory.
Ultimately, the research concludes that there is no universal winner. The choice between these two database models cannot be made based on a single speed test. If the goal is to rapidly ingest massive amounts of new data, the document-based system showed a distinct edge. If the goal is to retrieve a single record, both systems perform equally well as long as a proper index is in place. However, for complex calculations involving large volumes of data, the performance can swing depending on how the data is structured and indexed. The study serves as a reminder that in the world of data, the right tool depends entirely on the specific job at hand, and that the most efficient path is often found not by choosing a brand, but by understanding the unique shape of the work to be done.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.