Looks Right, Works Right: A Project-Level Benchmark for Multi-Screen Mobile App Generation
This paper introduces MobileForge, the first project-level benchmark for multi-screen mobile app generation that evaluates multimodal LLMs across build, navigation, visual fidelity, maintainability, and efficiency, revealing that while current models can compile and navigate apps, they still struggle with reliable interaction and code quality.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a world where you could show a computer a folder full of photographs taken from a smartphone, and it would instantly write the entire computer program needed to build that exact app. This is the promise of a field called design-to-code, where artificial intelligence attempts to translate visual images into the functional instructions that make software work. For years, researchers have tested these systems by showing them a single picture of a webpage and asking for the code to recreate that one screen. While this has led to impressive results for simple, isolated pages, it misses the complexity of real-world software. A true mobile application is not just one image; it is a connected world of many screens, buttons that move you from one place to another, and shared design elements that must stay consistent across the entire experience. The question remains: can an artificial intelligence look at a complete set of screenshots from a real app and build a working, multi-page project that a human team could actually use?
A team of researchers has set out to answer this question with a new test called MobileForge. Instead of asking models to build a single page, they asked them to build entire applications from scratch. The researchers gathered screenshots from 29 real, popular mobile apps that people use every day, covering everything from social media to finance and navigation. These apps contained a total of 309 different screens. They then asked six of the most advanced artificial intelligence models available to look at these images and generate the full source code for the apps. To ensure the test was fair and rigorous, the team created a detailed map of how the screens should connect to one another and wrote hundreds of specific tests to check if the resulting apps actually worked. They checked not just if the code could be compiled, but if the navigation buttons led to the right places, if the visual design matched the original photos, and if the code was clean enough for other engineers to maintain later.
The results revealed a clear divide between what these models can do and what they still cannot. On the positive side, every single model was able to produce code that could be built into a working project. The strongest models could even navigate through the generated apps and reach the correct pages about 92 percent of the time. However, the journey from a working app to a perfect one is still long. While the code could run, the visual details often drifted from the original photos. The models struggled to keep colors and layouts consistent across different screens, often re-creating the same navigation bar in slightly different ways on every page rather than using a single shared component. This inconsistency meant that while the apps looked similar to the originals, they lacked the polished, unified feel of a professionally designed product.
Perhaps the most surprising finding was that being visually accurate did not mean the code was well-written. The model that produced the most visually faithful apps actually generated code that was twice as long and twice as messy as the code produced by other models. It created many unused parts and failed to reuse common elements efficiently. In contrast, another model produced much shorter, cleaner code that was easier for humans to edit and maintain, even though its visual output was slightly less perfect. This suggests that the current generation of artificial intelligence has not yet learned to balance the look of an app with the engineering quality of its underlying code. The researchers also discovered that the models often failed in predictable ways. Sometimes a button would appear on the screen but do nothing when clicked; other times, the screen would load but be completely blank, leaving the user stuck. These failures were not random glitches but specific breakdowns in how the models understood the connection between a visual element and its function.
To conduct this test, the researchers had to invent new ways to measure success that went beyond simple visual comparison. They developed a method to test navigation by starting each test from a specific, fixed point rather than letting the computer wander through the app in a chain, which prevented one mistake from ruining the entire evaluation. They also created a system where an artificial intelligence judge compared the generated screens against the original photos, but with a safety check built in to ensure the judge was actually paying attention and not just guessing. This rigorous approach allowed them to see the subtle differences between the models that older tests would have missed. The study concludes that while artificial intelligence has made a significant leap in turning pictures into code, it is not yet ready to replace human designers and engineers for complex, multi-screen projects. The technology can build the skeleton of an app, but it still struggles with the muscle memory of navigation and the fine details of design consistency that make an application feel real and reliable.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.