Physics-Informed Multimodal Learning for Wheel-Force Prediction and Grip Utilization Optimization Using Image Detection and Chassis Sensors in Electric Vehicles

Every electric car today carries a silent, invisible tension between the road and its tires. When a driver presses the accelerator or turns the steering wheel, the tires must grip the pavement to translate that command into motion. If the road is dry and smooth, this grip is strong and predictable. But when the surface shifts to wet asphalt, loose gravel, or a patch of ice, the available grip changes instantly and unpredictably. For the car's safety systems to work, they need to know exactly how much force each tire is currently using and how much more it can handle before it slips. This knowledge is the difference between a controlled stop and a skid. The challenge is that tires are hidden beneath the vehicle, and the forces they generate cannot be measured directly without expensive, fragile sensors that are not found in production cars. Instead, engineers must estimate these forces by watching the car's body move and by looking at the road ahead, trying to piece together a complete picture from incomplete clues.

A researcher has developed a new way to solve this puzzle by teaching a computer to think like a physicist while looking at the road like a driver. The study, conducted by Timileyin Akande at Abiola Ajimobi Technical University, focuses on creating a system that can predict the exact forces acting on each of the four wheels of an electric vehicle in real time. The researcher combined two different types of information: a stream of data from the car's own sensors, which track speed, steering angle, and acceleration, and a stream of visual information from a forward-facing camera that identifies the type of road surface ahead. By fusing these two streams, the system can anticipate changes in grip before the car even feels them. The goal was not just to guess the forces, but to do so in a way that respects the fundamental laws of physics, ensuring the predictions never suggest a tire is holding more grip than is physically possible.

To build and test this system, the researcher faced a significant hurdle: real-world data that includes precise measurements of tire forces is incredibly rare and dangerous to collect, especially on slippery surfaces like fresh snow or ice. Instead of risking a crash on a test track, the researcher created a highly detailed virtual world. They used a computer simulation that mimics the physics of a real electric car, complete with a mathematical model of how tires behave under stress. They ran thousands of virtual driving scenarios, ranging from steady cruising to emergency braking, across five different road conditions, from dry asphalt to ice. To make the visual part of the system realistic, they did not try to generate fake images of roads. Instead, they simulated the output of a camera system that had been trained on a large public dataset of real road images, allowing the computer to "see" the road with the same level of uncertainty and accuracy as a real-world camera. This hybrid approach gave them a massive dataset of 9,000 labeled examples, pairing sensor readings and road views with the exact ground-truth forces that were happening in the simulation.

The core of their solution is a lightweight computer program, or neural network, designed to be small enough to run on the car's existing onboard computer without slowing it down. The system has two main parts. One part looks at the car's sensors, analyzing a short window of time to understand how the vehicle is currently moving. The other part looks at the road classification provided by the camera, telling the system whether it is driving on dry pavement, wet gravel, or ice. These two streams of information are brought together using a clever, efficient method that allows the visual information to adjust how the sensor data is interpreted. Crucially, the system is trained with a built-in rulebook based on physics. It is penalized if it predicts a force that would cause the tire to slip more than the road surface allows. This ensures that even if the data is noisy or the situation is unfamiliar, the system's predictions remain physically possible and safe.

The results of the simulation showed that this combined approach works significantly better than relying on just one type of information. On slippery surfaces like wet asphalt, snow, and ice, the new system reduced the error in its force predictions by between 22 and 42 percent compared to systems that only looked at the car's sensors or simply combined the data without the physics rules. Perhaps most importantly, the system made fewer dangerous mistakes. It violated the physical limits of tire grip far less often than other methods, which is critical for safety systems that need to know exactly when a tire is about to lose traction. The system also proved to be very stable, producing consistent results even when tested on road conditions it had never seen before, such as fresh snow, which suggests it could generalize well to real-world driving. The entire process is incredibly fast, taking less than one millisecond to make a prediction, which means it can keep up with the rapid decisions required for modern vehicle safety controls.

This work demonstrates that adding a layer of physical understanding to data-driven learning creates a more robust and reliable tool for vehicle safety. By teaching the computer to respect the laws of friction and load transfer, the researcher created a system that is not only accurate but also trustworthy. The study suggests that in the future, electric vehicles could use this kind of lightweight, physics-aware system to manage their stability and traction more effectively, especially in dangerous weather conditions where human reaction times are not enough. While the results were achieved in a simulation, the method provides a clear and practical path forward for building safer, more intelligent cars that understand the road as well as they understand their own movement.