Real-Time Joint Classification and Heat-Source Localization on Native 8×8 Thermal Frames with a Lightweight Dual-Head CNN for Microcontroller Deployment
This paper presents JFilterLocalizationCNN, a lightweight dual-head convolutional neural network that enables real-time joint classification of thermal scenes and heat-source localization on native 8×8 infrared frames, achieving high accuracy and efficient deployment on resource-constrained microcontrollers while preserving user privacy.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine a world where we can watch over our homes, hospitals, and offices without ever capturing a face or a recognizable body. This is the promise of a specific kind of sensing that relies on heat rather than light. While standard cameras record detailed images that can identify who is in a room, thermal sensors see only temperature differences. The most basic version of this technology uses a tiny grid of sixty-four sensors, arranged in an eight-by-eight square. To the human eye, the data from such a grid looks like a blurry, low-resolution smudge of warmth. It is far too coarse to show a person's features, yet it contains enough information to tell if a room is empty, if someone is sitting there, or if a dangerous heat source, like a fire, has started. The challenge for scientists has been figuring out how to make this tiny, blurry data useful in real time, especially on small, cheap computers that can be hidden in a ceiling or a wall.
A team of researchers has now solved this problem by teaching a small computer program to read these thermal smudges directly. They created a system that can look at a single frame of this eight-by-eight grid and instantly do two things at once: it decides what is happening in the room, and it points to exactly where the main source of heat is located. This is a significant step forward because previous attempts to use this low-resolution data relied on complex, hand-written rules that often failed to spot fires or locate people accurately. The new approach uses a lightweight neural network, a type of computer model inspired by the brain, designed specifically to work with this tiny amount of data. Instead of trying to guess what the image might look like if it were clearer, the system learns to find patterns in the raw, blurry numbers themselves.
The researchers built a custom model that acts like a dual-purpose tool. In a single glance at the data, it classifies the scene into one of three categories: an empty room, a room with a person, or a room with a person and a fire. At the same time, it calculates the precise location of the hottest spot in that grid. To make this work reliably, they first cleaned the data with a simple filter that removes the background temperature of the room, ensuring the system focuses only on the relative warmth of objects. This allows the model to work even if the room gets slightly warmer or cooler over time. When tested on a dataset of twelve thousand balanced images, the system was remarkably accurate, correctly identifying the scene in nearly every single case and pinpointing the heat source with an error of less than half a pixel.
What makes this discovery particularly powerful is that it runs on hardware as small as a microcontroller, the kind of chip found in everyday smart devices. The researchers took their model and compressed it so much that it fits into a tiny amount of memory, shrinking from a size of nearly half a megabyte down to just one hundred and twenty-six kilobytes. This compression did not hurt its performance. When they installed this tiny version on a small development board connected to a real thermal sensor, it continued to work in real time. In live tests involving thousands of frames from a real-world setting, the system maintained a high level of accuracy, correctly identifying scenes about eighty-two percent of the time and locating heat sources with an average error of just under one and a half pixels.
The study also explored how well this system could adapt to different environments. When the researchers tested the model on data from a different type of sensor with a higher resolution, it struggled at first, showing that the system is tuned to the specific geometry of the eight-by-eight grid. However, after a brief period of retraining on the new data, the system adapted quickly and regained high accuracy. This suggests that while the specific design is optimized for the small sensor, the core ability to learn thermal patterns is robust enough to transfer to other devices with a little help. The researchers confirmed that the system works best when it is trained to handle both classification and location finding together, as trying to do just one task made the other significantly worse.
This work demonstrates that useful perception is possible even at resolutions that most vision systems would discard. By proving that a tiny, privacy-preserving sensor can be paired with a lightweight computer to detect people and fires in real time, the researchers have opened a path for safer, more private indoor monitoring. The system does not need to see a face to know someone is there, nor does it need a powerful server to process the data. It simply reads the heat, understands the scene, and acts, all within the space of a few milliseconds. This capability could allow for early fire warnings in care homes or occupancy tracking in private spaces without ever compromising the privacy of the people inside.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.