💻 computer science

Explainable Part-Based Vehicle Classifier with Spatial Awareness

This paper presents an enhanced explainable vehicle classification system that integrates spatial probability maps of vehicle parts into a decision-tree framework, achieving accuracy comparable to state-of-the-end CNNs while significantly improving robustness against false detections and model interpretability.

Original authors: Andreas Caduff (Competence Center for Intelligent Sensors and Networks, Lucerne University of Applied Science and Art), Klaus Zahn (Competence Center for Intelligent Sensors and Networks, Lucerne Univ

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Andreas Caduff (Competence Center for Intelligent Sensors and Networks, Lucerne University of Applied Science and Art), Klaus Zahn (Competence Center for Intelligent Sensors and Networks, Lucerne University of Applied Science and Art), Jonas Hofstetter (Competence Center for Intelligent Sensors and Networks, Lucerne University of Applied Science and Art), Martin Rechsteiner (Competence Center for Intelligent Sensors and Networks, Lucerne University of Applied Science and Art), Patrick Flaig (SICK AG)

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to identify a vehicle in a busy traffic camera feed. You have two main ways to do this:

The "Black Box" Method (Traditional AI): You feed the whole picture into a super-smart computer brain (a Convolutional Neural Network or CNN). It looks at the image and instantly shouts, "That's a Truck!" or "That's a Van!" It's incredibly fast and accurate, but if you ask why it made that choice, it can't really tell you. It's like a magician who pulls a rabbit out of a hat but won't show you the trick.
The "Detective" Method (This Paper's Approach): Instead of looking at the whole picture at once, you break the vehicle down into its puzzle pieces. You look for specific parts: wheels, the front cabin, the cargo box, the roof. Then, you use a simple, logical rulebook (like a flowchart) to decide what the vehicle is based on which pieces you found.

The Problem with the First "Detective" Attempt
The authors had previously built a "Detective" system that worked well, but it had a fatal flaw: it was too rigid. It made "hard" yes-or-no decisions.

The Analogy: Imagine a security guard checking a list. If the list says "Must have 4 wheels," and the camera misses one wheel because of a shadow, the guard immediately says, "Not a car!" and stops looking. Even if the rest of the car is clearly visible, the system fails.
In their old system, if the camera made a tiny mistake (like seeing a shadow as a wheel, or missing a real wheel), the whole classification would collapse. It was like a house of cards; one wrong move, and the whole thing fell down.

The New Solution: "Spatial Awareness"
In this new paper, the authors upgraded their Detective system to have "Spatial Awareness."

Instead of just asking, "Do I see a wheel? Yes/No," the new system asks, "Where is the wheel, and does that location make sense for a Truck?"

The Map Analogy: Imagine you are trying to identify a person by their features.
- Old Way: "I see a hat. I see shoes. Therefore, it's a firefighter." (If you see a hat and shoes on a dog, you might get confused).
- New Way: "I see a hat on top of a head, and shoes at the bottom of legs. The distance between the hat and shoes matches a human. Therefore, it's a firefighter."

The new system creates a "probability map." It knows that for a specific type of truck, the wheels should be in a certain area relative to the cabin. If the camera sees a "wheel" in a weird spot (like floating in the sky), the system doesn't panic. It says, "That detection is weirdly placed, so I'll give it a low score, but I'll still look at the other parts."

How It Works (The Simple Steps)

The Eye (Detector): A smart camera (YOLO) scans the image and finds parts like "Wheel," "Truck Cabin," or "Cargo Box." It gives them a location and a confidence score.
The Brain (Spatial Logic): Instead of just counting parts, the system checks the geometry. It calculates: "If this is a Truck, the cabin should be here, and the wheels should be there." It uses a mathematical tool called Softmax Regression to weigh all these clues together.
- If a clue is in the right place, it gets a big "vote" for that vehicle type.
- If a clue is in the wrong place, it gets a small vote or is ignored.
The Verdict (Classifier): The system adds up all the weighted votes and picks the vehicle type with the highest total score.

Why This is a Big Deal

Robustness (The "Noise" Filter): The paper proves that even if the camera makes mistakes (seeing a shadow as a wheel, or missing a wheel), the new system doesn't crash. Because it looks at the relationship between parts, it can ignore the bad clues and still get the right answer. The old system would fail completely with these mistakes; the new one stays calm and accurate.
Explainability (The "Why"): Because the system works by checking specific parts and their locations, we can actually see why it made a decision. We can say, "It called this a Truck because it saw a cabin here and wheels there." This removes the "Black Box" mystery.
Easy Updates: If a new type of vehicle appears (like a new kind of electric truck), you don't have to retrain the whole super-computer brain. You just teach the system the new "parts" and update the rulebook.

The Results
The authors tested this against the "Black Box" AI and their own old "Rigid" system.

Accuracy: The new system is just as accurate as the super-smart Black Box AI (around 98.6% accuracy).
Reliability: When they intentionally messed up the camera's detection (making it very sensitive to false alarms), the old system's accuracy dropped to 93%. The new system stayed at 98.6%.
Speed: It is slightly slower than the Black Box AI (25 milliseconds vs. 7 milliseconds), but still fast enough for real-time traffic monitoring.

In a Nutshell
The authors took a complex, unexplainable AI and broke it down into a logical, part-based system. By teaching the system to understand where parts belong relative to each other (spatial awareness), they created a vehicle classifier that is as smart as the "Black Box" but is also transparent, explainable, and much harder to fool by camera errors. They proved you don't have to choose between being accurate and being understandable.

Technical Summary: Explainable Part-Based Vehicle Classifier with Spatial Awareness

Problem Statement

Intelligent Transportation Systems (ITS) require fine-grained vehicle classification to manage tolls, optimize traffic flow, and enhance safety. While state-of-the-art Convolutional Neural Networks (CNNs) achieve high accuracy, they face two critical limitations in practical ITS deployment:

Lack of Explainability: CNNs operate as "black boxes," making it difficult to understand, diagnose, or suppress misclassifications.
Inflexibility: Modifying existing categories or adding new ones typically requires retraining the entire network, which is inefficient and prone to catastrophic forgetting.

Previous work by the authors addressed these issues by decomposing the classification process into a CNN-based part detector, a feature construction phase, and a decision tree classifier. However, this initial approach suffered from a significant drawback: sensitivity to false part detections. Because the decision tree relied on hard binary decisions (part present/absent), even a single false positive or negative detection could force the classifier down an incorrect decision path, drastically reducing overall accuracy (dropping from 98.5% to <93% under simulated noise).

Methodology

The paper proposes an improved part-based classification framework (v2) that integrates spatial awareness of vehicle parts to overcome the robustness issues of the previous decision-tree-based model (v1). The workflow consists of three main stages:

1. Preprocessing and Part Detection

Preprocessing: Images undergo homography transformation to align the vehicle's side plane with the image sensor, ensuring metric consistency across different camera views.
Part Detection: A state-of-the-art YOLOv5 detector identifies 16 semantically strong vehicle parts (e.g., wheels, truck cabins, load types) and outputs bounding boxes, confidence scores, and part categories.

2. Feature Construction with Spatial Awareness

Unlike the previous approach which used binary features, the new method constructs spatial probability maps for each part relative to specific vehicle categories.

Location Score ( $L^k_i$ ): For each detected part instance $i$ at location $x^{(i)}$ , a location score is calculated for every possible feature $k$ (a combination of a part and a vehicle category). This score represents the probability of the part's presence at that location for that category. It is modeled using an unnormalized Gaussian kernel function fitted to training data via Gaussian Mixture Models (GMMs).
Part Score ( $P_k$ ): The final feature score is derived by multiplying the location score by the detector's confidence score, summing over all instances contributing to feature $k$ , and normalizing by the expected number of detections.
$P_k = \frac{1}{n^{exp}_k} \sum_{j=1}^{n^{det}_k} s_i L^k_i$
This process allows the system to handle multiple part instances (e.g., multiple wheels) and mitigates the impact of individual false detections by leveraging the collective spatial distribution.

3. Classification via Softmax Regression

The vector of part scores ( $P_k$ ) is fed into a Softmax Regression classifier.

This linear classifier maps the spatially-aware features directly to the 19 fine-grained vehicle categories.
Unlike the sequential decision tree, this approach processes all features in parallel, utilizing the full spatial relationship of parts to determine the final class probabilities.

Key Contributions

Spatial Awareness Integration: The authors extend their previous part-based approach by incorporating spatial probability maps for vehicle parts, significantly improving robustness against false detections without sacrificing accuracy.
Balanced Performance: The proposed method achieves accuracy comparable to state-of-the-art end-to-end CNNs while maintaining explainability (via visible part scores and linear weights) and extensibility (new classes can be added by extending the feature list without retraining the heavy part detector).
Comprehensive Validation: The paper presents the largest study to date on fine-grained vehicle detection (19 categories, 16 parts) with thorough validation in a practical application context, including a detailed dataset preparation process using semi-automated annotation.

Results

The study utilized a dataset of 4,779 images covering 19 fine-grained vehicle categories and 16 annotated parts. Performance was evaluated using 5-fold cross-validation against a baseline VGG-type CNN and the authors' previous decision-tree approach.

Accuracy: The Softmax Regression approach achieved an overall accuracy of 98.6%, comparable to the baseline CNN (98.3%) and the previous decision tree method (98.5%). Individual category accuracies were generally above 92%, with the exception of the rare "Tractor Truck" category (79%).
Robustness: A critical finding was the resilience to false part detections. When the part detector's confidence threshold was lowered from 0.5 to 0.001 (simulating extreme false positives):
- The Decision Tree accuracy dropped to 92.9%.
- The Softmax Regression approach maintained an accuracy of 98.6% (with confidence scores retained) or 98.0% (with scores adapted), demonstrating near-insensitivity to detection noise.
Explainability: The model allows for the visualization of which detected parts strengthen or weaken a classification (e.g., identifying that a misclassification was caused by a tall trough being mistaken for a cuboid load), a capability difficult to achieve with black-box CNNs.
Efficiency: The part-based approach is slower than the baseline CNN (approx. 25ms vs. 7ms on an embedded platform) but remains suitable for real-time ITS applications.

Significance and Claims

The paper claims that this research represents a significant advance in ITS vehicle classification by successfully challenging the presumed trade-off between accuracy and explainability.

The authors assert that their method provides a framework that:

Disentangles the classification process into interpretable components (part detection, spatial feature construction, and linear classification).
Ensures extensibility, allowing for the addition of new vehicle categories without the need to fully retrain the intensive part detector.
Delivers robustness, making the system viable for real-world applications where detection noise is inevitable.

The work is presented as a step toward systems that combine high accuracy with intuitive interpretability, offering a practical alternative to black-box deep learning models in safety-critical transportation environments.