CNN-based IoT Device Identification: A Comparative Study on Payload vs. Fingerprint

Technical Summary: CNN-based IoT Device Identification: A Comparative Study on Payload vs. Fingerprint

Problem Statement
The rapid proliferation of Internet of Things (IoT) devices has significantly expanded network attack surfaces, introducing diverse security vulnerabilities. Traditional identification methods relying on MAC addresses or static headers are increasingly ineffective due to spoofing and encryption. While Machine Learning (ML) and Deep Learning (DL) have emerged as alternatives, there is a need to evaluate the trade-offs between data granularity and computational efficiency. Specifically, it remains unclear whether the high computational cost of analyzing raw packet payloads via Convolutional Neural Networks (CNNs) yields sufficient accuracy gains over lighter, feature-based fingerprinting methods for real-time IoT monitoring.

Methodology
This study utilizes the Aalto dataset to compare two distinct CNN-based identification frameworks:

1. Payload-to-Image Approach:

   • Data Representation: Raw packet payloads are converted into 2D pseudo-images. Following the methodology of Wang et al. [6], the study adopts a "Session+L7" approach where the first 13 bytes represent session headers (IPs, Ports, Protocol) and the subsequent bytes represent the application layer payload.
   • Preprocessing: The first 784 bytes of the packet are extracted. Payloads smaller than 784 bytes are zero-padded, while larger ones are truncated to form a fixed 28 × 28 matrix. Empty packets (e.g., SYN/ACK without data) are discarded.
   • Model Architecture: A CNN model with an input shape of (28, 28, 1) is employed. The architecture includes multiple Conv2D layers, a Flatten layer, and a Dense output layer, resulting in approximately 267,678 trainable parameters.

2. Fingerprint-based Approach:

   • Data Representation: Pseudo-images are generated from pre-extracted feature sets defined in IoTDevIDv1 [3].
   • Model Architecture: A smaller CNN model is used with an input shape of (5, 5, 1). This architecture utilizes Global Average Pooling and contains significantly fewer parameters (28,955).

Both models were optimized using AutoKeras [7] to determine hyperparameters. The study focuses on the first packet of network flows to evaluate feasibility.

Key Contributions

• Comparative Analysis: The paper provides a direct performance comparison between a raw payload inspection method (Deep Packet Inspection via CNN) and a feature-based fingerprinting method within the same experimental framework.
• Efficiency vs. Accuracy Trade-off: It quantifies the specific costs associated with deep payload analysis, demonstrating that while payload-based methods offer slightly higher granularity, they incur substantial computational overhead.
• Architectural Standardization: The study proposes a standardized input strategy (28 × 28 matrices) to avoid the multi-model complexity seen in prior works like Lim et al. [4].

Results
The experimental evaluation yielded the following findings:

• Accuracy: The payload-based CNN achieved a marginally higher accuracy of 0.631, compared to 0.625 for the fingerprint-based CNN.
• Computational Efficiency: The fingerprint-based method was approximately 10 times faster in training time (20.81 seconds vs. 209.93 seconds), representing a ~90% reduction in training duration.
• Model Complexity: The fingerprint model required significantly fewer trainable parameters (28,955) compared to the payload model (267,678).

Significance and Claims
The authors claim that while raw payload analysis offers a slight accuracy advantage, the feature-based fingerprinting approach provides a far more efficient solution for resource-constrained IoT environments. The study concludes that the negligible loss in accuracy (0.6%) is a worthy trade-off for the massive reduction in computational overhead.

Consequently, the paper posits that the fingerprint-based method is a viable candidate for deployment on IoT gateways where memory and processing power are limited. In such scenarios, the latency induced by deep packet inspection (payload analysis) may be unacceptable, making the lightweight fingerprinting approach preferable for real-time monitoring and rapid response, even if it sacrifices marginal classification precision.

Future Work
The authors state that future efforts will focus on validating these findings across larger and more diverse datasets, specifically naming UNSW-DI, UNSW-AD, LSIF, MonIoTr, and CIC-IoT-22. They also plan to explore hybrid architectures capable of dynamically switching between lightweight fingerprinting for general monitoring and deep payload inspection for ambiguous cases.