Enabling Cloud-Level Accuracy in Edge AI through IoT Data Preprocessing

Technical Summary: Enabling Cloud-Level Accuracy in Edge AI through IoT Data Preprocessing

Problem Statement
Large Language Models (LLMs) offer a natural-language interface for interpreting Internet of Things (IoT) sensor data in smart environments, potentially replacing rigid dashboards with flexible, user-centric analytics. However, high-capacity LLMs typically require cloud deployment, introducing latency, privacy risks, and connectivity dependencies unsuitable for real-time or emergency edge scenarios. While local (edge) deployment mitigates these issues, compact edge-deployable models often exhibit weak numerical reasoning when presented with raw sensor readings. This paper investigates whether the performance gap between local and cloud LLMs in environmental monitoring is a limitation of model capacity or a failure of data representation. Specifically, it addresses the challenge of enabling small, local LLMs to reliably classify air quality and thermal comfort without relying on cloud resources or computationally expensive fine-tuning.

Methodology
The authors propose a structured prompt construction framework that transforms raw IoT sensor measurements into progressively enriched textual representations before they are fed to the model. This "prompt-side preprocessing" acts as a lightweight interface layer between numerical data and language reasoning. The study evaluates three prompt variants:

1. Variant A (Raw): Direct presentation of sensor values and user queries.
2. Variant B (Threshold-Aware): Textual descriptions that explicitly compare sensor values against regulatory or comfort limits (e.g., "Temperature is 28°C, exceeding the upper limit of 26°C by 7.7%").
3. Variant C (Status-Enriched): Variant B augmented with compact summary flags indicating the overall safety status of air quality and thermal comfort (e.g., "Air quality safety status: unsafe").

The evaluation utilizes a binary query dataset covering air quality, thermal comfort, and joint environmental conditions. The dataset is derived from indoor sensor data (Raspberry Pi nodes with BME680 sensors) collected at Tampere University and outdoor data from Helsinki, Katowice, and Warsaw. The study compares five local LLMs (ranging from 3B to 8B parameters) and five cloud LLMs across three prompt variants and two inference modes: direct answering (No-CoT) and Chain-of-Thought reasoning (CoT). Performance is measured via classification accuracy, parse rate, and response latency.

Key Contributions

• Prompt-Side Preprocessing Framework: The paper introduces a method to convert raw sensor readings into threshold-aware and status-enriched textual representations, demonstrating that this lightweight preprocessing significantly improves local LLM accuracy without model retraining.
• Environmental Monitoring Query Dataset: Construction of a binary question-answer dataset specifically designed for air quality and thermal comfort classification across diverse indoor and outdoor environments.
• Comprehensive Local-Cloud Evaluation: A controlled comparison of five local and five cloud LLMs, analyzing the interplay between prompt enrichment, model scale, deployment location, and reasoning strategies (CoT vs. No-CoT).
• Accuracy-Latency Analysis: Joint analysis showing that enriched prompts enable local models to achieve cloud-level accuracy with sub-0.3s latency, while CoT inference often degrades performance and increases latency in edge settings.

Results

• Prompt Enrichment as the Primary Driver: The most significant finding is that prompt enrichment, not model size, is the dominant factor in accuracy. For local models in No-CoT mode, accuracy increased from 50.9% (Variant A) to 81.7% (Variant C) indoors (+30.8 percentage points) and from 63.7% to 89.3% outdoors.
• Closing the Edge-Cloud Gap: Under the most enriched prompt (Variant C) with No-CoT inference, local models achieved near-cloud accuracy. For instance, the local phi4-mini model reached 97.4% accuracy indoors, within 0–2 percentage points of the best cloud models, while operating at a fraction of the cost.
• Inference Mode Trade-offs: Chain-of-Thought (CoT) prompting was found to be counterproductive for this specific task when prompts were enriched. CoT increased latency significantly (e.g., local No-CoT latency ~0.22s vs. local CoT ~3.6s) and introduced "answer-reasoning inversions" where models correctly identified violations in their reasoning trace but output the wrong binary label. No-CoT with enriched prompts proved to be the most reliable and efficient configuration.
• Cost Efficiency: The optimal local configuration (Variant C, No-CoT) reduced operational costs by approximately 134x to 499x compared to cloud API alternatives, with negligible additional compute overhead for the prompt enrichment itself.

Significance and Claims
The paper claims that the performance gap between local and cloud LLMs in IoT analytics is largely a representation problem rather than a model-capacity problem. By transforming raw numerical data into threshold-aware textual descriptions, smaller local models can achieve the reliability of larger cloud models for routine environmental classification.

The authors argue that for structured, threshold-driven IoT tasks, prompt-side knowledge integration is a more effective and resource-efficient strategy than model scaling or Chain-of-Thought reasoning. The proposed approach enables practical, low-latency, and privacy-preserving IoT analytics, allowing local devices to handle routine decisions (e.g., air quality alerts) while reserving cloud resources for complex or uncertain queries. This finding suggests that edge AI deployments can achieve high reliability without the computational overhead of fine-tuning or the latency of cloud dependency, provided the sensor data is appropriately contextualized within the prompt.