Technical Summary: Information Density as a Quantitative Measure for AI-enabled Virtual Sensing

1. Problem Definition

Modern Internet of Things (IoT) and sensor networks, particularly in large-scale smart city environments like Madrid, generate massive volumes of data that exceed the capacity of conventional storage, transmission, and processing infrastructures. While traditional approaches such as compressive sensing and machine learning-based dimensionality reduction (e.g., PCA, SVD, autoencoders) have been employed to manage data, they often suffer from computational inefficiencies, irreversible data loss, or an inability to adapt to resource-constrained devices.

A critical gap identified in existing literature is the lack of a quantitative metric to determine the optimal number, type, and spatial distribution of physical sensors required for a given sensing performance. Current methods often rely on heuristic sensor placement or assume that highly correlated sensors are redundant without a formal framework to quantify when and how physical sensors can be replaced by virtual ones. This paper addresses the need for a principled criterion to assess sensing sufficiency and guide the transition from physical to AI-driven virtual sensing.

2. Methodology

The paper introduces Information Density as a novel quantitative metric to evaluate the redundancy and substitutability of sensing resources. The methodology is built upon an extension of the Data-driven Modality Fusion (DMF) framework, incorporating spatial sensing correlations to enable sensing in regions devoid of physical sensors.

2.1 Core Concepts

The framework distinguishes between two scenarios:

• Intra-modality Virtual Sensing (ImVS): Inferring data from sensors of the same modality (e.g., reconstructing temperature profiles from a subset of temperature sensors).
• Cross-modality Inference (CmI): Inferring one phenomenon from a different modality (e.g., estimating air quality using traffic or noise data).

2.2 Quantitative Measures

To quantify Information Density, the authors propose two complementary measures:

1. Phase in Eigen Space:

   • This measure analyzes the alignment of principal eigenvectors derived from the covariance matrices of sensor data.
   • It assumes sensor signals are weakly stationary and represented in a common feature space.
   • Logic: A small phase angle (high alignment) between the principal eigenvectors of two sensors indicates high redundancy (linear dependence). Conversely, orthogonal vectors indicate complementary information.
   • Application: Sensors with the lowest average phase angles are selected as the optimal set ($S_{opt}$) because they represent the dominant variance directions, allowing other sensors to be inferred.

2. Mutual Information (MI):

   • This measure calculates the statistical dependence between sensor readings using Shannon's Entropy.
   • Logic: High mutual information indicates strong non-linear statistical dependence, implying that one modality can be inferred from another.
   • Application: Sensors with the highest average mutual information are selected as the optimal set, as they capture the most representative patterns.

2.3 Optimization Framework

The paper formulates an optimization problem to find the minimal subset of sensors ($S_{opt}$) that allows for accurate inference of all phenomena ($\Phi$) within a bounded error threshold ($\epsilon$). The framework utilizes these metrics to rank sensors and select a minimal set for training deep learning models, which then perform the virtual sensing tasks.

3. Key Contributions

• Extension of DMF: The paper extends the Data-driven Modality Fusion concept to explicitly incorporate spatial sensing correlations, enabling virtual sensing in the absence of physical sensors.
• Introduction of Information Density: It proposes a formal, quantitative metric to assess the sufficiency of sensing resources, distinguishing between intra-modality and cross-modality scenarios.
• Dual-Measure Framework: It develops and compares two specific measures (Eigen Space Phase and Mutual Information) to capture both linear and non-linear dependencies in sensor data.
• Real-World Validation: The framework is validated using real-world data from Madrid's smart city infrastructure, specifically focusing on traffic and environmental monitoring networks.

4. Experimental Results

The proposed framework was evaluated using a dataset from Madrid's District 19 traffic network (45 sensors, 2023 data) and multi-modal environmental data (pollutants, weather, traffic).

4.1 Intra-modality Virtual Sensing (ImVS)

• Setup: Deep learning models (Feed-forward Neural Networks) were trained on subsets of physical sensors ($|S_{opt}|$) to infer the remaining sensors.
• Performance:
  • Even with a single physical sensor ($|S_{opt}| = 1$), the system achieved a Mean Absolute Error (MAE) of 3.21% (Normalized MAE) with a maximum error bounded at 6.41%.
  • Increasing the number of physical sensors to 3, 5, 8, or 10 reduced the error further, though with diminishing returns beyond a certain point.
  • Selecting sensors based on high Information Density significantly outperformed random selection or selection based on low information density.

4.2 Cross-modality Inference (CmI)

• Setup: The model attempted to infer environmental modalities (wind speed, temperature, humidity, radiation, traffic) solely from pollutant concentration data.
• Performance:
  • Mutual Information and Eigen Space Phase successfully identified strong correlations between pollutants and other modalities (e.g., traffic and pollutants showed a similarity score of 0.199, while random signals showed near-zero correlation).
  • The inference error for estimating various environmental modalities from pollutant data was bounded by 11%.
  • Time-series analysis confirmed that the model could accurately reconstruct trends and amplitudes of the target signals.

4.3 Comparison with Baselines

The proposed Information Density approach was compared against random selection, variance-based selection, and correlation-based selection. The results indicated that Information Density is superior in:

• Capturing dominant variance structures.
• Accounting for redundancy.
• Handling non-linear dependencies (specifically via Mutual Information).
• Applicability to cross-modality selection.

5. Significance and Claims

The paper claims that Information Density provides a principled, interpretable mechanism to guide sensor deployment and replacement decisions in large-scale, data-rich infrastructures.

• Feasibility of Virtual Sensing: The study demonstrates that physical sensors can be replaced or supplemented by virtual, AI-enabled sensors under bounded error conditions, significantly reducing hardware costs, energy consumption, and data redundancy.
• Scalability: The framework offers a pathway to scalable sensing systems where the optimal sensor configuration is determined by data characteristics rather than arbitrary placement.
• Operational Scope: The authors explicitly state that the metric is a decision-support tool rather than a theoretical guarantee of optimality in all conditions. It relies on historical data and assumes approximate stationarity. In highly non-stationary or data-scarce environments, the reliability of the estimates may degrade.
• Impact: By enabling accurate sensing with fewer physical devices, the approach supports the development of more energy-efficient and cost-effective smart city and industrial automation systems.

The paper concludes that the strong correlation between high Information Density and low estimation error validates the use of these metrics for optimizing sensor networks, paving the way for adaptive, AI-enhanced sensing infrastructures.