💻 computer science

Head Count: Privacy-Preserving Face-Based Crowd Monitoring

This paper proposes a privacy-preserving crowd monitoring system that uses face recognition combined with fuzzy extractors and homomorphically encrypted Bloom filters to count individuals across time and locations without ever revealing or storing their actual identities.

Original authors: Fatemeh Marzani, Thijs van Ede, Geert Heijenk, Maarten van Steen

Published 2026-04-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Fatemeh Marzani, Thijs van Ede, Geert Heijenk, Maarten van Steen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are standing in a busy city square. You want to know two things:

How many people are here right now? (This is easy; you just count heads.)
How many of these people walked from the train station to the park? (This is much harder.)

To answer the second question, you usually have to "tag" people. In the past, we used their smartphones (like tracking a unique ID card) to see if the same phone showed up at the station and then the park. But now, phones are smart enough to hide their IDs for privacy, so that trick doesn't work anymore.

The authors of this paper, "Head Count," have come up with a clever new way to solve this puzzle without ever knowing who the people are or seeing their faces. They use a system that acts like a magic, invisible tally counter.

Here is how their system works, broken down into simple steps with some fun analogies:

1. The Problem: The "Blurry Photo" Issue

If you take a photo of your friend at the train station and another at the park, they might look slightly different. Maybe the lighting is different, or they are wearing sunglasses, or they turned their head. A computer trying to match these photos might get confused and think they are two different people.

Also, if you just save the photo to a database, you violate privacy. You don't want a master list of everyone's face.

2. The Solution: The "Fuzzy Fingerprint"

The authors propose a pipeline that turns a face into a secret code, but one that is "fuzzy" enough to handle mistakes, yet "stable" enough to recognize the same person later.

Think of it like this:

Step A: The Snapshot. A camera takes a picture of a face.
Step B: The Sketch. Instead of saving the photo, the computer draws a rough "sketch" of the face using math. It turns the face into a long string of 1s and 0s (like a barcode).
Step C: The Magic Eraser. The original photo is immediately deleted. It's gone forever. Only the math-sketch remains.

3. The "Fuzzy Extractor": Fixing the Wobbly Hand

Here is the tricky part. Because the "sketch" might look slightly different depending on the angle or light, the computer needs a way to say, "Hey, this sketch is close enough to the one from the station to be the same person."

They use a tool called a Fuzzy Extractor.

Analogy: Imagine you are trying to match two fingerprints, but one is smudged. A fuzzy extractor is like a smart detective who says, "These prints are 90% the same. Even though there are a few smudges, I'm confident this is the same person."
It takes the messy, slightly different sketches and turns them into a perfectly identical, secret code for that specific person. Crucially, you cannot turn that code back into a face. It's a one-way street.

4. The "Secret Box" (Bloom Filters & Encryption)

Now that the system has a secret code for every person, it needs to count how many people appear in both locations without anyone seeing the codes.

The Bloom Filter (The Magic Sack): Imagine a giant, magical sack with thousands of tiny slots. When a person arrives, you drop their secret code into the sack. The sack doesn't tell you who is inside, but it can tell you if a specific code is already in there. It's like a "Yes/No" check for presence.
Homomorphic Encryption (The Locked Vault): To keep things super safe, the system puts these sacks into a locked vault that can still do math. This is called Fully Homomorphic Encryption.
- Analogy: Imagine you have a locked box with numbers inside. You can ask a robot to add two numbers together without opening the box. The robot does the math on the locked numbers, and when you finally open the box, the result is correct.
- In this system, the server adds the "sacks" from the station and the park together while they are still locked. It calculates the overlap (the people who were in both places) without ever seeing the actual codes.

5. The Result: The Final Count

Finally, an authorized person (the "Client") opens the vault. They don't see a list of names or faces. They only see a single number: "150 people moved from the station to the park."

Why is this a big deal?

Privacy First: No faces are stored. No names are known. Even the system designers can't figure out who the people are.
No Phones Needed: You don't need to carry a smartphone or wear a badge. Your face is enough, but your identity remains hidden.
Smart Cities: This helps city planners understand how crowds move (e.g., "Is the park getting too crowded after the train arrives?") without spying on citizens.

The Catch

The system isn't perfect yet. If the camera is too blurry, or if the person is wearing a mask, the "sketch" might be too different, and the system might miss them. The authors tested this with a dataset of faces and found it works very well, but they are still working on making it even more robust for real-world chaos (like rain, crowds, or bad lighting).

In short: They built a system that counts people moving between places by turning faces into unbreakable, un-reversible secret codes, doing the math on locked boxes, and giving us the total count without ever revealing who anyone is.

1. Problem Statement

The paper addresses the challenge of crowd flow estimation (counting how many unique individuals move between different locations over time) while strictly adhering to privacy regulations (e.g., GDPR).

Limitations of Existing Methods: Traditional footfall counting (counting people at a single location) is well-established. However, estimating crowd flow requires re-identifying individuals across space and time.
- Device-based approaches: Previously relied on MAC addresses from smartphones. These are now ineffective because modern OSs randomize MAC addresses to prevent tracking.
- Biometric approaches: Using facial recognition introduces severe privacy risks. Storing raw facial images or even standard embeddings allows for the reconstruction of identity, enabling unauthorized profiling and tracking.
The Core Conflict: There is a need to count unique individuals moving between cameras without ever revealing their identity or storing linkable Personally Identifiable Information (PII). Furthermore, facial data is inherently "noisy" (variations in pose, lighting, occlusion), making direct comparison of embeddings unreliable for re-identification.

2. Methodology

The authors propose a pipeline that transforms noisy biometric data into stable, non-invertible identifiers and performs set operations on encrypted data. The system involves two cameras ( $A$ and $B$ ), a central server ( $S$ ), and authorized clients ( $C$ ).

The Pipeline Stages:

Face Embedding Extraction:
- Cameras detect faces and use a pre-trained Deep Neural Network (DNN) to extract high-dimensional visual embeddings ( $v \in \mathbb{R}^d$ ).
- Raw images are immediately discarded; only embeddings are processed.
Similarity Preservation (SimHash):
- To handle the continuous nature of embeddings, SimHash (a Locality-Sensitive Hashing method) is applied.
- It maps similar embeddings to similar binary strings (bit vectors) with a small Hamming distance, bridging the gap between continuous vectors and discrete cryptographic inputs.
Stabilization and Privacy (Fuzzy Extractors):
- Even with SimHash, the same person may produce slightly different bit strings due to environmental noise.
- A Fuzzy Extractor is applied to these noisy binary strings. It generates a stable, reproducible, and non-invertible secret key ( $R$ ) and public helper data ( $P$ ).
- If a new observation falls within a specific Hamming distance of the original, the helper data allows the reconstruction of the exact same key $R$ . This ensures the same person yields the same identifier across different cameras without revealing the original face.
Compact Set Representation (Bloom Filters):
- The stable identifiers ( $R$ ) are inserted into Bloom Filters (probabilistic data structures).
- This allows each camera to represent a large set of unique individuals in a compact bit array without storing the actual identifiers.
Secure Computation (Fully Homomorphic Encryption - FHE):
- The Bloom filters are encrypted using Fully Homomorphic Encryption (FHE).
- The encrypted filters are sent to the central server.
- The server performs homomorphic operations (multiplication and addition) on the ciphertexts to compute the intersection of the two sets (identifying common individuals) directly on encrypted data.
- The server learns nothing about the underlying data or identities.
Decryption and Cardinality Estimation:
- The authorized client decrypts the result (the count of set bits in the intersection).
- Using the Bloom filter cardinality formula, the client estimates the number of unique individuals present at both locations ( $|A \cap B|$ ).

3. Key Contributions

Privacy-Preserving Flow Estimation: The first system to estimate crowd flow across locations using facial data without ever revealing identities or storing linkable PII.
Handling Noisy Biometrics: Successfully integrates Fuzzy Extractors with SimHash to convert unstable, noisy facial embeddings into deterministic cryptographic identifiers.
End-to-End Encryption: Utilizes FHE to perform set intersection operations on encrypted Bloom filters, ensuring the central server cannot access raw data or intermediate identifiers.
Device Independence: Unlike previous MAC-address-based systems, this approach does not require individuals to carry smartphones or access cards.

4. Experimental Results

The authors evaluated the reliability of the fuzzy extractor stage using the SCface dataset (130 individuals, 9 images each under varying conditions).

Setup: Images were split between two simulated sites ( $A$ and $B$ ). The system tested the ability to reconstruct the same key $R$ from different images of the same person.
Parameters: Varied SimHash length ( $n_{bits} \in \{64, 128, 256\}$ ) and error tolerance ratios ( $r \in \{10\%, 15\%, 20\%, 25\%\}$ ).
Performance:
- With $n_{bits} \ge 128$ and error ratio $r \ge 20\%$ , the system achieved near-perfect Recall (0.95–0.99) and Precision (0.99–1.00).
- This indicates that the fuzzy extractor effectively corrects variations caused by pose and lighting, ensuring consistent identifiers for the same individual while rejecting different individuals.
- Lower error ratios resulted in high precision but low recall (too strict), while higher ratios balanced the trade-off effectively.

5. Significance

This work represents a significant step forward in Smart City analytics and Privacy-Preserving Analytics.

Regulatory Compliance: It offers a technical solution that aligns with strict privacy regulations (like GDPR) by design, ensuring that re-identification is mathematically impossible for the system operators.
Scalability: By using Bloom filters and FHE, the system can theoretically scale to large populations without compromising privacy.
Future Utility: It enables accurate crowd flow analysis for urban planning and public safety without the ethical and legal hurdles associated with traditional surveillance, paving the way for deployment in real-world environments where device-based tracking is no longer viable.

The paper concludes that while the current evaluation is preliminary, the integration of fuzzy extractors with homomorphic encryption provides a robust foundation for future research into real-time, privacy-preserving crowd monitoring.