Developing a Network Discovery Protocol for the Constellation Control and Data Acquisition Framework

This paper presents a network discovery protocol designed to simplify the configuration and integration of diverse devices within the Constellation Control and Data Acquisition Framework, specifically addressing the complexities of managing dynamic test beam environments by eliminating the need for fixed IP addresses.

The Gist

Imagine you are setting up a complex science experiment in a lab, like a temporary city of sensors and computers working together to catch tiny particles. In the past, getting all these different machines to talk to each other was like trying to organize a group of strangers at a party without a name tag system. You had to manually write down everyone's specific address (IP address) on a piece of paper, hand it to each person, and hope they didn't get lost or confused with people from a different party happening in the same building.

This paper introduces a new way to solve that problem, called CHIRP (Constellation Host Identification and Reconnaissance Protocol). Here is how it works, using simple analogies:

The Problem: The "Fixed Address" Nightmare

In the old way of doing things, every computer in the experiment needed a permanent, fixed address. If you moved the experiment to a new room or swapped out a sensor, you had to stop everything, re-calculate everyone's addresses, and update a list. It was slow, annoying, and prone to errors.

The Solution: The "Shout and Listen" System

The authors created a protocol that acts like a smart, automated roll call. Instead of needing a list of addresses beforehand, the computers simply shout into the room, and the others shout back.

1. The Shout (Broadcast): When a computer (called a "satellite") turns on, it doesn't wait for instructions. It immediately shouts out a message to everyone on the local network saying, "I am here! I am part of the 'Particle Experiment' group, and I am ready to talk on this specific channel."
2. The Listen (Discovery): The main control computer (the user interface) is also listening. When it hears a shout from a machine that belongs to the right "group," it adds that machine to its list automatically.
3. The Group ID: Imagine there are three different experiments happening in the same lab building. To make sure Experiment A doesn't accidentally talk to Experiment B, every shout includes a secret "Group ID" (like a team jersey color). The control system only listens for the specific color of the team it is managing.
4. The Dynamic Channel: Since many computers might be running on the same physical machine, they can't all use the same "phone line" (port). CHIRP lets each computer pick a free line on the fly and announce, "I'm using line number 5001," so the main system knows exactly where to call.

How It Works in Real Life

The paper describes testing this system during a real particle physics experiment. They had eight different computers (satellites) running on just three physical machines.

• Before: The team would have had to spend time manually configuring every single IP address.
• With CHIRP: They just turned the machines on. The control screen automatically "saw" all eight satellites appear on the screen, organized by their group, with zero manual address typing.

The "Depart" Message

Just as importantly, if a machine is turned off or leaves the experiment, it sends a final "Goodbye" message (a depart message) before it goes silent. This ensures the control system knows immediately that the machine is gone, rather than waiting for it to time out.

What's Next?

The authors note that while this "shouting" method (using UDP broadcasts) works great for their local lab, it might be too loud for very large networks where routers block shouting. In the future, they plan to upgrade the system to use "whispering" (multicasting) to be more polite to the network, and they want to make the names easier to read for humans instead of just using long codes.

In short: This paper describes a tool that lets a network of scientific computers find each other automatically, organize themselves into specific teams, and start working together instantly without a human needing to write down a single address.

Technical Summary

1. Problem Statement

The characterization of particle detectors in test beam environments presents unique challenges due to the dynamic nature of experimental setups. These environments frequently involve:

• Heterogeneous Systems: Multiple diverse detectors, each with specialized Data Acquisition (DAQ) software, must operate synchronously.
• Frequent Reconfiguration: Components (e.g., reference detectors) change often depending on the facility.
• Network Configuration Bottlenecks: Existing control systems (e.g., EUDAQ2, DAQling) typically require fixed IP addresses for all participating machines. This necessitates manual configuration steps (assigning IPs, editing scripts/config files) after every physical setup change, which is time-consuming and error-prone.
• Multi-Instance Constraints: Multiple control nodes (satellites) may run on a single machine, making it impossible to pre-assign fixed TCP ports for services.

The core problem is the lack of an automated, flexible network discovery mechanism that allows these distributed control systems to identify and connect to each other without manual IP configuration.

2. Methodology

The authors propose a custom solution called the Constellation Host Identification and Reconnaissance Protocol (CHIRP), designed specifically for the Constellation framework.

A. Core Concept: Zero-Configuration Networking

Instead of relying on fixed IPs, CHIRP utilizes Zero-Configuration Networking (Zero-Conf) principles. It leverages UDP broadcasts to automatically discover network components.

• Mechanism: Satellites (control nodes) broadcast "offers" containing their service details. A control interface (or another satellite) can broadcast a "request" to discover existing nodes.
• Departure Handling: Satellites broadcast a "depart" message upon shutdown to clean up the network state.

B. Protocol Specifications

CHIRP is defined as an IETF RFC-style document with the following technical specifications:

• Transport: UDP on port 7123.
• Message Structure: Fixed size of 42 octets.
  • Header (6 bytes): ASCII "CHIRP" + Version (0x01).
  • Body (36 bytes):
    • Message Type (1 byte): Request (0x01), Offer (0x02), or Depart (0x03).
    • Group UUID (16 bytes): Identifies the specific experiment to isolate traffic in shared networks.
    • Host UUID (16 bytes): Unique identifier for the machine/instance.
    • Service ID (1 byte): Identifies the specific service type.
    • Port (2 bytes): The ephemeral port assigned by the OS for the actual TCP communication.
• Addressing: Uses 16-octet UUIDs (compatible with MD5 hashes of arbitrary names) for both hosts and groups.

C. Implementation Strategies

• Language Support: Implemented in C++, Python, and for the ESP32 microchip.
• Multi-Interface Handling: To ensure broadcasts reach all network interfaces on a machine (a common failure point with default 255.255.255.255), the implementation iterates over all active interfaces and sends interface-specific broadcasts.
• Socket Configuration: Uses the SO_REUSEADDR socket option to allow multiple satellites on the same machine to bind to the same discovery port (7123).

3. Key Contributions

1. CHIRP Protocol: A lightweight, custom network discovery protocol tailored for distributed DAQ systems, eliminating the need for fixed IP addresses.
2. Experiment Isolation: Introduction of a Group UUID allows multiple independent experiments to run on the same physical local network without cross-discovery interference.
3. Dynamic Port Resolution: The protocol decouples the discovery mechanism (fixed port 7123) from the actual service communication (ephemeral ports), enabling multiple instances to run on a single host.
4. Dual Joiner Support: The protocol supports both "early joiners" (discovering satellites that join later) and "late joiners" (discovering satellites already running) via a request/offer pattern.
5. Practical Validation: Successful deployment and testing in real-world test beam environments, demonstrating seamless integration without manual IP configuration.

4. Results

• Operational Success: The protocol was tested in a test beam environment involving eight satellites running across three different machines.
• User Experience: A user interface (UI) successfully discovered all satellites automatically. Neither the satellites nor the UI required any manual IP address configuration.
• Robustness: The implementation successfully handled complex network topologies, including machines with multiple active network interfaces, ensuring no services were missed due to broadcast limitations.

5. Significance and Future Outlook

• Significance: CHIRP significantly reduces the setup time and complexity for particle physics experiments in dynamic test beam environments. By removing the dependency on fixed IP addresses, it allows researchers to focus on the physics data rather than network administration. It provides a scalable solution for the Constellation framework, which is designed for volatile and dynamic experimental setups.
• Future Improvements: The authors plan to:
  • Replace UDP broadcasts with multicasts to reduce network traffic and comply with routers that restrict broadcasting.
  • Replace fixed-length UUIDs with arbitrary-length names to improve log readability.
  • Adopt standardized serialization formats (e.g., MessagePack) to simplify code paths and improve maintainability.

In conclusion, this paper presents a practical, robust solution to a common bottleneck in experimental physics infrastructure, enabling a "plug-and-play" approach for complex, distributed data acquisition systems.