Experiential Learning of Runtime Monitoring Using Pachinko

In the high-stakes world of aviation and autonomous flight, engineers face a daunting challenge: how to ensure that complex machines behave correctly when they are running in real time. These systems, often built by large teams of people, are too intricate to check perfectly before they are ever turned on. To solve this, specialists use a technique called runtime monitoring. Imagine a dedicated safety guard standing watch over a machine's data stream, constantly checking if the numbers and events match a set of strict rules. If the machine starts to drift from its safe path, this guard sounds an alarm or takes corrective action before a disaster occurs. While this concept is vital for keeping aircraft and drones safe, teaching it to students is difficult. The scenarios usually involve massive, abstract systems that are hard to visualize in a classroom, making it tough for learners to see why such strict rules are necessary.

A team of educators at Barnard College and Columbia University has found a way to make this abstract concept tangible by turning a classroom into a workshop for a Japanese game called Pachinko. In their recent work, they designed a course assignment where students build interactive Pachinko boards that are monitored by a computer system in real time. The goal was not just to build a game, but to teach students how to write formal specifications—precise, logical rules that tell a computer exactly what to watch for and how to react. By using a physical game where steel balls bounce unpredictably off thousands of nails, the researchers created a scenario where the need for a safety monitor felt immediate and real. The students learned to program a small computer chip to watch the balls, detect patterns, and trigger sounds or motor movements based on those patterns, effectively turning a game into a lesson on safety-critical software.

The project centers on a wooden board filled with hundreds of brass nails and a path for steel balls to roll down. At the bottom of this board sits a small electronic device that acts as the game's brain and its safety guard. This device is equipped with sensors that can detect when a ball passes through a specific channel, bridging a tiny gap in copper tape. The researchers set up the system so that this central device runs two separate tasks simultaneously on its internal processors. One processor handles the physical work of reading the sensors and talking to other devices, while the second processor runs the "monitor." This monitor is a piece of software that constantly checks the stream of ball movements against a set of rules written by the students.

The students, working in small groups, were tasked with writing these rules using a specialized language designed for describing time-based events. They had to decide what patterns of ball movement would trigger a response. For instance, a student might write a rule that says, "If a ball passes through this specific channel twice in a row, send a signal to a motor." When the monitor detects that the rule has been met, it instantly sends a wireless message to other devices on the board. These devices control stepper motors that can physically move parts of the game or trigger lights and sounds. The entire process happens in real time, creating a dynamic loop where the game's behavior changes based on the logical rules the students defined.

The researchers found that the students were surprisingly successful at writing these formal rules to control the game's behavior. The main difficulties the students encountered were not with the logic or the code, but with the physical construction of the game. Connecting the sensors, wiring the motors, and ensuring the wireless communication between the different electronic boards worked smoothly proved to be the most challenging part. This outcome was actually a valuable lesson in itself. In the real world, runtime monitoring is often used in complex systems where hardware and software must work together perfectly. By struggling with the wiring and the physical setup, the students experienced the same kind of integration challenges that professional engineers face when building safety systems for drones or aircraft.

There were some technical limits to what the students could achieve with the tools provided. The software they used to write the rules did not support certain complex time-based operations, such as counting the total number of balls over a long period. To work around this, the students had to be creative, using the main processor to count the balls and then sending those totals to the monitor. While this was not the ideal way to teach those specific concepts, it showed that the students could still think in terms of formal logic to solve the problem. The researchers noted that they are working to improve the software so that it can handle these more complex operations in the future.

The success of this project suggests that teaching advanced safety concepts does not require abstract simulations or massive industrial setups. By grounding the lesson in a physical, interactive game, the educators made the need for runtime monitoring clear and engaging. The materials for the project were relatively inexpensive, costing about seventy-five dollars per board, and the setup was designed to be easily replicated in other classrooms. The system is flexible enough that the number of sensors and motors can be changed, allowing the assignment to be adapted for different class sizes or creative goals. Ultimately, the project demonstrated that students can grasp the principles of formal verification and real-time safety monitoring when they are given a concrete, hands-on way to see those rules in action.