Logi-PAR: Logic-Infused Patient Activity Recognition via Differentiable Rule

Logi-PAR is a novel framework that enhances patient activity recognition in clinical settings by integrating contextual fact fusion and learnable differentiable logic rules to provide auditable explanations and counterfactual interventions, achieving state-of-the-art performance on benchmarks like VAST and OmniFall.

The Gist

The Big Picture: From "Guessing" to "Reasoning"

Imagine you are a nurse watching a patient in a hospital room through a security camera. Your job is to spot if the patient is about to fall out of bed.

The Old Way (Current AI):
Most current AI systems are like a super-fast but slightly confused guesser. They look at the whole picture and say, "That looks like a person sleeping," or "That looks like someone walking."

• The Problem: If the patient is sitting on the edge of the bed with their legs dangling, but the camera angle is bad or the lighting is dim, the AI might get distracted by a pillow or a blanket. It might confidently say, "All clear, they are sleeping!" even though they are actually in danger. It sees the pattern but doesn't understand the logic of why it's dangerous.

The New Way (Logi-PAR):
The authors created Logi-PAR, which is like hiring a detective instead of a guesser.
Instead of just looking at the whole scene, the detective breaks the scene down into tiny, specific clues (facts) and then uses a rulebook to decide if there is a risk.

---

How Logi-PAR Works: The Detective's Toolkit

The system works in three main steps, which we can compare to a detective solving a mystery:

1. Gathering the Clues (Multi-View Fact Fusion)

Imagine the hospital room has three cameras: one looking from the side, one from the top, and one from the foot of the bed.

• The Old AI tries to blend all three images into one blurry picture. If the side camera is blocked by the patient's body, the AI gets confused.
• Logi-PAR acts like a team of detectives.
  • Detective A (Side Camera) says: "I can't see the bed rail, but I see the patient's legs hanging over the edge."
  • Detective B (Top Camera) says: "I can see the bed rail clearly, and it is down."
  • Detective C (Foot Camera) says: "I see the patient, but I don't see a nurse nearby."
  • The Magic: Logi-PAR combines these specific clues into a "Fact Graph." It doesn't just say "Patient"; it says: Fact 1: Legs over edge. Fact 2: Rail is down. Fact 3: No nurse present.

2. Applying the Rulebook (Differentiable Logic)

Now, the detective takes these facts and runs them through a Rulebook.

• The Rule: "IF (Legs over edge) AND (Rail is down) AND (No nurse) THEN = HIGH RISK."
• The Innovation: In the past, computers had to be told these rules by humans. Logi-PAR is special because it learns the rules itself while it is being trained. It figures out, "Hey, whenever I see these three things together, a fall happens." It builds its own logic chain.

3. Explaining the "Why" (Causal Explanation)

This is the most important part for doctors.

• Old AI: "Alert! Patient falling!" (But it can't tell you why. It's a "black box.")
• Logi-PAR: "Alert! Patient falling! Reason: The bed rail is down, the patient is sitting on the edge, and no one is helping them."
• The Superpower: It can even run "What if" scenarios (Counterfactuals). It can say: "If a nurse were standing here, the risk would drop by 65%." This helps doctors know exactly what to fix.

---

Why This Matters: The "Pillow" Problem

The paper mentions a specific problem called "Background Bias."
Imagine a patient is trying to get out of bed, but they are lying still for a moment. A standard AI sees a person lying down and a pillow, and thinks, "Safe! Sleeping." It misses the tiny, crucial detail that the bed rail is down.

Logi-PAR is like a detective who ignores the pillow and focuses strictly on the critical clues. It doesn't care about the color of the sheets; it cares about the position of the rail and the location of the feet.

The Results: Better than the Giants

The researchers tested Logi-PAR against the biggest, smartest AI models currently available (like the ones used by Google or Microsoft).

• The Result: Logi-PAR won. It was much better at spotting rare, dangerous situations that the big models missed.
• Why? Because the big models try to memorize what a "fall" looks like. Logi-PAR understands the logic of a fall. If it sees a new situation it has never seen before (like a patient in a weird chair), it can still figure out if it's dangerous because it understands the rules, not just the pictures.

Summary Analogy

• Current AI is like a parrot. It has heard the phrase "Person sleeping" a million times. If it sees a person lying down, it repeats "Person sleeping." It doesn't know why.
• Logi-PAR is like a smart doctor. It looks at the symptoms (clues), checks the medical rules (logic), and gives a diagnosis with a clear explanation of why the patient is sick.

In short: Logi-PAR makes hospital safety systems smarter, safer, and more honest by teaching them to think logically rather than just guess based on patterns.

Technical Summary

1. Problem Definition

Patient Activity Recognition (PAR) in clinical settings is critical for safety monitoring (e.g., detecting falls, unattended bed exits, or mobility deterioration). However, current state-of-the-art (SOTA) approaches, including Vision-Language Models (VLMs) and standard Transformers, face significant limitations:

• Black-Box Nature: They rely on holistic pattern matching and global attention, often missing sparse, fine-grained, and relational cues (e.g., a lowered bed rail, foot proximity to the edge, or the absence of a caregiver).
• Lack of Explainability: They classify what is happening but fail to explain why a specific configuration constitutes a risk.
• Fragility: They struggle with domain shifts, occlusions, and rare compositional scenarios (e.g., a patient sitting on the edge without a rail), often leading to "hallucinations" or false negatives in safety-critical contexts.

The core challenge is to move from correlation-based classification to causal, compositional reasoning that can handle missing data and provide auditable explanations.

2. Methodology: Logi-PAR Framework

Logi-PAR is a Logic-Infused PAR framework that bridges deep visual perception with differentiable symbolic logic. It factorizes the mapping from raw images to risk decisions into two coupled, differentiable stages:

A. Multi-View Fact Fusion (Perception Module $\phi$)

Instead of entangling features into a latent vector, Logi-PAR explicitly extracts Atomic Facts (semantic predicates).

• Atomic Predicates: The system defines a vocabulary of clinically meaningful primitives (e.g., RailDown, EdgeSit, CaregiverNear).
• Uncertainty-Aware Fusion: Given synchronized multi-view images, the model predicts a confidence score ($c_k$) and a view attribution vector for each fact.
• Mechanism: It uses a shared backbone (Swin-Transformer) with parallel heads to output logits and reliability scores. A weighted logit pooling mechanism fuses evidence across views, prioritizing high-reliability views while handling occlusions (e.g., if the rail is hidden in one view but visible in another).
• Output: A probabilistic fact graph $G$ representing the state of the scene as a set of confidence scores for atomic facts.

B. Neural-Guided Differentiable Logic (Reasoning Module $\psi$)

This module learns soft logic rules to compose atomic facts into high-level risk states.

• Rule Induction: A neural-guided rule learner automatically discovers the structure of logical rules (e.g., $Risk \leftarrow RailDown \land EdgeSit \land \neg Caregiver$).
• Differentiable Components:
  • Literal Selection: Uses Gumbel-Softmax to "softly select" relevant facts from the pool for each rule slot.
  • Negation Gating: Learns whether to use the presence or absence (negation) of a fact as evidence via a learnable gate parameter.
  • Composition: Rules are modeled as soft conjunctions (T-norm fuzzy logic) of literals, and the final state is a weighted disjunction of all rules.
• Optimization: The entire pipeline is trained end-to-end using a multi-task loss function comprising:
  1. Classification Loss ($L_{CE}$): For final risk state accuracy.
  2. Fact Grounding Loss ($L_{fact}$): Aligns predicted fact confidences with ground truth.
  3. Sparsity Loss ($L_{sparse}$): Encourages simple, interpretable rules by penalizing redundant connections.

C. Causal Explanation & Counterfactuals

• Rule Traces: The system outputs the specific rule chain that triggered a decision (e.g., "High Risk because Rail is Down AND Patient is at Edge AND No Caregiver").
• Counterfactual Interventions: It can simulate "what-if" scenarios (e.g., "Risk would decrease by 65% if a caregiver were present") by perturbing the fact vector and re-evaluating the rules.

3. Key Contributions

1. First Logic-Infused PAR Framework: Logi-PAR is the first framework to recognize patient activities by applying learnable logic rules to symbolic mappings derived from visual data.
2. End-to-End Differentiable Rule Learning: It jointly learns rule structures and visual fact grounding, allowing the model to automatically discover sparse, clinically relevant logic patterns without manual rule engineering.
3. Multi-View Fact Fusion: Introduces a mechanism to transform visual cues into a probabilistic scene activity graph, explicitly handling occlusions and view inconsistencies by treating evidence as atomic facts rather than entangled features.
4. Auditable "Why" Explanations: Provides transparent, human-verifiable explanations and counterfactual interventions, addressing the "black box" limitation of current VLMs.
5. Compositional Generalization: Demonstrates the ability to generalize to unseen combinations of atomic facts (e.g., new risk scenarios) by recombining learned logical rules.

4. Experimental Results

The framework was evaluated on two clinical benchmarks: OmniFall (controlled, multi-view fall detection with atomic annotations) and VAST (real-world hospital surveillance with occlusions).

• Performance Metrics:
  • OmniFall: Achieved a Compositional Generalization Score (CGS) of 89.4%, significantly outperforming foundation models like InternVideo2 (68.3%). This proves Logi-PAR learns causal structures rather than overfitting to scene correlations.
  • VAST: Achieved an F1-Score of 91.8% and AUC of 0.96. Crucially, it reduced the False Positive Rate (F@R) to 0.04, compared to 0.18–0.25 for VLMs, demonstrating superior reliability in avoiding false alarms.
• Ablation Studies:
  • Removing the logic module (Variant B) dropped CGS from 89.4% to 65.0%, confirming the necessity of explicit reasoning.
  • Removing the Fact Fusion mechanism (Variant A) increased the False Positive Rate to 11.7%, showing the importance of multi-view evidence fusion.
• Visualization: In case studies (e.g., unattended bed exits with occluded rails), Logi-PAR successfully inferred risk by fusing complementary views, whereas baseline models failed by focusing on irrelevant background features (e.g., pillows).

5. Significance and Impact

• Clinical Safety: Logi-PAR addresses the critical need for trustworthy AI in healthcare. By providing auditable "why" explanations and counterfactual reasoning, it enables clinicians to verify alerts and understand the causal factors of risk.
• Paradigm Shift: It moves patient monitoring from passive, black-box classification to active, reasoning-based decision support.
• Robustness: The logic-infused approach makes the system robust to domain shifts, occlusions, and rare events, which are common in real-world hospital environments but often cause failures in pure data-driven models.
• Interpretability: The ability to extract and prune rules allows domain experts to inspect, validate, and potentially modify the logic, bridging the gap between AI and clinical expertise.

In summary, Logi-PAR establishes a new paradigm for intelligent clinical monitoring by integrating the pattern recognition power of deep learning with the rigor and explainability of symbolic logic.