Enhancing Systems Thinking and Community-Based Innovation in Integrated Behavioral Health: A Retrospective Competency Assessment of the NJ Integrated Behavioral Health Summit

Technical Summary: Enhancing Systems Thinking and Community-Based Innovation in Integrated Behavioral Health

Problem Statement
The United States faces a critical behavioral health workforce crisis, with 137 million Americans (40% of the population) residing in federally designated Mental Health Professional Shortage Areas as of December 2025. While Integrated Behavioral Health (IBH) is recognized as a vital model to reduce care fragmentation and improve access, medical education has been slow to integrate the necessary Health Systems Science (HSS) competencies. Specifically, curricula often lack universality in teaching systems thinking, change agency, and community-based innovation. Furthermore, the efficacy of conference-based educational interventions in building these specific competencies among interprofessional learners remains underexplored, with existing literature lacking formalized curricula and rigorous program evaluations for such formats.

Methodology
The authors conducted a cross-sectional study utilizing a retrospective pre/post survey design to evaluate the 2026 New Jersey Integrated Behavioral Health Summit, a single-day virtual interprofessional conference.

• Participants: The study included 28 complete responses (75.7% completion rate) from a diverse group of health professions learners, predominantly medical students (57.1%) and graduate students (21.4%).
• Instrument: An anonymous Qualtrics survey assessed six HSS competencies across two domains: Systems Thinking (payment/reimbursement models, workflow/organizational gaps, community-based partnerships) and Change Agency (proposing feasible solutions, interdisciplinary collaboration, structural/institutional barriers).
• Measurement: Competencies were rated on a 5-point Likert scale (1 = Not at all capable to 5 = Advanced) for both "Before the Summit" and "Now (after the Summit)." The survey also measured behavioral intent, confidence to pursue change, program relevance, and included open-ended reflections.
• Statistical Analysis: Paired t-tests and Wilcoxon signed-rank tests were used for pre/post comparisons. A Bonferroni-adjusted significance threshold of p < 0.0083 was applied to account for six simultaneous comparisons. Cohen's d was calculated to determine effect sizes.

Key Results
The study found statistically significant improvements across all six assessed HSS competencies, with all p-values < 0.001, surviving Bonferroni correction.

• Competency Gains: The mean pre-post gain across all items was +1.59 points. Effect sizes were uniformly large, ranging from d = 1.06 (interdisciplinary collaboration) to d = 2.05 (community-based partnerships).
  • Largest Gains: Community-based partnerships (+1.68, d = 2.05) and proposing feasible solutions (+1.82, d = 1.67).
  • Smallest Gain: Interdisciplinary collaboration (+1.29, d = 1.06), which had the highest baseline score (pre-mean = 2.61).
• Behavioral Intent: 71.4% (20/28) of respondents reported an intent to make changes in their IBH-related education or professional activities. The most common intended actions included engaging in institutional/community efforts (65.0%) and incorporating IBH concepts into research or quality-improvement projects (55.0%).
• Relevance and Impact: 92.9% (26/28) rated the Summit content as relevant to their career goals. Case Competition participants (32.1% of the sample) reported a mean impact score of 4.22/5.0 regarding their ability to analyze IBH challenges and develop systems-oriented solutions.

Key Contributions

1. Evidence for Scalable Interventions: The paper provides empirical evidence that single-day, virtual, interprofessional summits can serve as effective, high-impact complements to longitudinal IBH curricula, particularly for building competencies in community-based partnerships and solution design.
2. Competency Specificity: The study highlights that conference-based formats may be particularly effective for teaching complex, systems-level competencies (e.g., community partnerships) that are difficult to address through didactic instruction alone.
3. Pedagogical Innovation: The inclusion of a student case competition is identified as a promising pedagogical approach for developing systems thinking and solution design skills, yielding high participant impact ratings.
4. Framework Alignment: The study operationalizes the Gonzalo et al. HSS framework within a conference setting, demonstrating measurable gains in specific domains of systems thinking and change agency.

Significance and Claims
The authors claim that the 2026 NJ Integrated Behavioral Health Summit successfully produced significant gains in self-perceived HSS competency and strong behavioral intent among a diverse group of learners. They posit that such conference-based interventions offer a scalable model for cultivating the next generation of systems-informed healthcare leaders, addressing the gap in HSS integration within medical education.

However, the paper maintains a modest tone regarding its generalizability. The authors explicitly frame these findings as an "early proof-of-concept" rather than evidence generalizable to the broader IBH education landscape. They acknowledge limitations including:

• Reliance on self-reported data and the potential for response-shift bias inherent in retrospective pre/post designs.
• A modest sample size (n=28) drawn predominantly from a single institution (medical students).
• The lack of a control group and longitudinal follow-up to assess the durability of gains.
• The use of a study-developed instrument that had not undergone formal psychometric validation prior to administration.

Consequently, the paper concludes that while the results are promising, future research requires objective competency measures, longitudinal follow-up, and multi-site replication with validated instruments to strengthen the evidence base.