The Relatives Experience Questionnaire for Acute Inpatient Child and Adolescence Mental Health Services (REQ-AICAMHS): reliability and validity following a Norwegian survey

This study aims to develop and psychometrically evaluate the first instrument, the Relatives Experience Questionnaire for Acute Inpatient Child and Adolescent Mental Health Services (REQ-AICAMHS), to measure and improve the experiences of relatives of adolescents in Norwegian acute inpatient mental health units.

The Gist

Imagine a hospital ward for teenagers with mental health struggles as a busy, high-stakes airport. The teenagers are the travelers going through a difficult journey, and their parents or family members (the "relatives") are the people waiting in the terminal, trying to help them navigate the chaos.

This paper is essentially a blueprint for building a new feedback survey specifically designed for those family members waiting in the terminal.

Here is the breakdown of the project using simple analogies:

1. The Problem: Flying Blind

Currently, the airport managers (the mental health services) don't have a good way to ask the family members how the journey felt. They have surveys for adult travelers and for the teenagers themselves, but nothing that specifically asks the family, "How was the experience for you while your child was in the hospital?"

Without this feedback, the airport is flying blind. They don't know if the waiting area is too loud, if the staff was too confusing, or if the family felt supported. The authors say that to fix the airport, you need to listen to the people waiting outside.

2. The Solution: The "REQ-AICAMHS"

The researchers are creating a new tool called the Relatives' Experience Questionnaire (REQ-AICAMHS). Think of this as a custom-made "Customer Satisfaction Report Card" for the families.

• Who is it for? Parents, guardians, and relatives of teenagers admitted to acute mental health units in Norway.
• What does it ask? It asks about the "structure" (was the building clean and safe?), the "process" (did the staff talk clearly and kindly?), and the "outcome" (did the family feel the treatment helped?).

3. How They Built It: The "Recipe"

The researchers didn't just guess what questions to ask. They followed a strict recipe developed by the Norwegian Institute of Public Health:

• Reading the Cookbook: They looked at existing surveys and research to see what questions usually work.
• Consulting the Chefs: They talked to experts (doctors, researchers, and family advocates) to get their opinion on the questions.
• Tasting the Dish: They interviewed real family members to make sure the questions made sense and weren't too confusing.
• The Taste Test (Cognitive Testing): They had a small group of relatives try out the draft survey to find any "bugs" or confusing wording before rolling it out to everyone.

4. The Big Test: The "Taste Test" for the Whole Country

Once the survey is ready, they plan to send it out to all 17 of these specialized mental health units across Norway.

• The Sample: They want to hear from the families of almost every teenager admitted to these units.
• The Method: They will send digital invitations (like a text message or email link) to the families.
• The Goal: They need a lot of responses to make sure the results are accurate, like needing a large number of taste-testers to know if a new ice cream flavor is actually good.

5. Checking the Quality: The "Lab Test"

Before they can trust the results, they have to prove the survey itself is a good tool. They will use advanced math (statistics) to check:

• Reliability: If you asked the same question twice, would you get the same answer? (Is the ruler straight?)
• Validity: Does the survey actually measure what it's supposed to measure? (Does the thermometer actually measure temperature, or just humidity?)
• Trimming the Fat: They plan to analyze the data to see which questions are the most useful. If a question is confusing or doesn't add value, they will cut it out. The goal is to create a shorter, punchier version of the survey that is easier to use in the future.

6. The "Secret Sauce": Open-Ended Questions

Besides the standard "1 to 5 star" ratings, the survey includes a space for families to write their own stories. The researchers plan to use both human reading and computer programs (machine learning) to scan these stories. This helps them find hidden themes, like "everyone complained about the waiting room" or "everyone loved the food."

Summary

In short, this paper describes the construction and testing phase of a new tool. The researchers are building a specialized "report card" for families of hospitalized teenagers, testing it rigorously to make sure it works, and planning to use the results to help Norwegian hospitals improve their care.

Important Note: The paper explicitly states that this is a preprint (a draft) and has not yet been peer-reviewed by other scientists. It is a proposal and a plan for how they will analyze the data, rather than a report on final, proven clinical results. The goal right now is to prove the survey tool works, not to change medical treatments yet.

Technical Summary

Technical Summary: The Relatives' Experience Questionnaire for Acute Inpatient Child and Adolescence Mental Health Services (REQ-AICAMHS)

Problem Statement
Adolescents with mental health disorders represent a vulnerable population with complex care needs, yet their experiences and those of their relatives in acute inpatient mental health settings remain poorly understood. While Patient-Reported Experience Measures (PREMs) are recognized as essential for improving healthcare quality, there is a significant lack of validated instruments specifically designed for child and adolescent mental health inpatient (CAMHS) settings. Existing tools, such as the Experience of Service Questionnaire (ESQ), are primarily designed for outpatient care and focus on satisfaction rather than the specific experience domains relevant to inpatient stays (e.g., privacy, activities, and family engagement). Furthermore, systematic collection of individual user experiences in Norwegian acute inpatient units is currently uncoordinated and lacks validated measurement tools, hindering the ability to assess service performance from the user perspective and limiting potential improvements in adherence and outcomes.

Methodology
This study outlines the development and psychometric evaluation of the Relatives' Experience Questionnaire for Acute Inpatient Child and Adolescent Mental Health Services (REQ-AICAMHS). The research follows the standard methodology established by the Norwegian Institute of Public Health (NIPH).

• Instrument Development: The questionnaire is constructed through a multi-step process including a comprehensive literature review, expert consultations (clinicians, researchers, and patient organization representatives), qualitative interviews with relatives, and cognitive testing. The final instrument includes closed-ended items using a standardized five-point Likert scale (1 = not at all to 5 = to a very large extent) and open-ended questions for qualitative feedback.
• Data Collection: The study targets relatives of all adolescents admitted to 17 acute inpatient CAMHS units across Norway. Data collection is digital, utilizing the Norwegian Patient Registry (NPR) and Helsenorge.no for invitations. The protocol includes a pilot phase with 12–15 relatives for qualitative interviews and cognitive testing, followed by a larger survey. Data collection aims to occur either during the inpatient stay (preferred) or shortly after discharge.
• Psychometric Analysis: The study employs a rigorous statistical framework to assess data quality, factor structure, construct validity, and internal consistency.
  • Descriptive Statistics: Analysis of missing data, ceiling effects, and item-level means. Items with >20% missing data are excluded.
  • Factor Analysis: Exploratory Factor Analysis (EFA) using principal-axis factoring with promax rotation to identify underlying structures, guided by Donabedian's model of structure, process, and outcome.
  • Reliability: Internal consistency is evaluated using Cronbach's alpha (target ≥0.7) and item-total correlations.
  • Validity: Construct validity is tested via internal correlations and group differences. Discriminant validity is assessed by analyzing item correlations with hypothesized scales versus other scales.
  • Item Response Theory (IRT): The Generalized Partial Credit Model (GPCM) is applied to estimate item discrimination ('a'), difficulty thresholds ('b'), and model fit (S–χ²) to support the development of a shortened questionnaire version.
  • Qualitative Analysis: Open-ended responses are analyzed using both established qualitative methods and machine learning techniques to identify key themes and subdomains.

Key Contributions

• First Specialized Instrument: The REQ-AICAMHS is the first instrument specifically designed to capture the experiences of relatives in Norwegian acute inpatient child and adolescent mental health units.
• Methodological Rigor: The study applies a comprehensive validation framework combining classical test theory (EFA, Cronbach's alpha) with modern psychometric approaches (IRT/GPCM) and machine learning for qualitative data.
• Foundation for Quality Improvement: By establishing a validated tool, the study aims to fill the gap in routine PREMs for this specific setting, enabling systematic quality monitoring and improvement initiatives across the national network of 17 units.
• Dual-Perspective Approach: While this paper focuses on relatives, it is part of a broader project that also includes a corresponding questionnaire for adolescents, promoting a person-centered approach that includes both the patient and their support network.

Results
Note: As this is a protocol paper describing the study design and planned analysis, specific empirical results (e.g., specific factor loadings, exact Cronbach's alpha values, or specific IRT parameters) are not yet presented. The "Results" section of the paper outlines the anticipated outputs:

• Descriptive statistics of the respondent population (sex, age, background).
• Item-level analysis including missing data rates, mean values, and ceiling effects.
• Factor structure, factor loadings, and reliability coefficients from the EFA.
• Correlation matrices between items and scales.
• Construct validity testing results regarding associations between scales, background variables, and responses.
• IRT parameters including discrimination values, difficulty thresholds, and item fit statistics.
• The identification of a shortened version of the questionnaire based on the most informative items.

Significance and Claims
The paper claims that the development of the REQ-AICAMHS is critical for addressing the current lack of validated PREMs in acute inpatient CAMHS. By capturing the perspective of relatives—who are often deeply involved in the care process—the study aims to provide insights into accessibility, communication, coordination, and treatment experiences that are currently unmeasured. The authors posit that this instrument will facilitate evidence-based quality improvement, support the development of a shortened version for broader implementation, and ultimately contribute to a more patient-centered healthcare system for adolescents with mental health disorders in Norway. The study emphasizes that without such validated instruments, assessing service performance from the user perspective remains impossible, thereby diminishing the potential for improved clinical outcomes.