Multilevel implementation assessment of a national health and life skills curriculum policy: development and content validation
This study details the development and content validation of the Health and Life Skills Implementation Assessment (HLS-IA), a coordinated multilevel instrument designed to evaluate the local enactment of a national health curriculum policy through iterative stakeholder engagement, expert review, and pilot testing.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Schools are more than places for learning math and history; they are where children spend a large part of their lives, making them powerful settings for improving health and teaching young people how to navigate the world. For decades, researchers have tried to measure how well schools succeed at these health goals. Usually, this involves checking if a specific program was followed exactly as written, like a recipe. But what happens when the goal is not a single recipe, but a broad idea woven into the entire school day? In Norway, a major education reform introduced a new, flexible topic called Health and Life Skills. Instead of a strict manual, it gave teachers and school leaders the freedom to decide how to teach it, trusting their professional judgment to fit the lessons into their specific classrooms. This approach is smart and adaptable, but it creates a difficult problem for scientists: how do you measure something that changes shape depending on who is doing it and where it is happening? If every school interprets the rules differently, how can researchers tell if the policy is working, or if it is simply being ignored?
To solve this puzzle, a large team of researchers from universities and education organizations across Norway set out to build a new kind of measuring tool. They did not want to use old questionnaires designed for rigid, one-size-fits-all programs, because those would miss the nuance of this flexible reform. Instead, they spent years developing a set of four connected surveys, one for each level of the school system: the people who own and fund the schools, the principals who run them, the teachers who do the work, and the actual lessons happening in the classroom. The goal was to create a tool that could capture the story of implementation from the top down to the very last minute of a class. They called this new tool the Health and Life Skills Implementation Assessment.
The process of building this tool was careful and deeply collaborative. The researchers started by looking at how other studies had measured school programs, but they quickly realized those methods would not work here. They brought in experts from education, psychology, and public health, but they also invited the people who actually do the job: school leaders, teachers, and union representatives. In workshops and meetings, these groups helped decide what questions mattered most. They asked themselves what a school owner needs to know to support a teacher, or what a teacher needs to feel capable of teaching life skills. This back-and-forth ensured the questions used the right words and focused on real-world challenges rather than abstract theories.
Once the questions were drafted, the team tested them rigorously. They sat down with teachers and principals for one-on-one interviews, asking them to read the questions out loud and explain what they thought each one meant. This step, known as cognitive interviewing, revealed hidden problems. Some questions were confusing, others used jargon that didn't fit the local context, and some response options didn't allow for the full range of experiences a teacher might have. The researchers listened to every comment and rewrote the surveys repeatedly. They also ran small pilot tests to see how long it took to finish and whether people got tired of answering. Through this iterative process, they refined the tool until it was clear, practical, and sensitive to the unique way Norwegian schools operate.
The final result was a coordinated system of four instruments. The surveys for school owners and principals asked about resources, priorities, and the overall climate of the school. The teacher survey asked about their own training, confidence, and how they planned to teach the subject. Crucially, the team also created a lesson log for teachers to fill out after specific classes. This log asked teachers to record what they had planned to teach versus what they actually ended up teaching, capturing the difference between intention and reality. This distinction is vital because in a flexible system, a teacher might plan a big lesson but end up having a quick, spontaneous conversation about health instead, and both are valuable data points.
In 2025, the team put their new tool to the test on a national scale. They invited school owners, principals, and thousands of teachers across the country to participate. The response rates were mixed, which is common in large-scale school research, but they gathered enough data from 458 schools to show that the system worked. They successfully collected information from all four levels of the school hierarchy, proving that it is possible to gather coordinated data across a whole national system without forcing everyone into the same box. The data showed that while some schools were deeply engaged with the new curriculum, others were still figuring out how to fit it in, and the tool captured this variation without judging it as a failure.
The study did not claim to have solved the question of whether the new curriculum improved student health outcomes; that is a job for future research. Instead, the paper demonstrated that it is possible to build a measurement tool that respects professional judgment and local adaptation. By creating a system that listens to the different voices in the school ecosystem, the researchers provided a way to understand how broad national policies actually land in the classroom. They showed that when a policy is flexible, the measurement tool must be flexible too, capable of seeing the shape of the work rather than just checking off a list of requirements. This approach offers a blueprint for studying other complex education and health initiatives where the goal is not uniformity, but thoughtful, local adaptation.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.