Measuring AI Readiness in Health Professions Education: Development and Validation of the AI Competence and Preparedness Evaluation Resource (AICPER)

Imagine you're trying to teach a whole school of future doctors, nurses, and public health experts how to use a new, super-smart robot assistant. Before you can build the perfect training manual, you need to know exactly what the students think about the robot. Do they love it? Are they scared it will make mistakes? Do they think it's cheating if they use it for homework?

For a long time, the tools used to ask these questions were a bit like using a ruler to measure temperature—they just didn't fit the job. They were often too simple, focused only on one type of student, or written before the explosion of "generative AI" (the kind of AI that writes essays and creates images).

Enter the AICPER. Think of this as a brand-new, high-tech "AI-Attitude Thermometer" designed specifically for health students in the United States. A team of researchers at Rutgers University built this tool to see exactly how nursing, pharmacy, and public health students feel about artificial intelligence.

The Big Discovery: It's Not About Where, It's About How

When the researchers first built the AICPER, they thought they were measuring three specific places where AI is used: Clinical Practice (in the hospital), Research (in the lab), and Education (in the classroom). They expected the answers to sort themselves into these three neat piles.

But here's the twist: the students' brains didn't sort the answers that way. When the researchers ran the numbers, the "piles" rearranged themselves into three completely different categories based on feelings rather than locations.

1. Functional Benefits (The "Cool Tool" Factor): This group measures how much students think AI will make their jobs easier, faster, and smarter. It's like asking, "Will this robot help me carry heavy boxes?"
2. Ethics and Trust (The "Worry" Factor): This group captures the fears. Will the robot make mistakes? Is it biased? Can we trust it with patient secrets? It's the "Is this robot going to steal my job or lie to me?" pile.
3. Educational Engagement (The "Let's Learn" Factor): This measures students' level of engagement with AI-related learning. It's the "I want to take a class on this" pile.

The paper found that this new three-part structure is a much better way to understand students than just asking, "Do you like AI in the hospital?" A student might think AI is a fantastic tool for saving time (High Functional Benefits) but still be terrified of its ethical risks (Low Ethics and Trust). The old tools would have missed that nuance, but AICPER catches it.

How They Tested It (The "Stress Test")

To make sure this new thermometer actually worked, the researchers didn't just guess. They put it through a rigorous stress test involving 193 students and 7 expert judges.

• The Experts: Seven smart people with advanced degrees checked every single question to make sure it made sense. They agreed that the questions were relevant, giving the tool a "content validity" score of 0.86 (which is a very good score).
• The Students: 193 students from nursing, pharmacy, and public health programs took the survey.
• The Math: The researchers used some heavy-duty statistics (like "Exploratory Factor Analysis" and "Confirmatory Factor Analysis") to see if the questions stuck together properly. The result? The tool held up beautifully. The internal consistency score was 0.86, which means the questions reliably measured the same underlying feelings.

What the Numbers Say

The study didn't just say "it works." It gave specific numbers to back it up:

• The final tool has 18 questions (they dropped two because they didn't fit well).
• The three factors (Functional Benefits, Ethics, and Engagement) are distinct but related, with correlations between 0.42 and 0.61.
• The tool works the same way for nursing, pharmacy, and public health students. The researchers checked this by comparing the groups and found the difference in how the tool measured them was tiny (-0.001), meaning a nurse and a pharmacist are answering the questions in the same "language."

What It's NOT (The "No" List)

It's important to know what this paper doesn't claim.

• It's not a magic wand: The paper explicitly states this is a "preliminary" step for some parts. They used the same group of students to both discover the structure and confirm it. While they used advanced math (bootstrap resampling with 2,000 repeats) to prove the results were stable, they admit that testing it on a different group of students is the next necessary step.
• It's not perfect yet: The researchers noticed that some of the "reverse-scored" questions (the ones where you have to think "no" to mean "yes") created a little bit of extra noise in the data. About 24.8% of the variation in those specific answers was due to the way the question was phrased, not just the student's opinion. They suggest future versions should rephrase these to be clearer.
• It's not for everyone (yet): The study only included nursing, pharmacy, and public health students. It did not include medical students (future doctors), so we can't be sure if it works for them just yet.

The Bottom Line

The AICPER is a rigorously tested, 18-item tool that successfully measures how health students feel about AI. It proved that students' attitudes aren't just about "where" they use AI, but about three core feelings: utility, trust, and engagement.

The researchers suggest that school leaders can use this tool to find out exactly where their students are stuck. If a class is great at seeing the benefits but scared of the ethics, they can build a specific lesson to fix that fear. If they are ethically sound but don't want to learn the tech, they can try a different teaching approach.

While the paper stops short of calling it a "final solution" (since more testing is needed), it provides the first solid, validated map for navigating the complex world of AI readiness in health education. It turns a foggy guess into a clear, three-dimensional picture.