Development and Validation of a Faculty Artificial Intelligence Literacy and Competency (FALCON-AI) Scale for Higher Education
Grounded in the Critical Tech-resilient Literacies framework, this study develops and validates the 23-item FALCON-AI Scale through expert review and confirmatory factor analysis to provide a reliable, role-specific instrument for assessing faculty AI literacy across teaching, research, and service domains in higher education.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are walking into a massive, bustling university campus. For decades, the professors there have been the masters of their craft: teaching students, discovering new knowledge, and running the show. But recently, a new, incredibly powerful, and sometimes confusing guest has arrived: Artificial Intelligence (AI).
AI is like a super-smart, super-fast intern that can write essays, analyze data, and grade papers in seconds. But here's the problem: Just because the intern is there doesn't mean everyone knows how to use them well. Some professors are terrified of it, some are using it recklessly, and many are just confused about what it can and cannot do.
This paper is about building a specialized "driver's license test" for university professors to see how well they can handle this new AI intern. The researchers call this test the FALCON-AI Scale.
Here is the story of how they built it, explained simply:
1. The Problem: No Map for the New Terrain
Before this study, there were already tests to measure AI skills. But they were like driving tests for different vehicles:
- Some were for kids (learning what a car is).
- Some were for K-12 teachers (learning how to drive a school bus).
- Some were for the general public (learning how to ride a bike).
But there was no test for university professors. A professor's job is unique. They aren't just teaching; they are also doing complex research and running administrative departments. They needed a test that understood their specific "job description."
2. The Blueprint: The "CTRL" Framework
To build this test, the researchers used a blueprint called the CTRL Framework. Think of this as a three-layer cake that every professor needs to eat to be "AI Resilient":
- Layer 1: Functional Literacy (The "How-To" Cake): Can you actually use the tool? Can you ask the AI the right questions (prompts) to get it to write a syllabus or analyze data?
- Layer 2: Evaluative Literacy (The "Sniff Test" Cake): Can you tell if the AI is lying or hallucinating? Can you look at two different AI tools and say, "Tool A is better for writing, but Tool B is better for math"?
- Layer 3: Ethical Literacy (The "Conscience" Cake): Just because you can do something, should you? Is it fair to let students use AI on a test? Are you stealing data? This layer is about making moral decisions.
The researchers took these three layers and spread them across the four main areas of a professor's life:
- General (Just knowing about AI)
- Teaching (Using AI in the classroom)
- Research (Using AI for science and papers)
- Service (Using AI for admin work and committees)
This created a 3x4 grid (12 different boxes of skills) to check.
3. Building the Test: From 43 Questions to 23
The team started with a messy pile of 43 questions. It was too long and confusing.
- The Human Experts: They invited four professors who are AI experts to read the questions. They said, "This question is too vague," or "This doesn't make sense for a researcher."
- The Robot Expert: Here's the cool part. They also asked an AI (ChatGPT) to grade the questions! They asked the AI to pretend it was a professor and rate the questions for clarity and importance.
- The Result: The humans and the AI agreed on most things. They cut out the bad questions, fixed the confusing ones, and ended up with a sleek, 23-question test.
4. The Pilot Test: Trying it Out
They gave this new 23-question test to 269 university staff and professors.
- They asked: "On a scale of 1 to 5, how good are you at these things?"
- They ran the numbers through a statistical machine (called Confirmatory Factor Analysis) to see if the test actually measured what it was supposed to.
The Verdict: The test worked!
- It successfully separated the three layers (Functional, Evaluative, Ethical).
- It successfully separated the four job areas (Teaching, Research, Service, General).
- The questions were consistent (if you were good at one ethical question, you were likely good at the others).
5. Why This Matters (The "So What?")
Imagine a university wants to train its professors on AI. Without this test, they are flying blind. They might give a workshop on "How to write prompts" to a professor who already knows that but needs help with "AI Ethics."
With the FALCON-AI Scale, the university can:
- Diagnose: "Oh, our professors are great at using AI (Functional), but terrible at checking if it's lying (Evaluative)."
- Train: They can build workshops specifically to fix those weak spots.
- Measure: They can give the test again after the training to see if the professors actually improved.
The Bottom Line
This paper is the story of creating a compass for professors navigating the AI jungle. It's not about making professors into computer scientists; it's about giving them a clear map to ensure they use this powerful new tool wisely, ethically, and effectively in their teaching, research, and daily work.
In short: They built a custom-made "AI Driver's License" for professors, tested it with real people, and proved it works. Now, universities have a way to make sure their faculty are ready for the future.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.