💻 computer science

A Discipline-Agnostic AI Literacy Course for Academic Research: Architecture, Pedagogy, and Implementation

This paper presents the design, theoretical framework, and successful implementation of a discipline-agnostic AI literacy course at Lehigh University that specifically targets rigorous AI-assisted literature review through a four-module architecture, demonstrating significant gains in student confidence regarding hallucination detection, responsible use, and attribution practices.

Original authors: Gideon K. Gogovi

Published 2026-05-01

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Gideon K. Gogovi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a group of students how to navigate a massive, chaotic library. In the past, they had to walk the aisles, read every book, and take notes by hand. Now, they have a super-fast, incredibly knowledgeable, but occasionally hallucinating robot assistant who can summarize books, organize shelves, and even write drafts for them in seconds.

The problem? Many students are letting the robot do the thinking for them. They trust the robot blindly, even when it invents fake books or mixes up facts. They get the "answer" but lose the "understanding."

This paper describes a new 13-week course designed to fix that. It's like a "Driver's Ed" class for using AI in academic research, but instead of teaching them how to build the car (coding AI), it teaches them how to be a safe, smart driver who knows when to take the wheel and when to let the autopilot help.

Here is the breakdown of the course, explained simply:

The Core Idea: The "Verified Engagement" Rule

The most important rule of this class is: You must be able to explain everything you submit, even if the AI helped you write it.

Think of it like a chef using a sous-chef. The sous-chef (the AI) can chop the vegetables and mix the sauce, but the head chef (the student) must taste the dish, know exactly what's in it, and be able to tell the customer, "Yes, I made this, and here is why it tastes good." If the head chef can't explain the recipe, they aren't the chef; they're just a delivery person.

The Course Structure: Four Steps to Mastery

The course is built like a ladder with four rungs. You can't skip a rung.

Step 1: Understanding the Individual Paper (The Detective)
- The Task: Students learn to read a single research paper.
- The AI Role: The AI acts as a translator. If a paper uses confusing jargon, the AI explains it.
- The Catch: The student must read the paper first or alongside the AI. They learn to spot when the AI invents a fake citation (a "hallucination") or gets a fact wrong.
- The Metaphilosophy: The AI is a flashlight, not a map. It helps you see the path, but you still have to walk it.
Step 2: Organizing the Knowledge (The Librarian)
- The Task: Now that they understand one paper, they have to organize 50 papers into a logical system (a "taxonomy").
- The AI Role: The AI helps sort the papers into categories (e.g., "studies about kids" vs. "studies about adults").
- The Catch: AI is bad at seeing deep patterns; it often groups things just because they share a keyword. Students learn to double-check the AI's sorting to make sure it actually makes sense.
- The Metaphilosophy: The AI is a filing clerk, but the student is the manager who decides if the filing system is actually useful.
Step 3: Finding the Missing Pieces (The Explorer)
- The Task: Look at the organized pile of papers and find what isn't there. Where is the gap in knowledge?
- The AI Role: The AI helps scan for trends and suggests where research is missing.
- The Catch: AI often lies about what is "important" or invents gaps that don't exist. Students learn to verify if a gap is real or just an illusion created by the AI.
- The Metaphilosophy: The AI is a compass, but the student has to decide which direction is worth exploring.
Step 4: Writing the Review (The Architect)
- The Task: Combine everything into a final literature review.
- The AI Role: The AI can draft paragraphs or check for consistency.
- The Catch: The final argument must be the student's. If the AI writes a paragraph, the student must rewrite it so deeply that it sounds like them.
- The Metaphilosophy: The AI provides the bricks and mortar, but the student designs the building.

How They Taught It (The "Secret Sauce")

The course had a few clever tricks to make sure students actually learned:

Wait to Scare Them: Instead of starting the class by saying, "AI lies a lot!", the teacher let students use AI for a few weeks first. By the time they learned about "hallucinations," the students had already seen the AI make mistakes themselves. It was like showing a student a flat tire after they tried to drive on one, rather than just talking about tires in a lecture.
The "Receipts" System: Every time a student used AI, they had to keep a detailed log: "I asked the AI this specific question, it gave me this answer, and I checked these three sources to make sure it was true." This log was graded. If you didn't have the receipts, you failed, even if the final paper looked good.
Mixed Classrooms: The class had both undergraduates and PhD students. The PhD students shared how AI worked differently in their specific fields (like biology vs. history), helping everyone see that AI isn't one-size-fits-all.

What Happened? (The Results)

The paper reports on the very first time this class was taught in Spring 2026. They asked students how confident they felt before and after the course.

The Big Wins: Students felt massively more confident in three areas:
1. Spotting AI lies: They went from "I have no idea" to "I can catch these errors."
2. Using AI responsibly: They learned exactly how to credit the AI and when not to use it.
3. Organizing research: They felt much better at sorting large piles of information.
The "Ceiling" Effect: For things they were already pretty good at (like just reading papers), their confidence didn't jump as much. This is normal; it's hard to get much better if you're already starting at a high level.
Student Feedback: Students said the hardest part was the amount of work (keeping the logs and checking everything), but they agreed it was worth it. One student said, "I learned how to use AI to structure my thinking, not to generate it."

The Bottom Line

This paper argues that we shouldn't ban AI in schools, and we shouldn't just teach people how to code it. Instead, we need to teach them how to be critical supervisors of AI.

The course proves that you can teach students to use AI as a powerful tool without letting it replace their own brains. It's about building a habit of asking, "Is this true?" and "Did I check this?" before hitting "submit." The goal isn't to make students AI-dependent; it's to make them AI-literate.

1. Problem Statement

The rapid integration of generative AI (GenAI) into academic workflows has outpaced curricular adaptation, creating a critical gap in AI research literacy. Current educational offerings fall into two inadequate categories:

Technical AI courses: Focused on building AI systems (ML/NLP), serving only specialist audiences and failing to address how researchers consume AI tools.
General literacy interventions: Brief workshops that raise ethical awareness but fail to develop the sustained, practice-based competencies required for rigorous research.

The specific consequence of this gap is unsupported AI use: students offload cognitive tasks (like literature reviews) without understanding tool limitations, verification protocols, or attribution standards. This leads to "AI dependency" rather than "AI literacy," characterized by shallow engagement with primary literature, hallucinated citations, and integrity violations. The paper identifies AI-assisted literature review as the most consequential domain for intervention due to its complexity and the unique risks of AI hallucination in scholarly synthesis.

2. Methodology and Course Architecture

The paper reports on the design, implementation, and preliminary evaluation of BSTA 495/395: Getting Started with AI-Assisted Research, a 13-week, prerequisite-free, discipline-agnostic course at Lehigh University (Spring 2026).

A. Pedagogical Framework

The course is grounded in three theoretical pillars:

Scaffolded Learning (Vygotsky): AI tools act as scaffolds to handle complexity, but students must develop independent analytical skills to eventually operate without them. The core principle is "Verified Engagement": students must independently explain and defend every claim, regardless of AI contribution.
Responsible Reliance Theory: Trust in AI must be calibrated to the specific task and tool reliability. For example, citation generation by general LLMs (high hallucination risk) requires 100% verification, whereas structured extraction from index-backed platforms requires only spot-checking.
Threshold Concepts: The course treats "AI as a cognitive scaffold (not replacement)" as a transformative threshold concept that reorients how students engage with research.

B. Four-Module Architecture

The curriculum is organized into four sequential modules mapping to the cognitive arc of a literature review:

Module A (Comprehension): Focuses on understanding individual papers. Students learn strategic reading (independent abstract/tables first, then AI for terminology) and critical evaluation of methodology.
Module B (Organization): Focuses on constructing and validating knowledge taxonomies. Students use AI to classify papers but must validate the structure using inter-rater reliability (Cohen's $\kappa$ ) to ensure conceptual distinctness.
Module C (Innovation): Focuses on gap identification. Students learn to distinguish between genuine research gaps and "spurious convergence" (AI grouping papers by keywords rather than concepts). This module includes a systematic typology of hallucinations (citation, statistical, theoretical) taught after students have experienced them firsthand.
Module D (Synthesis): Focuses on producing a complete literature review. The distinction between AI-assisted and AI-authored writing is central. Students must pass "revision depth" and "deletion" tests to prove authorial mastery.

C. Implementation Design

Tool-Agnostic: The course uses five platforms (ChatGPT, Claude, Elicit, Consensus, Semantic Scholar) to teach transferable competencies (prompt engineering, bias detection) rather than platform-specific skills.
Dual-Enrollment: Undergraduate (395) and Graduate (495) students learn together, fostering cross-disciplinary comparison of AI failure modes.
Assessment & Ethics: Ethics is treated as a subject of study, not just policy. A standardized AI Attribution Template is required for every assignment, documenting prompts, outputs, verification steps, and error corrections. The final grade heavily weights process (60% weekly assignments) over product (25% final review).

3. Key Contributions

Curriculum Model: Provides a replicable, modular architecture for AI research literacy that integrates technical proficiency, critical evaluation, ethics, and methodology into a single semester-long course.
Pedagogical Innovation:
- Experience-Before-Taxonomy Sequencing: Hallucination and bias instruction is placed after students have used tools extensively, turning abstract warnings into retrospective explanations of observed phenomena.
- Ethics-as-Subject: Moves beyond compliance policies to teach the principled judgment required for ethical AI use.
- Process-Centered Assessment: Uses AI use logs and attribution templates to make the research process visible and evaluable.
Design Principles: Establishes five core principles for future AI curricula: design for transfer (not tools), organize around research workflows, sequence failure-mode instruction after experience, treat ethics as subject, and assess process, not just product.

4. Results (Preliminary Outcome Evidence)

Data was collected from the inaugural Spring 2026 cohort ( $N=27$ pre-survey, $N=26$ matched pairs for analysis) using pre- and post-course surveys.

Confidence Gains: Significant increases were observed across all competency domains.
- Largest Gains:
  - AI Attribution Practice: $\Delta M = +2.00$ , $d = +2.40$ (Very Large).
  - Hallucination Detection: $\Delta M = +1.20$ , $d = +1.45$ (Very Large).
  - Responsible AI Use: $\Delta M = +1.03$ , $d = +1.33$ (Very Large).
  - Taxonomy Organization: $\Delta M = +0.91$ , $d = +1.23$ (Very Large).
- Moderate Gains: Reading papers, evaluating methodology, and recognizing limitations showed moderate gains ( $d \approx 0.5-0.7$ ), likely due to a "ceiling effect" where students entered with higher baseline confidence.
Skills Assessment: Students reported near-ceiling mastery in "AI as scaffold" ( $M=4.58$ ) and "Ethical AI practice" ( $M=4.50$ ).
Qualitative Feedback: Students identified verification/hallucination detection and prompt engineering as the most valuable skills. They noted the course helped them structure their thinking rather than just generate content.
Workload: Students reported weekly assignments were time-intensive, suggesting a need for future optimization of documentation requirements.

5. Significance

This paper establishes a foundational model for the emerging genre of AI research literacy curricula. Its significance lies in:

Bridging the Gap: It addresses the specific need for researchers to use AI as a consumer of tools, not just a developer, filling a void between technical CS courses and superficial ethics workshops.
Scalability: The discipline-agnostic, prerequisite-free design allows institutions to deploy this curriculum across diverse fields (STEM, Humanities, Social Sciences) without requiring prior technical knowledge.
Redefining Integrity: By shifting the focus from "banning AI" to "teaching verified engagement," the course prepares students for a future where AI is ubiquitous in research. It provides a concrete framework for authorial ownership in the age of generative AI.
Empirical Validation: The preliminary data demonstrates that a structured, scaffolded approach can rapidly and significantly improve student confidence and competency in high-risk areas like hallucination detection and attribution, validating the course's theoretical design.

The paper concludes that AI research literacy is a teachable, transferable competency that must be integrated into the core of research methods education to ensure the integrity and rigor of future scholarly work.