💻 computer science

Four Types of LLM Reliance and Their Predictors Among Undergraduate Writers: A Mixed-Methods Study at a Minority-Serving R1 University

This mixed-methods study at a minority-serving R1 university identifies four distinct types of undergraduate LLM reliance (Strategic, Instrumental, Dialogic, and Dependent), revealing that AI literacy predicts reliance type while value and cost beliefs predict intensity, and highlighting how current assessment methods inadvertently penalize strategic users who demonstrate the greatest independent thinking.

Original authors: Shahin Hossain

Published 2026-06-30

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Shahin Hossain

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a classroom where almost every student has a super-smart, invisible writing assistant (an AI) sitting on their shoulder. For years, researchers have been trying to figure out how much students are using this assistant. But they've been asking the wrong question. They've been asking, "How often do you use the AI?" as if using it a lot is the same as using it badly.

This study says: Stop counting the minutes; start looking at the method.

Here is the simple breakdown of what the researchers found, using some everyday metaphors.

1. The Four Types of "AI Drivers"

The researchers discovered that students don't just "use" AI; they drive it in four very different ways. Think of the AI as a car, and the student as the driver.

The Strategic Driver (34%): These students treat the AI like a co-pilot. They have a map (their own ideas), and they ask the co-pilot to check the route, point out traffic, or verify a landmark. They stay in the driver's seat the whole time. They are the most in control.
The Instrumental Driver (31%): These students use the AI like a power tool. They need to tighten a specific screw (fix a grammar error) or sand a rough edge (summarize a paragraph). They use the tool for a quick job, then put it down and keep working themselves.
The Dialogic Driver (30%): These students treat the AI like a sparring partner. They bounce ideas back and forth. "What if I say this?" "No, try that." They are having a conversation to build their ideas together, but they are still the one making the final decisions.
The Dependent Driver (4.5%): These students let the AI take the wheel. They tell the AI, "Drive me to the destination," and they sit back and watch the scenery. They aren't steering; they are just along for the ride.

2. The Big Surprise: The "Score Trap"

Here is the most confusing part of the study, and why it matters so much.

When the researchers looked at the grades and "writing quality" scores, the Strategic Drivers (the ones in control) had the lowest scores. The Dependent Drivers (the ones who let the AI drive) had the highest scores.

Why?
The researchers realized the tests were rigged. The questions on the tests were like asking, "How much did the GPS help you get to the store?"

The Strategic Driver says, "Not much, I knew the way." So they get a low score.
The Dependent Driver says, "The GPS did everything!" So they get a high score.

The study calls this a "Measurement Artifact." It's like weighing a fish on a scale that only measures how much water is in the bucket, not the weight of the fish. The tests were measuring how much the AI did, not how good the student's writing was. The students who thought the most and used the AI the least were accidentally punished by the grading system.

3. Two Different Engines

The study found that two different things control how students use AI, and you can't fix one by fixing the other.

Engine A: "How much do you use it?" (Intensity)
This is driven by motivation. Do you think the AI will make your life easier? Do you feel like you can't finish the work without it? If you feel high pressure or think the AI is a magic shortcut, you will use it a lot, regardless of whether you are a Strategic or Dependent driver.
- Analogy: This is like how hungry you are. If you are starving, you will eat a lot, whether you are eating a healthy salad or a whole pizza.
Engine B: "How do you use it?" (Type)
This is driven by AI Literacy (knowing how the AI works). If you understand that the AI can lie, make things up, or be biased, you will naturally become a Strategic or Dialogic driver. You won't let it take the wheel.
- Analogy: This is like knowing how to cook. If you know how to cook, you might use a microwave to heat up leftovers (Strategic). If you don't know how to cook, you might just order takeout and pretend you made it (Dependent).

4. The "Ethical Abstainers"

There was a small group of students (about 13%) who didn't fit into any of the four driving categories. They simply refused to use the AI at all.

Some refused because of ethical reasons (it feels like cheating).
Some refused because of environmental reasons (AI uses too much energy).
Some refused because they believed it stunted their brain.

The study found that current tests can't tell the difference between a Strategic driver who chooses not to use the AI much, and an Ethical Abstainer who never uses it. They both look the same on the test (low scores), but their reasons are completely different.

5. Why This Matters for Schools

The study was done at a university with many first-generation and minority students. The researchers found that students who didn't have other support systems (like family connections to writing centers) relied more heavily on AI to catch up.

The big takeaway for schools is: Don't just try to stop students from using AI.

If you only try to reduce the amount they use (the intensity), you might accidentally stop the smart students from using it as a helpful tool.
Instead, schools need to teach how to use it (literacy) so students become Strategic Drivers, and they need to understand why students feel they need it so much (motivation).

Summary

Old Way: "How often do you use AI?" (Bad question).
New Way: "Are you the driver, the co-pilot, or the passenger?"
The Problem: The current tests reward the passengers and punish the drivers.
The Fix: We need new tests that measure the student's brain, not the AI's output, and we need to teach students how to drive the AI, not just how to sit in the back seat.

Technical Summary: Four Types of LLM Reliance and Their Predictors Among Undergraduate Writers

Problem Statement
Despite near-universal adoption of Large Language Models (LLMs) in undergraduate academic writing (approx. 92% of students), the field lacks validated frameworks to distinguish qualitatively different forms of AI reliance. Existing research operationalizes reliance as a single, continuous frequency construct (e.g., frequent vs. infrequent users), which obscures critical distinctions in cognitive engagement. This structural limitation generates outcome measures that systematically reward cognitive dependency over intellectual authorship. Specifically, current instruments measure "AI throughput" (how much AI contributed to a task) rather than writing quality or cognitive growth, leading to a measurement artifact where students exercising the highest cognitive agency (strategic restraint) appear as the weakest performers. Furthermore, the literature has not adequately examined these dynamics within Minority-Serving Institutions (MSIs), where preparation heterogeneity and first-generation status may compound AI literacy disparities.

Methodology
This study employed a sequential explanatory mixed-methods design at a public R1 Minority-Serving Institution (MSI) and Asian American and Native American Pacific Islander-Serving Institution (AANAPISI).

Participants: The quantitative phase included $N=382$ undergraduate students. The qualitative phase involved 14 purposively sampled interviews, 35 open-ended survey responses (SUR35), and 361 brief open-ended responses (SUR361), totaling 1,435 coded instances.
Theoretical Framework: The study integrated three frameworks:
1. AI Literacy Framework: To predict the type (qualitative direction) of reliance.
2. Expectancy-Value Theory (EVT): To predict the intensity (quantitative magnitude) of reliance.
3. Biggs's Presage-Process-Product (3P) Model: To provide structural architecture linking student characteristics, reliance behaviors, and outcomes.
Instrumentation: A new 20-item LLM Reliance Scale was developed and validated, measuring four distinct types: Strategic, Instrumental, Dialogic, and Dependent. Predictor measures included AI Literacy (Core and Exposure) and Expectancy-Value beliefs.
Analysis: Quantitative analysis involved exploratory factor analysis (EFA), reliability testing, one-way ANOVA, hierarchical multiple regression, and multinomial logistic regression. Qualitative analysis utilized reflexive thematic analysis to validate the typology and explain quantitative patterns.

Key Contributions

Empirically Validated Typology: The study identifies and validates four distinct LLM reliance types:
- Strategic (34.3%): Deliberate, goal-directed engagement with active verification and full authorial control.
- Instrumental (30.9%): Bounded use for mechanical tasks (outlining, editing) without delegating ideational content.
- Dialogic (30.4%): Iterative co-construction where AI acts as a thinking partner while authorial agency is preserved.
- Dependent (4.5%): Cognitive abdication involving wholesale outsourcing of writing tasks with minimal oversight.
Two-Tier Predictive Architecture: The study establishes that two separate, non-interchangeable systems govern reliance:
- AI Literacy predicts which type of reliance a student adopts (e.g., higher literacy predicts Strategic over Dependent).
- Expectancy-Value Beliefs predict how intensely a student relies on LLMs, regardless of type.
Measurement Artifact Identification: The study exposes a fundamental validity crisis in existing LLM outcome research. Instruments structured around "I use AI to achieve X" capture AI contribution rather than independent writing quality. Consequently, Strategic users—who deliberately limit AI's role—score lowest on these measures, creating a false impression of underperformance.
Two-Model Architecture: Qualitative findings revealed a population (13% of respondents) of "principled non-users" who abstain based on ethical or environmental values. These students are distinct from low-intensity Strategic users and fall outside the explanatory scope of existing reliance frameworks, necessitating a separate abstention model.

Results

Distribution: Strategic (34.3%), Instrumental (30.9%), Dialogic (30.4%), and Dependent (4.5%) were the dominant types.
Outcome Disparities: Quantitative results showed Strategic users scoring significantly lower than Dependent users on all ten writing process and outcome variables (e.g., Quality, Originality, Critical Thinking), with large effect sizes ( $\eta^2$ range .166–.329).
Predictive Modeling:
- Intensity: Expectancy-Value beliefs were the strongest predictor of reliance intensity ( $\beta = .630$ , $\Delta R^2 = .256$ ), explaining more variance than demographics or AI literacy combined.
- Type: AI Literacy was the strongest predictor of reliance type. Each unit increase in AI literacy reduced the odds of Dependent vs. Strategic classification by approximately tenfold ($OR = 0.11, p < .001$).
- Demographics: First-generation status was a significant predictor of higher reliance intensity ( $\beta = .153$ ), suggesting a compensation dynamic where AI supplements academic resources.
Qualitative Validation: Thematic analysis confirmed the typology and revealed developmental trajectories (movement from Dependent to Strategic over time) and the existence of principled non-use driven by ethical constraints rather than low literacy.

Significance and Claims
The paper claims that the field's current understanding of LLM outcomes is structurally flawed. The finding that Strategic users score lowest is not evidence of poor performance but a measurement artifact that penalizes cognitive agency. The study argues that:

Interventions must be distinct: AI literacy programs should target the type of engagement (shifting from uncritical to critical), while motivational interventions must address the intensity of engagement. Treating them as interchangeable is ineffective.
Equity implications are central: At MSIs, AI literacy disparities compound existing structural inequities. First-generation students may rely more heavily on AI to compensate for resource gaps, and current outcome measures misrepresent their sophisticated restraint as failure.
Future instrumentation: The field requires new instruments that distinguish between AI-mediated attainment and independent writing competence to avoid systematically misrepresenting the most cognitively sophisticated students.

The study concludes that without addressing these measurement and theoretical gaps, the evidence base regarding LLMs in education will remain distorted, rewarding dependency and obscuring the benefits of principled, strategic AI use.