Counsellor review of artificial intelligence recommendations improves feasibility adjusted career fit and reduces socioeconomic disparity

Technical Summary: Counsellor Review of Artificial Intelligence Recommendations

Problem Statement
As artificial intelligence (AI) increasingly mediates school-to-work transitions, recommender systems are being deployed to guide students' educational and career decisions. Current evaluation of these systems relies heavily on predictive accuracy and user acceptance. However, this focus overlooks critical dimensions for an inclusive future of work: who benefits across socioeconomic positions, whether recommendations are practically feasible for the student, and how guidance affects student agency. The paper identifies three "blind spots" in current career-guidance AI:

1. Feasibility: Recommendations may align with interests and ability but remain practically unpursuable due to financial, geographical, or social constraints (conversion factors) that models often ignore.
2. Distribution: Average improvements in recommendation quality can coexist with widening socioeconomic gaps.
3. Agency: Treating guidance as a pure prediction problem abstracts away the social context, potentially depressing student self-efficacy or inducing uncritical deference to algorithms.

The central research question is: When AI recommends careers to students, who benefits across socioeconomic positions, and does adding explanation or human oversight change the answer?

Methodology
The study employs a two-pronged approach combining a transparent policy simulation with a randomized field experiment.

• Study 1: Transparent Policy Simulation

  • Design: A simulation of 20,000 student profiles varying in interests, latent ability, and socioeconomic status (SES).
  • Variables: The model incorporates "conversion factors" (financial capacity, mobility) that determine whether a pathway is feasible.
  • Policies Tested: Five guidance policies were compared across baseline, technology-boom, and downturn labor-market scenarios:
    1. Conventional guidance (counsellor heuristics).
    2. Accuracy-optimised AI (maximizes predicted fit, ignores feasibility).
    3. Demographic-parity-constrained AI (equalizes access to high-opportunity careers).
    4. Constraint-aware AI (optimizes fit $\times$ feasibility).
    5. Human-in-the-loop AI (counsellor reviews AI shortlist).
  • Outcome: Simulated Feasibility-Adjusted Fit (FAF), defined as the alignment between recommendation and student attributes, adjusted for practical capacity.

• Study 2: Randomized Experiment

  • Participants: 987 secondary and university students across 14 institutions in India.
  • Design: A four-arm between-subjects experiment:
    1. Counsellor-only guidance.
    2. AI-only guidance (no rationale).
    3. Explainable AI guidance (rationale + uncertainty statement).
    4. AI + Counsellor review (counsellor reviews AI output and can modify/replace recommendations based on feasibility).
  • Measures: Primary outcome was Perceived Feasibility-Adjusted Fit (FAF). Secondary outcomes included decision clarity, agency, uncertainty awareness, algorithmic deference, trust, and perceived fairness.
  • Analysis: Linear mixed models with institution random intercepts and wild-cluster bootstrap for socioeconomic interaction effects.

Key Results

• Study 1 (Simulation):

  • Accuracy vs. Benefit: Accuracy-optimised AI achieved the highest unadjusted fit but the lowest mean simulated FAF because it routed many bottom-quintile students to infeasible pathways (51.2% infeasibility rate for the bottom quintile).
  • Parity Limitations: Demographic-parity constraints equalized access to high-opportunity recommendations but failed to improve outcomes; bottom-quintile infeasibility actually rose to 53.6%.
  • Feasibility Information: Constraint-aware AI and Human-in-the-loop review achieved the highest FAF and lowest infeasibility rates. However, feasibility-informed policies also "steered" constrained students away from high-opportunity sectors almost entirely (e.g., only 3.1% access for bottom quintile under constraint-aware AI), suggesting a risk of "aspirational conservatism" or system-level circumscription.

• Study 2 (Experiment):

  • Perceived FAF: The AI + Counsellor review condition produced the highest perceived FAF ($d = 0.52$ vs. AI-only) and the smallest socioeconomic disparity. Bottom-quintile students in the reviewed arm reported the highest perceived FAF of any bottom-quintile group ($M = 4.77$ versus $3.97$ under AI-only).
  • Agency and Deference: AI-only guidance significantly depressed reported agency and uncertainty awareness while elevating algorithmic deference ($d = 1.20$ vs. counsellor-only). Counsellor review fully restored reported agency.
  • Explanation: Explainable AI improved uncertainty awareness and reduced deference compared to AI-only, but did not match the FAF gains of the reviewed condition.
  • Equity: The socioeconomic gradient in perceived FAF was steepest under AI-only guidance and flattest under AI + Counsellor review. Human oversight did not redistribute benefits away from advantaged students but extended benefits to disadvantaged students that the unreviewed system denied them.

Key Contributions

1. Redefining Evaluation Metrics: The paper argues that career guidance AI must be evaluated on feasibility-adjusted fit, equity, calibrated uncertainty, and preserved agency, rather than predictive accuracy alone.
2. The Role of Conversion Factors: It demonstrates that the primary failure of accuracy-optimised guidance is the omission of "conversion factors" (resources and constraints) from the feature space. The value of human oversight lies not in superior statistical skill, but in possessing and applying this contextual knowledge.
3. The "Feasibility-Aspirational" Trade-off: The study identifies a critical tension: feasibility-aware designs reduce infeasible advice but risk institutionalizing conservative expectations, potentially narrowing the aspirations of the most constrained students unless paired with capability-expanding support (e.g., scholarships, mobility aid).
4. Human-in-the-Loop Efficacy: It provides empirical evidence that counsellor review of AI recommendations improves perceived fit, reduces socioeconomic disparities, and restores student agency, acting as a mechanism to redistribute the perceived benefits of AI.

Significance and Claims
The paper claims that AI-assisted career guidance should be viewed as infrastructure for a just school-to-work transition. The author argues that:

• Accuracy is insufficient: Systems optimized solely for predictive fit can systematically disadvantage constrained students by recommending infeasible pathways.
• Oversight is redistributive: Human review does not merely correct errors; it actively expands the "real freedom" (in the Capability Approach sense) of disadvantaged students by making high-opportunity pathways feel attainable and by preventing the premature foreclosure of aspirations.
• Design Implications: To avoid the risk of "aspirational conservatism," feasibility-aware guidance must be coupled with capability-expanding responses. The study suggests that counsellor review can multiply scarce counselling capacity (by reviewing AI shortlists rather than handling full caseloads) while preserving the accountability and contextual knowledge necessary for equitable outcomes.

The author concludes that while the findings are specific to the Indian context (a site of acute counsellor scarcity and rapid AI adoption), the mechanisms—constraint-blind recommendation, SES-graded feasibility, and algorithmic deference—are generalizable to other economies with stratified school-to-work transitions.