Large Language Models for Assisting American College Applications

1. Problem Statement

The American college application process is characterized by high stakes, fragmented policies, and significant administrative burden. Students face several critical challenges:

• Heterogeneous Structures: Application portals (e.g., Common App, Coalition, university-specific) use different field names, data types, and organizational structures for identical concepts.
• Conditional Logic: Forms contain dynamic fields that appear or disappear based on user inputs (e.g., visa status, dual enrollment), creating complex state spaces.
• Ambiguity: Questions often lack context or require cross-referencing external documents (transcripts, FAQs) to interpret correctly.
• High Stakes & Low Error Tolerance: Errors can lead to rejection. Furthermore, "hallucinations" from standard Large Language Models (LLMs) pose severe risks, as fabricated information can permanently damage an application.
• Academic Integrity: There is a need to assist with factual data entry without violating policies regarding AI-generated essays or misrepresenting a student's voice.

2. Methodology: EZCollegeApp System

EZCollegeApp is a human-in-the-loop system designed to assist students by structuring forms, grounding answers in authoritative sources, and maintaining full human control. It does not auto-submit forms or generate creative content.

A. Core Design Principles

1. Grounded Assistance: All suggestions are derived from explicit, verifiable sources (uploaded student docs or official admissions websites), not parametric LLM knowledge.
2. Human-in-the-Loop: The system suggests content but never auto-fills or submits. Users must manually copy/paste or edit suggestions.
3. Structured Reasoning (Mapping-First): The system decouples form understanding from answer generation. It maps external form fields to a Canonical Form Schema before generating answers.
4. Extraction Over Generation: The system extracts and reorganizes factual data (e.g., GPA, activities) but refuses to generate creative essays, preserving the student's authentic voice.
5. Transparency: Every suggestion includes inline citations linking to the specific source document and page range.

B. System Architecture

The system consists of three main components:

1. Conversational Front-End (Next.js/TypeScript): Guides users to upload documents (transcripts, CVs, essays) and specify intent.
2. Back-End Processing Pipeline (Go/Python):
   • Ingestion: Uses Playwright to crawl official admissions websites (BFS with keyword prioritization) and stores content in dual formats (HTML for text, PDF for archival).
   • Extraction: Uses pdfplumber and OCR (pytesseract) for documents, and Vision-Language Models (VLMs) for images.
   • Indexing: Implements Semantic Chunking (300-500 tokens) and Structure-Aware Retrieval. It builds a hierarchical tree representation of documents (e.g., separating "Education" from "Experience" in a resume) to allow LLM-guided navigation without relying solely on vector embeddings.
   • Knowledge Bases: Maintains a separate Institutional KB (policies, deadlines) and a Personal KB (student data).
3. Application Copilot (Browser Extension):
   • Activates on supported portals to analyze the DOM.
   • Maps form fields to the internal Canonical Schema using a Three-Tier Matching Engine:
     • Tier 1: Direct keyword matching.
     • Tier 2: Contextual/structural analysis (parent containers, ARIA labels).
     • Tier 3: Semantic similarity scoring.
   • Presents suggestions via non-intrusive overlays, requiring explicit user action to transfer data.

C. Retrieval-Augmented Generation (RAG) & Agentic Workflow

• Mapping-First Generation: The system first resolves a form question to a canonical field (e.g., user.activities.list), then constructs a structured query.
• Agentic Retrieval: The LLM acts as an agent, deciding when to invoke retrieval tools (search_knowledge_base, list_documents) rather than retrieving for every query.
• Answer Synthesis: The LLM generates answers strictly based on retrieved chunks, formatting them to meet field constraints (e.g., date formats, character limits) and appending inline citations.
• Memory Subsystem: A persistent, user-governed memory layer stores preferences and task states across sessions, distinct from the factual knowledge base to prevent hallucination.

3. Key Contributions

1. Mapping-First Paradigm: A novel approach that separates semantic intent from surface-level wording, enabling consistent reasoning across heterogeneous application portals.
2. Source-Separated RAG: A practical implementation of retrieval-augmented generation for high-stakes domains that strictly separates institutional policy data from personal student data, ensuring provenance and preventing data leakage.
3. Human-Centric Design: A rigorous framework for AI assistance that prioritizes transparency (citations), control (no auto-submission), and integrity (extraction over generation), serving as a blueprint for regulated AI deployment.
4. Comprehensive Evaluation Framework: The use of synthetic student data (generated via multimodal LLMs to mimic real transcripts and photos) allows for scalable, privacy-preserving testing of the entire pipeline.

4. Evaluation Results

The system was evaluated using automated tests and human quality assessments on synthetic student profiles.

• Automated Testing:
  • Fill Rate: Achieved 80–85% fill rate for general application questions and 78% for school-specific forms.
  • Citation Validity: 92% of citations were semantically relevant to the answer (cosine similarity > 0.7).
  • Conditional Logic: Successfully handled dynamic field visibility and dependencies.
• Human Quality Assessment (10 evaluators: 5 students, 5 counselors):
  • Usefulness: 82% rated answers as "useful" or "very useful."
  • Clarity: 88% rated answers as "clear."
  • Grounding: 90% rated answers as "well-grounded" in factual data.
  • Trust: 89% reported that the citation mechanism increased their trust in the system.
  • Efficiency: Reviewing and correcting a suggested answer took 15–20 seconds, compared to 2–3 minutes for writing from scratch.

5. Significance and Future Work

Significance:
EZCollegeApp demonstrates that LLMs can be safely deployed in high-stakes, compliance-heavy workflows by prioritizing structured reasoning and provenance over raw generative capability. It offers a reusable blueprint for other domains like finance (loan origination) and healthcare (clinical intake) where errors have severe consequences. The system proves that "augmentation" (assisting human judgment) is a more viable and ethical path than "automation" (replacing human judgment) in sensitive contexts.

Limitations & Future Directions:

• Document Quality: Performance degrades with poor scans or non-standard layouts; future work aims for better interactive document annotation.
• Policy Drift: Manual re-crawling is labor-intensive; future systems will implement automated change detection for admissions policies.
• Multilingual Support: Currently optimized for English; expansion to international systems (e.g., UCAS) and low-resource languages is planned.
• Multimodality: Future iterations will index non-textual data (images, videos) more deeply.
• Scalability: Exploring cost-optimization via smaller models or open-source local deployment.

In conclusion, EZCollegeApp provides a robust, transparent, and ethically aligned framework for using AI to reduce the administrative burden of college applications while preserving the integrity of the admissions process.