Lexical variables that affect integrated speaking test scores among EFL learners

Technical Summary: Lexical Variables in Integrated Speaking Tasks

Problem Statement
While vocabulary knowledge is a well-established indicator of second language (L2) competence, its contribution to speaking proficiency is complex and mediated by task type and source text modality. Previous research has largely focused on independent speaking tasks or integrated writing tasks, leaving a gap in understanding how lexical richness (sophistication, diversity, and fluency) functions specifically within integrated speaking tasks. Furthermore, it remains unclear whether the linguistic features signaling proficiency differ when learners summarize auditory input (listen-to-speak) versus written input (read-to-speak). This study investigates these gaps by examining how source text modality mediates the relationship between lexical variables and oral proficiency among Korean EFL learners.

Methodology
The study utilized the Korean English Learners' Spoken Corpus (KELSC), comprising 474 transcribed one-minute oral responses from university students. Participants completed two types of integrated tasks:

1. Listen-to-speak: Summarizing a school announcement (aural source).
2. Read-to-speak: Summarizing a passage on social media or welfare (written source).

Responses were evaluated against the Common European Framework of Reference for Languages (CEFR) scale (A1–C2). The analysis involved automated text processing using TAALES 2.0, TAASSC, and TAACO to extract multi-dimensional indices of lexical sophistication, diversity, fluency, syntactic complexity, and cohesion.

Statistical Approach
To identify predictors of CEFR levels, the study employed a rigorous statistical pipeline:

1. Variable Reduction: Initial screening reduced 22 candidate variables using correlation analysis and LASSO (Least Absolute Shrinkage and Selection Operator) regression to minimize multicollinearity.
2. Modeling:
   • Listen-to-speak: An ordinal logistic regression was applied, as the data met the proportional odds assumption.
   • Read-to-speak: A Partial Proportional Odds Model (PPOM) was utilized because the data violated the proportional odds assumption for specific variables (Word Count, MATTR, and Clauses per Sentence). This allowed for threshold-specific coefficients to capture non-linear developmental patterns.

Key Results

• Listen-to-Speak Task:
  Four lexical variables collectively and linearly predicted higher CEFR levels:

  • Lexical Fluency (Word Count): The strongest predictor; higher word counts were associated with nearly three times the odds of a higher proficiency rating.
  • Lexical Diversity (MATTR): Greater diversity significantly increased the odds of higher ratings.
  • Lexical Sophistication: Measured by COCA Spoken Frequency (negative association) and COCA Spoken Bigram MI2 (positive association). Advanced learners used lower-frequency spoken words and stronger bigram associations typical of spoken registers.
  • Note: Syntactic complexity and cohesion devices did not emerge as significant predictors in this modality.

• Read-to-Speak Task:
  The predictive landscape differed significantly, showing distinct patterns across proficiency levels:

  • Lexical Sophistication (Age of Acquisition - AoA): This was the only variable with a uniform positive effect across all CEFR thresholds. Learners using words acquired later in native development consistently scored higher.
  • Lexical Fluency (Word Count) & Syntactic Complexity (C/S): These variables only differentiated beginner-level learners (A1/A2). Their predictive power diminished and became negligible for intermediate and advanced learners, suggesting that written source texts act as a scaffold that equalizes output quantity and structural complexity across higher levels.
  • Lexical Diversity (MATTR): Exhibited an inverted-U pattern. Diversity increased from beginner to intermediate levels but declined among advanced (C2) learners. The study suggests advanced speakers prioritize precise, succinct vocabulary over varied repetition when summarizing written texts.

Key Contributions

1. Modality-Specific Trajectories: The study demonstrates that the developmental trajectory of lexical competence is not uniform; it shifts depending on whether the source text is aural or written.
2. The Scaffolding Effect: Written source texts appear to function as a linguistic scaffold, reducing the discriminative power of fluency and syntactic complexity for intermediate and advanced learners, while aural tasks rely heavily on fluency and diversity to distinguish proficiency.
3. Inverted-U Diversity: The paper provides empirical evidence that lexical diversity may not be a linear predictor of proficiency in integrated speaking; advanced learners may intentionally reduce diversity to achieve precision and succinctness when recycling written source material.

Significance and Claims
The author claims these findings offer critical empirical benchmarks for applied linguistics, specifically challenging the assumption that a single set of lexical metrics applies to all integrated speaking contexts. The study suggests that:

• Assessment Design: Scoring rubrics for integrated speaking should be task-specific. Listen-to-speak tasks should weight fluency and diversity heavily, while read-to-speak tasks should prioritize lexical sophistication (specifically AoA).
• Pedagogy: Instruction should be tailored to proficiency bands. For intermediate learners in read-to-speak contexts, the focus should shift toward lexical sophistication (using cognitively demanding words) rather than merely increasing word count or diversity.
• Limitations: The author modestly notes that the uneven distribution of participants across CEFR levels (clustering at B1/B2) may limit the predictive power at the extremes of the spectrum. Additionally, the study excludes acoustic/phonological features and the specific alignment between source text and learner output, which warrant future investigation.

The paper concludes that understanding these nuanced, task-dependent variations is essential for refining automated speech evaluation systems and optimizing integrated test designs.