Constituency Structure over Eojeol in Korean Treebanks

Technical Summary: Constituency Structure over Eojeol in Korean Treebanks

Problem Statement
The design of Korean constituency treebanks faces a central representational challenge regarding the choice of terminal units. Korean words are morphologically complex, combining lexical stems with functional morphemes (e.g., case particles, verbal endings). Existing treebanks differ in how they handle this complexity: some treat morphemes as independent constituency terminals, while others treat the eojeol (the orthographic spacing unit in Korean) as the terminal.

Treating morphemes as constituency terminals creates significant issues:

1. Interpretability: It conflates word-internal morphology with phrase-level syntactic structure, obscuring the distinction between morphological composition and syntactic grouping.
2. Interoperability: It creates mismatches with the dominant eojeol-based dependency resources (e.g., Universal Dependencies, Korean dependency treebanks). Converting between morpheme-based constituency trees and eojeol-based dependency trees requires heuristic reconstruction of word boundaries, complicating cross-resource comparison and evaluation.
3. Parsing: Morpheme-based terminals require gold-standard morphological segmentation as a prerequisite for phrase-structure analysis, hindering the development of end-to-end constituency parsers that operate directly on surface input.

Methodology and Proposed Framework
The paper proposes an eojeol-based constituency representation where the eojeol serves as the stable terminal unit for phrase structure, while morphological information is encoded in a separate, non-constituent layer.

• Terminal Definition: The terminal units of the constituency tree are overt eojeol (surface-form units).
• Preterminals: Each eojeol is assigned a Universal Part-of-Speech (UPOS) label, acting as a preterminal category that mediates between phrase-level structure and surface form. This abstracts away from language-specific morphological details while maintaining syntactic interpretability.
• Morphological Layer: Fine-grained morphological segmentation and language-specific Part-of-Speech (XPOS) tags are recorded in a parallel, non-constituent annotation layer. This layer captures internal word structure (e.g., 프랑스/NNP+의/JKG) without projecting morphemes as independent nodes in the constituency hierarchy.
• Normalization Strategy: The authors demonstrate that existing Korean treebanks (Sejong, Penn Korean, and KAIST) can be mapped to this shared eojeol-based backbone through explicit normalization steps:
  • Sejong: Debinarization of strict Chomsky Normal Form expansions.
  • Penn Korean: Removal of null elements (e.g., pro) and normalization of functional morphology.
  • KAIST: Collapsing separated functional morphemes back into their host eojeol terminals.

Key Contributions

1. Representational Stance: The paper argues that syntactically relevant bound morphology need not be represented as independent constituency terminals. Instead, eojeol terminals provide a stable syntactic backbone, with morphosyntactic information preserved separately.
2. Comparative Analysis: The authors provide a comparative analysis of the Sejong, Penn Korean, and KAIST treebanks, showing that despite their differing internal designs (binary branching, null elements, or exposed morphology), they can be normalized to a shared eojeol-based constituency domain.
3. Annotation Scheme: A concrete six-column annotation format is proposed (similar to CoNLL-U) that integrates:
   • Token ID
   • Surface form (eojeol)
   • Morphological segmentation (XPOS)
   • Universal POS (UPOS)
   • Left and Right constituency bracketing
     This format allows constituency parsing and morphological analysis to be performed independently while remaining aligned at the eojeol level.
4. Structural Validation: A small-scale structural comparison (Table 2) quantifies the impact of normalization. The results show that normalizing to eojeol terminals reduces terminal counts (collapsing morphemes/nulls) and nonterminal counts (removing binarization/abstract nodes), thereby aligning the structural complexity of different treebanks without altering the overt surface sequence.

Results and Findings

• Structural Alignment: Normalization successfully aligns the three major treebanks over a shared eojeol-based yield. While the original trees differ in depth and branching due to resource-specific conventions (e.g., Sejong's strict binarity, Penn's null elements, KAIST's exposed morphology), the normalized versions share a common terminal domain.
• Interoperability: The proposed scheme establishes a direct correspondence between constituency and dependency representations, as both can operate over the same eojeol sequence. This eliminates the need for heuristic reconstruction of word boundaries during cross-resource alignment.
• Parsing Implications: The proposal supports future end-to-end Korean constituency parsing by providing a surface-form terminal layer that does not require gold morpheme segmentation as a prerequisite.

Significance and Claims
The paper modestly claims that its proposal is a representational and methodological foundation rather than a comprehensive theory of Korean syntax.

• Clarification of Roles: It clarifies the division of labor between syntax (phrase structure over eojeol) and morphology (internal structure recorded separately).
• Resource Development: It offers a principled basis for future corpus development and cross-treebank comparison, ensuring that differences in analysis are not confounded by differences in terminal granularity.
• Compatibility: It enables Korean constituency annotation to be interoperable with Universal Dependencies and other eojeol-based resources without sacrificing access to rich morphological information.
• Limitations: The authors acknowledge that the normalization process is not lossless; it abstracts away resource-specific details such as binarization decisions, empty categories, and the explicit syntactic projection of functional morphology found in the original treebanks. However, they argue this abstraction is necessary to establish a stable, comparable backbone for syntactic analysis.