French parsing enhanced with a word clustering method based on a syntactic lexicon

Technical Summary: French Parsing Enhanced with a Word Clustering Method Based on a Syntactic Lexicon

Problem Statement
The paper addresses the issue of data sparseness in Probabilistic Context-Free Grammar (PCFG) parsers, particularly for French. While syntactic lexicons (such as the Lexicon-Grammar) contain rich subcategorization information useful for parsing, directly incorporating this data into PCFGs often exacerbates data sparseness. This occurs because augmenting Part-of-Speech (POS) tags with syntactic functions or verb valence features increases the number of unique rules, making it difficult to estimate probabilities for rare rules, especially when training on smaller treebanks like the French Treebank (FTB). Previous attempts to integrate syntactic data without addressing sparseness resulted in significant performance decreases.

Methodology
The authors propose a word clustering method that leverages the Lexicon-Grammar (LG), a manual syntactic lexicon developed by Maurice Gross. Unlike previous clustering approaches that rely on unsupervised statistical algorithms (e.g., Brown clustering) or morphological simplification (desinflection), this method uses the LG's structured tables to group verbs.

The core methodology involves:

1. Lexicon-Grammar Integration: The LG organizes lexical items into tables based on shared syntactic properties (e.g., subcategorization frames). Each verb lemma is associated with one or more table identifiers.
2. Clustering Strategies:
   • TableClust: Verbs are replaced by the combination of their POS tag and their specific LG table identifiers (e.g., #tag 12).
   • LexClust: To reduce the number of clusters and handle ambiguity, the authors utilize a manually constructed hierarchy of LG tables. Verbs are replaced by the combination of their POS tag and the class identifiers at specific levels of this hierarchy (Levels 1–3). This creates coarser clusters as the hierarchy level increases.
3. Parser Configuration: The experiments utilize the Berkeley Parser (BKY), a non-lexicalized PCFG parser enhanced with latent annotations (Matsuzaki et al., 2005). The parser is trained on the French Treebank-UC (FTB-UC), a corrected variant of the FTB.
4. Preprocessing: A POS tagger (MElt) and a lemmatizer (Dela dictionary) are used to assign tags and lemmas to verbs, allowing the system to map words to their corresponding LG table IDs or hierarchy classes.

Key Contributions

• Novel Clustering Source: The paper demonstrates that a syntactic lexicon not originally designed for statistical parsing (the Lexicon-Grammar) can be effectively repurposed to generate word clusters that improve parser performance.
• Hierarchical Abstraction: The introduction of a hierarchical classification of LG tables allows for the creation of verb clusters that balance syntactic specificity with data density, effectively mitigating the sparseness problem.
• Empirical Validation: The study provides a comparative analysis of this LG-based clustering against state-of-the-art French clustering methods, including morphological desinflection (DFL) and unsupervised Brown clustering (Clust).

Experimental Results
The experiments were conducted using 10-fold cross-validation on the FTB-UC, evaluating performance via labeled F-Measure (F1), Unlabeled Attachment Score (UAS), and Labeled Attachment Score (LAS).

• Baseline: The baseline Berkeley Parser achieved an F1 of 83.82.
• TableClust: Replacing verbs with specific table IDs yielded a slight improvement (F1: 84.11), comparable to using only POS tags.
• LexClust (Hierarchical): Using the hierarchy of verb tables significantly improved results. The best performance was achieved at Level 2 of the hierarchy, reaching an F1 of 84.44. This method reduced the lexicon size from 27,143 to 21,833 words.
• Comparison: The LexClust Level 2 results were nearly identical to the DFL (desinflection) method (F1: 84.57) but were outperformed by the combination of DFL and unsupervised clustering (DFL+Clust), which achieved an F1 of 85.22.
• Error Analysis: The authors noted that the performance of their method is heavily dependent on the quality of the POS tagger (MElt). When using "Gold" tags (perfect tagging), the F1 score gap between the baseline and the proposed method was approximately 0.5%, suggesting that tagging errors currently limit the full potential of the clustering method.
• Phrase-Level Gains: The method showed the most significant absolute gains in recognizing verbal phrases, specifically participial phrases (VPpart, +4.4% gain) and relative clauses (Srel, +1.6% gain).

Significance and Claims
The paper claims that integrating data from a syntactic lexicon like the Lexicon-Grammar into a probabilistic parser is a viable strategy for improving parsing performance, provided that the data is abstracted through clustering to manage sparseness. The authors assert that their method, LexClust, is relevant and useful for PCFG parsers like BKY, as it improves performance over the baseline and approaches the results of more complex morphological methods while relying solely on verbal information.

The authors remain modest regarding their findings, acknowledging that their method does not yet surpass the best existing combination (DFL+Clust) and that the integration of their lexical clustering with unsupervised clustering methods remains an open challenge. They conclude by stating their intention to reproduce these experiments with other grammatical categories in future work.