Exploring Functional Shifts in Pos Tagging Across Various NLP Libraries: A Study of NLTK, spaCy and Textblob

Technical Summary: Exploring Functional Shifts in POS Tagging Across NLTK, spaCy, and TextBlob

Problem Statement
Part-of-Speech (POS) tagging is a foundational task in Natural Language Processing (NLP) essential for syntactic analysis and semantic understanding. However, current POS taggers face significant challenges when processing "functional shifts," where words transition between grammatical roles based on context. This difficulty is most pronounced in "garden path" sentences—syntactically complex structures that initially mislead the reader (and the parser) regarding a word's function. While existing literature (e.g., Li et al., 2022; Kumawat & Jain, 2015) acknowledges high token-level accuracy in modern taggers, there is a noted disparity in sentence-level accuracy, particularly in tasks like dependency parsing where the correct identification of syntactic relationships is critical. The specific problem addressed is the inability of prominent NLP libraries to consistently and accurately tag words undergoing functional shifts in unconventional contexts.

Methodology
This study employs a comparative empirical analysis of three widely used NLP libraries: NLTK, spaCy, and TextBlob.

• Dataset: The research utilized a corpus of 56 sentences categorized into three syntactic groups:
  1. Garden path sentences (21 items).
  2. Nominalized forms (11 items).
  3. Sentences involving shifts in positioning (24 items).
  • Note: The dataset was sourced from https://wordsmithingtools.com/list-of-garden-path-sentences.
• Procedure: Each sentence was processed using Python implementations of NLTK, spaCy, and TextBlob. The study focused on the accuracy of POS tagging for specific words exhibiting functional shifts (e.g., words acting as verbs in one context and nouns in another, such as "man," "round," "table," and "that").
• Evaluation: The output tags from each library were compared against the expected syntactic structures to determine the number of correctly and incorrectly tagged sentences within each category.

Key Results
The analysis revealed distinct performance disparities among the three libraries, particularly in handling syntactic complexities:

• Overall Accuracy: spaCy demonstrated superior performance, correctly tagging 32 out of 56 sentences (57.1%). In contrast, both NLTK and TextBlob achieved identical lower scores, correctly tagging only 21 out of 56 sentences (37.5%).
• Category-Specific Performance:
  • Garden Path Sentences: All libraries struggled, but spaCy performed best with 5 correct tags, compared to 4 each for NLTK and TextBlob.
  • Nominalized Forms: spaCy correctly identified 5 out of 11, while NLTK and TextBlob identified only 3 each.
  • Positioning Shifts: spaCy showed significant strength here, correctly tagging 22 out of 24 sentences, whereas NLTK and TextBlob managed only 14 each.
• Specific Word Analysis: The study highlighted specific instances where spaCy outperformed the others in disambiguating words with multiple functions:
  • "Round": spaCy correctly identified "round" as an adverb, adjective, preposition, and verb depending on context. NLTK and TextBlob frequently misclassified these instances, often defaulting to noun or verb tags.
  • "That": spaCy accurately distinguished "that" as a relative pronoun, subordinating conjunction, demonstrative pronoun, and demonstrative adjective. NLTK and TextBlob showed limitations in recognizing these deep contextual usages, often mislabeling subordinating conjunctions as determiners (DT).
  • Example Case: In the sentence "The Complex Houses Married and Single Students," spaCy correctly identified "Houses" as a noun and "Married" as a verb, aligning with the intended syntactic structure, whereas other libraries displayed inconsistencies in these ambiguous roles.

Key Contributions

• Empirical Comparison: The study provides a direct, quantitative comparison of NLTK, spaCy, and TextBlob specifically regarding functional shifts and garden path sentences, a niche often overlooked in broader POS tagging accuracy studies.
• Identification of Limitations: It explicitly documents the specific failure modes of rule-based and hybrid approaches (NLTK, TextBlob) when faced with high syntactic ambiguity, contrasting them with the deep learning-based statistical models of spaCy.
• Contextual Sensitivity: The research demonstrates that while token-level accuracy may be high, sentence-level accuracy varies significantly based on the library's ability to handle context-dependent functional shifts.

Significance and Implications
The paper posits that accurate syntactic analysis is paramount for the robustness of NLP applications. The findings suggest that:

• Library Selection: For tasks requiring nuanced syntactic analysis of complex or ambiguous sentences, spaCy is a more robust tool than NLTK or TextBlob.
• Algorithmic Refinement: The discrepancies highlight the need for continued algorithmic improvement to better handle syntactically complex sentences, moving beyond simple token accuracy to sentence-level coherence.
• Educational Value: The study serves as a resource for educators and researchers in computational linguistics, offering a basis for curriculum development and practical exercises to understand the challenges of real-world syntactic analysis.
• Future Development: The authors suggest that these insights could assist in improving libraries trained on African languages (such as AfriBerta, KINYaBert, and Serengeti) by leveraging the advanced functional shift handling capabilities observed in spaCy.

The study concludes that while current NLP libraries have made progress, significant hurdles remain in processing functional shifts, necessitating further research to enhance the adaptability and effectiveness of language processing models across diverse linguistic structures.