Pseudoperplexity Probes Memorization in Protein Language Models
This study employs pseudoperplexity to demonstrate that the protein language model ProtT5 exhibits detectable but limited memorization of its training data, suggesting it primarily generalizes the statistical grammar of proteins rather than simply rote-learning sequences.