📄 chemistry

History-Aware Text Representations for Polymer Property Prediction from Literature-Derived Records

This paper introduces Holistic History-aware Text (HHT), a framework that leverages natural language descriptions of polymer composition, synthesis, and processing via large language models to outperform traditional structure-only encoders in predicting macroscopic properties from heterogeneous literature-derived records.

Original authors: Pingwei Liu, Hetao Huang, Haifan Zhou, Cheng Zeng, Tayirjan Isimjan, Hanyu Gao, Bangban Zhu, Xingfen Huang, Wen-Jun Wang, Bogeng Li

Published 2026-07-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Pingwei Liu, Hetao Huang, Haifan Zhou, Cheng Zeng, Tayirjan Isimjan, Hanyu Gao, Bangban Zhu, Xingfen Huang, Wen-Jun Wang, Bogeng Li

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to guess the flavor of a cake just by looking at a list of its ingredients. If you only see "flour, sugar, and eggs," you might guess it's a vanilla sponge. But what if the baker used a secret spice, baked it at a scorching temperature for an extra hour, or folded in a secret layer of chocolate? The final cake would taste completely different, even though the main ingredients are the same. This is the heart of materials science, specifically the study of polymers—giant molecules that make up everything from plastic water bottles to high-tech medical implants. For a long time, scientists have tried to predict how these materials will behave (how strong, stretchy, or heat-resistant they are) by looking only at their chemical "recipe." But just like the cake, the final result depends heavily on how it was made and how it was cooked. If you ignore the cooking process, your predictions will often miss the mark.

This is the puzzle a team of researchers from Zhejiang University, the Hong Kong University of Science and Technology, and other institutions decided to solve. They realized that the old way of looking at polymers was like judging a movie only by its script, ignoring the director, the lighting, and the actors' performances. They wanted a way to feed a computer not just the chemical recipe, but the entire "story" of how the material was created. To do this, they built a new tool called HHT (Holistic History-aware Text). Instead of forcing complex experimental notes into rigid chemical codes, they let the computer read the actual text descriptions of the experiments, including the temperature, the mixing speed, and the specific equipment used. By using a powerful "brain" (a large language model) that was already trained on millions of scientific documents, they taught it to understand these stories and predict the material's properties.

The results were quite a surprise. The team gathered nearly 10,000 records from over 1,000 scientific papers, covering six different properties like how stiff a plastic is (Young's modulus) or at what temperature it melts. When they tested their new "story-reading" system against the old "recipe-only" systems, HHT won almost every time. It was especially good at predicting properties that depend heavily on how the material is processed, like strength and stiffness. For example, when predicting the stiffness of a material, the old systems made errors that were three times larger than HHT's errors.

However, the researchers were careful not to overhype the victory. They noticed something tricky: data from the same scientific paper often looked very similar to each other because the same lab used the same methods. If a computer just memorized "Paper A says the answer is X," it would look smart but wouldn't actually understand the science. To test for this, they created a "cheat sheet" baseline that simply guessed the average value from the same paper. HHT beat this cheat sheet for five out of the six properties, proving it was learning real patterns, not just memorizing papers. For one property, ultimate tensile strength (how much force it takes to snap a material), HHT was about as good as the cheat sheet, suggesting that for this specific trait, the "story" of the experiment matters just as much as the chemical recipe itself.

The team also found that their system was a master at learning with very little data. While the old systems needed huge amounts of examples to get better, HHT started outperforming them even when it only saw 20% of the data. This suggests that the "brain" they used already knew a lot about how chemistry and processing work together. They even showed that they could take this general "brain" and quickly teach it a new, specific job—like predicting the properties of a specific type of epoxy resin—just by showing it a few new examples.

In short, this paper suggests that to truly understand and predict how polymers will behave, we need to stop treating them as static chemical formulas and start treating them as dynamic stories with a history. By letting computers read the full experimental narrative, we can make much better predictions about the materials that will build our future, from stronger bridges to smarter electronics. The researchers admit that some challenges remain, especially with very complex materials like those used in electronics, but their new approach offers a promising, practical way to learn from the vast library of scientific knowledge we already have.

Technical Summary: History-Aware Text Representations for Polymer Property Prediction

Problem Statement
Macroscopic polymer properties are determined not only by repeat-unit chemistry but also by synthesis routes, processing conditions, and formulation details. However, existing machine learning pipelines for polymer property prediction predominantly rely on "structure-only" encoders (e.g., PolyBERT, PerioGT) that reduce materials to canonical repeat-unit representations (SMILES or molecular graphs). This approach discards critical experimental history, causing distinct polymer variants with identical monomer chemistry but different processing or synthesis histories to collapse into identical model inputs. Furthermore, structure-only encoders struggle with complex systems such as blends, composites, and commercial resins where a canonical repeat-unit SMILES is unavailable or insufficient. This representational gap limits the ability to learn realistic structure–processing–property relationships from heterogeneous literature-derived records.

Methodology: The HHT Framework
The authors introduce Holistic History-aware Text (HHT), a framework that treats polymer records as natural-language descriptions rather than compressed molecular structures.

Data Curation: The team curated 9,415 records from over 1,000 scientific papers covering six properties: glass transition temperature ( $T_g$ ), melting temperature ( $T_m$ ), Young's modulus ( $E$ ), ultimate tensile strength (UTS), dielectric constant ( $\varepsilon$ ), and refractive index ( $n$ ). The extraction pipeline converts full-text PDFs to Markdown, uses a two-stage Large Language Model (LLM) process for draft generation and review, and applies schema validation and unit normalization.
Representation: Each record is converted into "history-aware text" that explicitly includes monomer identity, synthesis routes, processing conditions, formulation, additives, and measurement context.
Model Architecture: HHT employs a frozen, pretrained LLM (specifically ChemDFM-v1.5-8B in the main experiments) as a feature extractor. The final-layer hidden states of the text input are cached on disk. A lightweight regression head, consisting of a learned sigmoid-gated pooling layer followed by a two-layer multilayer perceptron (MLP), is trained on these cached representations to predict properties. This design avoids repeated backbone inference during hyperparameter tuning.
Evaluation Protocol: Recognizing that literature-mined datasets exhibit strong article-level clustering (samples from the same paper are more similar to each other than to samples from other papers), the authors introduce an article-level baseline. This null model predicts test samples using the mean/median of training samples from the same source article. This baseline serves as a contextual reference to prevent inflated performance estimates caused by random train-test splits that leak article-level information.

Key Results

Performance vs. Structure-Only Encoders: HHT outperformed structure-only encoders (PolyBERT and PerioGT) across all six properties. The improvement was most pronounced for properties sensitive to processing and hierarchical structure, such as Young's modulus ( $E$ ) and ultimate tensile strength (UTS). For $E$ , HHT reduced the Root Mean Squared Error (RMSE) to 12.3, compared to 38.1 for PolyBERT and 39.0 for PerioGT (a roughly threefold reduction).
Performance vs. Article-Level Baseline: HHT surpassed the article-level baseline for five of the six properties ( $T_g$ , $T_m$ , $E$ , $\varepsilon$ , $n$ ), indicating that the learned text representations capture transferable information beyond simple publication-specific clustering. For UTS, HHT performed nearly indistinguishably from the article-level baseline, suggesting this property remains heavily influenced by article-level structure under the current data regime, though HHT still significantly outperformed structure-only encoders.
Data Efficiency: HHT demonstrated superior data efficiency. With only 20% of the training data, HHT outperformed structure-only encoders trained on the full dataset for five of the six properties.
Ablation Studies:
- Backbone: Domain-specialized chemistry LLMs (ChemDFM) generally provided better representations than general-purpose models, though model size alone did not correlate with performance.
- Input Strategy: The best-performing configuration used a dynamic training strategy mixing structured records with descriptive reformulations, evaluated on clean structured text. Hierarchical-structure descriptors (e.g., crystallinity) improved performance when available but were not required for the practical configuration.
- Continual Training: A domain-specific continual training strategy, where the regression head is fine-tuned on specific domains (e.g., epoxy composites or phase change materials) while keeping the LLM backbone frozen, yielded significant gains. For epoxy $E$ , adapted HHT achieved an $R^2$ of 0.976, vastly outperforming structure-only baselines ( $R^2 \approx 0.03-0.07$ ).

Significance and Claims
The paper claims that HHT provides a practical framework for extending structure–processing–property learning to heterogeneous literature-derived records that cannot be captured by repeat-unit chemistry alone. By retaining synthesis and processing history within the text representation, the framework allows for the inclusion of complex formulations and polymer variants that are typically discarded in structure-only pipelines.

The authors emphasize that the primary value of HHT is not merely replacing molecular encoders with text, but expanding the representational scope to include the experimental context from which properties arise. They argue that this approach offers a resource-efficient route for learning from literature-scale data, particularly for properties heavily dependent on processing and formulation. The work also highlights the necessity of article-aware evaluation metrics to accurately assess generalization in literature-mined datasets, cautioning that high performance on random splits may simply reflect article-level clustering rather than true predictive capability. Finally, the authors suggest that HHT-like models can serve as property evaluators in inverse-design workflows, screening candidate compositions and processing histories before experimental validation.

More like this