Trait, Not State: The Durability of Reading Identity in Social Highlighting

Technical Summary: Trait, Not State: The Durability of Reading Identity in Social Highlighting

Problem Statement
Prior research on social web highlighting established that individuality in reading behavior resides primarily in selection (which documents a user chooses to highlight) rather than salience (where within a document they highlight). However, these findings were cross-sectional, treating time as fixed. The central question addressed by this paper is temporal: Is a reader's selection signature a trait (a durable, stable identity) or a state (a transient condition driven by recency)?

This distinction has opposing practical implications. If selection is a state, recommendation systems must rely on short-term session models and aggressive forgetting. If it is a trait, long-term user profiles are a legitimate, persistent asset. While production recommenders often default to recency weighting, psychometric literature suggests digital traces can predict stable attributes. This study aims to resolve this tension by measuring the decay of a reader's selection advantage over time.

Methodology
The study utilizes data from Glasp, a social web highlighter, focusing on a strict cohort of heavy, long-tenured users.

• Cohort: From 191,223 randomly sampled user records, 405 qualifying readers were identified. Qualification required $\ge$60 clean web documents spanning $\ge$12 months with activity in $\ge$8 distinct months. The median user had 133 documents.
• Profile Construction: For each user, the first six calendar months of highlighting were "frozen" to create a profile. This profile consists of the centroid of text embeddings (512-dim text-embedding-3-small) of their highlight spans, capped at $K=20$ documents.
• Evaluation Design:
  • Positives: The user's subsequent document selections, binned by the time gap since the profile freeze (0–1, 1–3, 3–6, 6–12, 12–24, 24+ months).
  • Negatives: To prevent "supply drift" (changes in available content) from masquerading as personal drift, negatives were time-matched. They were drawn from other readers' selections in the exact same calendar intervals as the positives.
  • Two Regimes:
    1. Coarse (Easy): Negatives and controls drawn from the global cohort.
    2. Fine (Hard/Primary): Negatives and controls drawn from the user's top-25 most profile-similar peers (interest neighborhood). This regime was designated primary to test identity against a harder, more relevant baseline.
• Metric: The primary metric is the Average Precision (AP) advantage ($Own - Other$), where "Own" is the AP of the user's frozen profile ranking the candidate set, and "Other" is the mean AP of three control users' profiles on the same set.
• Retention Analysis: To avoid survivorship bias (where far bins only contain long-tenured users), the primary analysis uses a paired retention approach: comparing a specific user's advantage at a later bin (e.g., 6–12 months) against their own advantage at the 0–1 month baseline.

Key Results

1. Validation of the Harness: The "anchor" cell (evaluation immediately after the profile freeze) in the fine layer yielded an advantage of +0.188 (95% CI: [0.160, 0.216]). This reproduces the prior cross-sectional estimate of +0.169, validating the experimental setup.
2. No Detectable Decay (Trait Behavior):
   • Fine Layer: The paired retention ratio ($R$) at the 6–12 month horizon is 1.00 (CI: [0.85, 1.18]), indicating no statistically detectable decline. Even at 24+ months, the interval includes zero, though the point estimate allows for a modest decline.
   • Coarse Layer: A statistically significant decline was observed only in the coarse layer at 12–24 months (approx. 13% drop), suggesting that broad topic mixes may drift slightly while fine-grained identity remains stable.
   • Conclusion: No "half-life" bin was observed within two years in the fine layer.
3. Independence from Source Repetition: The durable signal is not merely a result of users repeatedly highlighting the same domains. When all profile domains were excluded from the candidate set, the advantage retained ~90% of its value across all horizons.
4. Slow Within-Person Drift: While the identity is stable relative to others, it is not static. A profile built from the recent half of a user's history outperformed a profile from the old half by +0.042 AP. However, the old half retained ~91% of the recent half's score, indicating slow accretion of new themes rather than replacement.
5. Prospective Actionability: Personal profiles (even those built from documents 20 months prior) ranked a user's actual next reads at roughly 3x the AP of non-personal priors (lifetime popularity, neighborhood co-reading, recency, random). Notably, global popularity priors performed at the level of random guessing in this individual-level evaluation.

Significance and Claims
The paper claims to provide the first identity-decay curve for naturalistic, engagement-verified reading selection. Its operational definition of "trait" is a stable, person-specific selection signature under continued platform engagement.

• Theoretical Contribution: The study demonstrates that while within-document salience signals are shared and situational, the selection layer is personal and persistent. This challenges the assumption in recommender systems that user preferences are purely state-based and require constant refreshing.
• Practical Implication: For systems utilizing this data, "aggressive forgetting" is unwarranted for selection signals. Long-term profiles should be retained and combined with modest recency adaptation. The signal remains actionable even for profiles built from a user's earliest documents (median 20 months prior).
• Limitations and Scope: The authors explicitly note that the findings apply to heavy, long-tenured users of a specific platform (Glasp). The study cannot separate exposure from choice (as impression logs are unavailable), meaning the "trait" is a property of the "person-in-environment" system. The durability observed is conditional on continued engagement; the study does not claim that engagement itself is permanent. Furthermore, the measurement instrument (frozen title/span embeddings) is deliberately simple; a more complex model might reveal different drift patterns.

In summary, the paper concludes that for engaged readers, the signature of what they choose to read is a durable trait that persists for years, transfers to unseen sources, and significantly outperforms non-personal baselines, whereas the signal of where they highlight within a text is transient and shared.