Personal Salience: Highlighting Is Social, but Individuality Lives in Selection

Technical Summary: Personal Salience in Digital Highlighting

Problem Statement
The paper investigates the extent to which an individual's unique identity can be recovered from naturalistic highlighting traces (marking passages while reading). While digital twin research often assumes behavioral traces reveal distinct interests and predict choices, this work questions whether highlighting is a "personal fingerprint" or primarily a reflection of shared salience. The core challenge is disentangling three conflated quantities in personalization research:

1. Generic Salience: Structural or semantic importance inherent to the document (e.g., topic sentences, conclusions).
2. Crowd Salience: Passages marked by other readers (collective consensus).
3. Personal Salience: The individual residual unique to a specific user.

The authors leverage Glasp, a web highlighter, to materialize "co-readership"—a rare scenario where multiple users highlight the exact same document. This allows for a within-document identity control, holding the document and topic constant to isolate individual behavior.

Methodology
The study defines two distinct prediction tasks and employs a rigorous, leakage-free evaluation framework:

1. Tasks:

   • Within-Document Salience: Predicting which sentences a user highlights from the entire document.
   • Selectivity: Predicting which of the already-highlighted passages (by the crowd) belong to a specific user.

2. Scorers & Baselines:

   • Crowd: The count of other co-readers who marked a candidate span (estimated from a dense crowd, median 38 readers).
   • Generic: Sentence centrality (cosine similarity to the document embedding centroid).
   • Personal Models:
     • Embedding Profile (M1): Cosine similarity to a mean-pooled embedding of the user's past highlights.
     • Twin: A proprietary frontier LLM (gpt-5.5) conditioned on the user's history to rank candidates.
   • Identity Control (Other): The same scoring method applied using a different co-reader's profile. This controls for the general skill of the extractor; a positive "own-versus-other" gap indicates genuine personalization.

3. Critical Methodological Controls:

   • Leakage-Free Profiles: Personal profiles are built per pair, explicitly excluding the target document. The authors demonstrate that naive pipelines often leak the target document into the profile (~42% of pairs), inflating personal scores by +0.07 to +0.15 Average Precision (AP).
   • Dense Crowd: Using the full set of co-readers (median 38) rather than small subsets (≤5), which were found to artificially understate crowd performance.
   • Topic Decomposition: Comparing users against "topically-nearest" peers to determine if the personal signal is driven by stable thematic preferences or finer-grained style.

Key Results

1. Highlighting is Social (Within-Document Salience):

   • The crowd is the dominant predictor of which sentences a user highlights (AP 0.321), outperforming generic structure (0.187), personal embedding profiles (0.172), and even frontier LLM twins (0.247).
   • The recoverable personal signal in this task is minimal. The "own-versus-other" identity gap is only +0.017 AP for the embedding scorer (small but significant) and +0.012 AP (not significant) for the twin.
   • Even a "lead/position" prior (predicting marks based on sentence order) outperforms personal history, though the crowd still beats the position prior.
   • Conclusion: Individuality is largely absent in determining what is salient; consensus rules.

2. Individuality Lives in Selection:

   • When the task shifts to selecting which of the already-salient passages a user chose, the personal signal becomes strong. The identity gap jumps to +0.143 AP.
   • Crowd popularity is anti-predictive here (AP 0.157, below the base rate) because the most-marked spans are shared by everyone, offering no individual differentiation.
   • Asymmetry: Under the same scorer, the individual signal in selection is 6–8× stronger than in salience.

3. Nature of the Selection Signal:

   • Decomposition reveals that the selection signal is largely stable thematic preference. When comparing a user to a "topically-matched" peer, the identity gap shrinks by ~8× (from +0.212 to +0.026).
   • A thin residual (+0.026) remains even against near-topic-twins, but the authors caution this may represent finer-grained topic nuance rather than non-thematic "style."
   • Scaling Law: For users with rich histories, the selection signal plateaus after roughly 60 past highlights (a few dozen), suggesting a modest amount of history is sufficient to capture the individual signature.

4. Secondary Observation (Consensus vs. Popularity):

   • There is a suggestive trend that crowd consensus declines as document popularity increases. With a fixed crowd size, documents with more total readers show lower agreement on what to highlight.

Significance and Claims
The paper claims to overturn a common premise in personalization: that individuals differ chiefly in what they find interesting (salience). Instead, the evidence suggests:

• Salience is Shared: What is interesting is largely determined by document structure and collective consensus.
• Individuality is Selective: Personal identity emerges in the selection of specific interesting items from a shared pool.
• Methodological Correction: Previous claims that "digital twins" beat the crowd were likely artifacts of methodological flaws: using artificially small crowds and failing to prevent data leakage from the target document into the user profile.

The authors conclude that highlights offer a "partial, selection-shaped window" onto a person, rather than a complete clone. The individual signature is real but exists as a "thin layer" over a strong shared foundation, surfacing primarily in selection tasks rather than salience detection.