🤖 machine learning

Toward Machine Learning with the Unit as a Primitive: Learning from Unit-Linked Events

This paper proposes formalizing machine learning by explicitly introducing the "unit" as a primitive semantic entity, defining supervised learning as the inference of a shared response law conditioned on a tokenizer-generated unit token to distinguish between homogeneous and heterogeneous world models.

Original authors: Heyang Gong

Published 2026-08-27

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Heyang Gong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of machine learning, computers are often taught by looking at rows of data, much like a student reviewing a stack of flashcards. Each card contains a question and an answer, and the computer's job is to find the pattern that links them. For decades, this method has worked remarkably well for many tasks, from recognizing faces to predicting the weather. However, this approach treats every single card as an isolated event, ignoring a fundamental truth about the real world: many of these cards actually belong to the same person, animal, or device. When a computer learns from a patient's medical history, a user's shopping habits, or a sensor's readings, it is often seeing multiple snapshots of the same persistent individual. Standard methods usually treat these snapshots as separate, unrelated entries, or they rely on hidden assumptions that the computer is simply guessing. This creates a blind spot. If the computer cannot distinguish between a pattern that belongs to a specific individual and a pattern that is common to everyone, it may fail when asked to predict what will happen to that same person in a new situation, or it might mistake a unique quirk of one person for a universal rule.

A new paper proposes a way to fix this by changing the very foundation of how these learning tasks are defined. Instead of starting with a list of records, the researchers suggest starting with the concept of the "unit"—the persistent individual to whom the data belongs. Imagine a library where, instead of just cataloging books, the librarian first identifies the specific reader who borrowed them. In this new framework, the computer is first told who the individual is, or at least given a clear rule for deciding when two pieces of data belong to the same person. This simple shift allows the computer to understand that while the events (the questions and answers) change, the underlying person remains the same. The researchers show that by making this "unit" an explicit part of the problem, machines can learn a shared set of rules that apply to everyone, while still respecting the unique differences of each individual.

The core of this discovery is a two-step process the computer uses to make sense of the world. First, it acts as a translator, taking the raw evidence about a person—such as their past behavior or physical characteristics—and converting it into a compact summary, which the author calls a "token." Think of this token as a unique ID card that the computer creates on the fly to represent that specific individual. Second, the computer uses a single, shared set of rules to read that ID card and predict what will happen next. This is a crucial distinction. In older methods, the computer might try to memorize a separate set of rules for every single person it encounters, which is impossible if it meets someone it has never seen before. In this new approach, the computer learns one universal rulebook. It simply looks at the ID card (the token) to see how that specific person fits into the general pattern. This allows the system to generalize: it can make accurate predictions for a new person it has never met, simply by creating a token for them based on the evidence it has, and then applying the same shared rules it learned from everyone else.

The researchers also explored what happens when the computer does not know exactly who the person is. Sometimes, the data is incomplete, or the link between two records is uncertain. In these cases, the computer cannot simply look up a name in a database. Instead, it uses the available evidence to form a best guess about the person's identity, creating a token that reflects this uncertainty. The paper proves that even with this guesswork, the system can still learn effectively, provided it separates the act of identifying the person from the act of predicting the outcome. The study demonstrates that if you only look at single, isolated records without knowing who they belong to, you cannot tell the difference between a world where everyone is exactly the same and a world where everyone is different but happens to look the same on average. However, if you can link multiple records to the same person with certainty, the computer can detect the hidden differences between individuals. This finding is vital because it sets a clear boundary for what is possible: a machine cannot learn about individual differences if it is only ever shown one snapshot of each person without any way to link them back to their source.

This work also clarifies how we should measure success. In many current systems, we judge a model by how well it predicts the average outcome across all data points. The author shows that this can be misleading. If some people appear in the data many times and others only once, a simple average might be dominated by the frequent users, ignoring the unique needs of the rare ones. The paper suggests that we need to evaluate models based on the individual, not just the record. This means checking if the model can predict what will happen to a specific person in a new situation, rather than just getting the overall statistics right. The researchers provide a checklist for how to design these tests properly, ensuring that we do not accidentally trick the computer into memorizing names instead of learning patterns.

Ultimately, this paper offers a more honest way to build intelligent systems. It acknowledges that the world is made of persistent individuals, not just a stream of disconnected events. By forcing the computer to declare who it is learning about before it starts guessing the answers, the system becomes more robust and more capable of handling the complexity of real life. It does not claim to have solved every problem in artificial intelligence, but it provides a solid, logical framework for understanding how machines can learn from people without losing sight of who those people are. The result is a method that is not just mathematically sound, but also conceptually clearer, bridging the gap between the messy reality of human data and the structured logic of machine learning.

Technical Summary: Toward Machine Learning with the Unit as a Primitive

Problem Statement

Standard supervised machine learning formalizations typically begin with a dataset of records $D = \{(x_i, y_i)\}_{i=1}^N$ and a mapping $f: \mathcal{X} \to \mathcal{Y}$ . In this notation, the index $i$ identifies a record, not the persistent individual (the "unit") to which that record refers. While this omission is harmless for row-level questions, it becomes consequential when events share an individual, when queries change while holding an individual fixed, or when attribution to a specific entity is uncertain.

Crucially, the standard formulation leaves unstated whether different individuals share a single response law or whether the observed row-level law is merely a marginal pooling of distinct, unit-conditioned relationships. Existing traditions—such as repeated-measures models, potential-outcome frameworks, and recommender systems—address these issues locally with specific assumptions, but lack a unified semantic primitive that elevates the persistent referent to the core of the learning problem.

Methodology: The Unit as a Primitive

The paper proposes elevating the unit ( $U$ ) to an explicit primitive at the level of task semantics. A learning task is defined by declaring:

A population of persistent referents ( $\mathcal{U}$ ).
A sameness criterion (when events concern the same referent).
The span over which this relation is retained.

The realized value $U=u$ denotes a selected referent. The semantic object of supervised learning is then a family of unit-conditioned response laws:
$u \mapsto \left[ x \mapsto P^*_u(dy | x) \right]$
where $P^*_u(dy | x) := P^*(Y \in dy | X=x, U=u)$ . Changing $x$ queries a location on the same response surface for a fixed unit; changing $u$ selects a different member of the family.

To make this family learnable from finite data, the paper introduces a computational factorization (Assumption 1) and a structured class:

Contextual Unit Token ( $Z^c_u$ ): A tokenizer $T_\phi$ produces a learner-side representation from factual evidence $O$ and context $c$ . This token mediates all paths from the unit to the prediction.
Shared Response-Law Form ( $R_\theta$ ): All units share a single response mechanism $R_\theta$ that reads the token. The structured class requires that $R_\theta$ depends on the token $z$ only through a declared low-complexity map (e.g., a linear predictor).

The learned object is the pair $(T_\phi, R_\theta)$ . The tokenizer organizes unit-specific regularities into a simple token space, allowing the shared form to generalize.

Access Modes

The paper distinguishes two modes of accessing the unit:

Direct Unit Access: A trusted resolver uses a key $k$ to fix the referent $u(k)$ and supply a lookup token (e.g., a learned embedding $z_\theta(k)$ ). This removes "which-unit" uncertainty but leaves token-and-form approximation error.
Unit Abduction: When no resolver identifies the unit, a token is formed from factual evidence $O$ via $Z^c = T_\phi(O, c)$ . This token serves as a contextual representation, which may be interpreted as an approximate identity posterior in specific specializations (e.g., identity-mixture learners).

Key Contributions

Formalization of the Unit: The paper formalizes the unit as a task-declared population primitive, separating the persistent referent from the event record. It clarifies that a sample records an event, while the task declares the referent to which events belong.
Shared-Form Interface: It establishes a shared-form interface (Assumption 1) where the structured class is a simple relation in the token. The running instance is a linear predictor (Definition 2), where the learned object is the pair $(T_\phi, R_\theta)$ .
Decomposition of Predictive Value: The paper separates oracle predictive value, evidence accessibility, learner approximation, and component identifiability. It establishes that row-level marginal fit does not identify the internal unit structure.
Impossibility and Separation Results: It proves that unlinked single-row observations cannot distinguish a heterogeneous unit world from a homogeneous pooled world. Conversely, trusted same-unit pairs (linked observations) can separate these worlds by exposing within-unit covariance.

Formal Results and Theoretical Bounds

Value–Access Decomposition

Under log loss, the paper decomposes the oracle predictive value of unit information ( $V_{oracle}$ ) into accessible value ( $V_{accessible}$ ) and residual value ( $V_{residual}$ ):
$R^*_W - R^*_{U,W} = (R^*_W - R^*_{O,W}) + (R^*_{O,W} - R^*_{U,W})$
where $R^*_W$ is the risk without unit info, $R^*_{O,W}$ is the risk with evidence $O$ , and $R^*_{U,W}$ is the oracle risk with perfect unit knowledge. The residual term represents information about the unit that is predictively valuable but inaccessible given the declared evidence.

End-to-End Error Bounds

The deployed predictive error is bounded by the sum of unit-belief error and fixed-unit response error. Specifically, the Kullback-Leibler divergence between the true marginal and the learned marginal is bounded by:
$E_{pred} \leq E_O [ D_{KL}(P(\cdot|O) \| Q_\phi(\cdot|O)) ] + E_{O,W} \left[ \int D_{KL}(K_{u,W} \| \hat{K}_{u,W}) P(du|O) \right]$
This separates the error arising from imperfect unit identification (belief error) from the error in modeling the response law for a fixed unit.

Single-Row Impossibility

Proposition 4 demonstrates that in an unrestricted response-kernel class, every sample-only law $R(dy|x)$ has a unit-constant representation $K^R_u(dy|x) = R(dy|x)$ . Consequently, marginal fit alone cannot identify whether the world is homogeneous or heterogeneous.

Indistinguishability: If observations come from fresh, unlinked units, a heterogeneous world (where units have different response probabilities) is statistically indistinguishable from a homogeneous world (where all units share the same probability) based on single-row marginals.
Separation: Trusted same-unit pairs (repeated linkage) allow the identification of the variance of the unit-specific parameters ( $Var(p_U)$ ) via within-unit covariance, separating the two worlds.

Evaluation Implications

The paper derives corollaries for evaluation protocols:

Weighting: Row-weighted and unit-weighted empirical risks agree only if all observed units have equal record multiplicity.
Splitting: A record-wise split mixes new-event prediction for known units with prediction for unseen units. To target generalization to unseen units, an explicitly unit-disjoint split is required.

Significance and Scope

The paper claims that machine learning learns shared structure across task-declared units from noisy, selectively observed, unit-linked events. Its primary contribution is a semantic and computational framework that explicitly handles the persistent referent.

Modest Claims: The author emphasizes that Assumption 1 is a computational interface, not a learnability theorem. The paper does not prove that a specific learning rule drives approximation penalties to zero; rather, it defines the conditions under which a shared form is simple enough to generalize.
Distinction from Existing Work: The framework does not replace classical statistical learning or causal inference but adds an explicit question: how a tokenizer organizes units so that one shared response-law form generalizes. It distinguishes unit variation from environment variation (domain adaptation) and clarifies that "unit abduction" (forming a token from evidence) is distinct from classical causal abduction (inferring exogenous noise).
Practical Implications: The formal results indicate that unlinked single-row observations cannot validate unit structure. Trusted same-unit linkage is required to separate heterogeneous from homogeneous worlds. Furthermore, evaluation must distinguish between row-level and unit-level estimands, and between known-unit and unseen-unit generalization.

The paper concludes that when preserving the unit changes none of a declared target, estimand, or evaluation protocol, the unit-marginal formulation is sufficient. However, when persistence, attribution, or unit-level evaluation matters, the unit must be declared as a primitive to correctly structure the learning problem.