🤖 AI

Toward Personal Intelligence Through Cooperative Observation

This paper proposes "cooperative observation" as a framework for personal AI, arguing that effective assistance relies on a feedback loop where users grant or restrict access based on the system's demonstrated usefulness and trustworthiness, a concept illustrated through a six-month prototype evaluation.

Original authors: Yashar Talebirad, Osman Jime, Ali Parsaee, Eden Redman, Yongbin Kim, Osmar R. Zaiane

Published 2026-08-19

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Yashar Talebirad, Osman Jime, Ali Parsaee, Eden Redman, Yongbin Kim, Osmar R. Zaiane

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a personal assistant that knows you not just by the tasks you give it, but by the rhythm of your life. For decades, the dream of artificial intelligence has been to build a machine that understands human goals, habits, and commitments well enough to act on our behalf. But there is a fundamental problem: a computer cannot see inside a person's mind or know what they truly value unless that person tells it. The quality of any personal AI is limited by what it can observe. If the system only sees a list of to-do items, it will plan poorly. If it sees a continuous stream of health data, calendar events, and voice notes, it might plan better, but only if it is allowed to see them. The central question for researchers is no longer just how to make smarter software, but how to build a relationship where the user is willing to share enough information to make the software useful, without feeling exposed or controlled.

This is the territory explored by a team of researchers at the University of Alberta and MacEwan University in a paper presented at the KDD 2026 conference. They propose a new way of thinking about personal AI called "cooperative observation." The core idea is simple yet profound: the flow of information between a person and their AI should not be a one-way street where the machine takes data, nor a static setup where the user gives everything at the start. Instead, it should be a feedback loop. The system tries to help based on what it knows; the user judges whether that help was actually useful; and based on that judgment, the user decides whether to let the system see more, less, or something different next time. In this framework, trust is not a given; it is earned through useful actions, and the right to control what is observed is the foundation of the relationship.

To test this idea, the researchers built a prototype system called Organizm. Unlike many commercial assistants that live on company servers and collect data for advertising or engagement metrics, Organizm was designed to be owned entirely by the user. It runs on the user's own files, keeping all memories, logs, and plans in a personal digital folder that the user can inspect, edit, or delete at any time. The system is structured like a small team of specialized agents. One agent might handle immediate notes, another might look at long-term goals, and a third might manage daily schedules. These agents do not all try to read every file at once. Instead, they work in layers, starting with broad summaries and only diving into specific details when a task requires it. This design mimics how a human brain manages information, focusing on what is relevant to the current moment while keeping a broader context available for reflection.

The researchers put this system to the test in a real-world setting, tracking its use by a single individual over six months, from January to June 2026. This was not a controlled laboratory experiment with dozens of participants, but a deep dive into how one person's relationship with their AI evolved over time. At the beginning, the user interacted with the system mostly through manual reports, typing in tasks and goals. As the weeks passed, the user began to trust the system enough to connect it to more sources of information. First, they linked their calendar so the AI could see scheduled events. Later, they added a shared registry for deadlines and even a channel for tracking finances. The system did not force these connections; the user added them voluntarily because the previous interactions had proven useful.

What the researchers found was a clear pattern of cooperation. When the system provided a plan that helped the user manage their time or catch a conflict, the user was more likely to share new information or correct the system's mistakes. For example, when the system initially missed a recurring expense in a budget plan, the user corrected it, and the system learned from that error to avoid it in the future. When the system suggested a daily schedule that was too ambitious, the user pushed back, and the system adjusted its planning style to be more realistic. This cycle of action, evaluation, and adjustment is what the authors call the cooperative feedback loop. The user's willingness to expand the observation channel—allowing the system to see more of their life—was directly tied to the system's ability to provide genuine value without overstepping.

The study also highlighted the importance of how information is handled. Because the system was built on user-owned files, the user could see exactly what the AI was remembering and how it was using that memory. This transparency was crucial. When the system made a mistake, the user could trace it back to a specific file or a misinterpreted note. This level of inspectability built the trust necessary for the user to share sensitive data, such as financial records or health-related goals. The researchers noted that if the system had been a "black box" where the user could not see the inner workings, the user likely would not have expanded the data channels in the same way. The ability to revoke access or delete specific memories gave the user a sense of control that made them comfortable with the system's growing knowledge.

However, the researchers are careful not to claim that this is a solved problem. The six-month study involved only one person, and while the results are encouraging, they cannot prove that this approach will work for everyone. The user in the study was an author of the paper, which means they were highly motivated and technically skilled, potentially making them more willing to engage with the system than the average person. The researchers acknowledge that sparse or inconsistent feedback from other users might make it harder for the system to learn what to observe next. They also point out that as observation channels become richer—potentially including continuous audio, video, or even neural signals in the future—the risks to privacy and autonomy will increase. The system might infer things about a person's health or mental state that the person never intended to share, and the consequences of such inferences could be significant.

The paper suggests that the future of personal AI depends on getting this balance right. It argues against the idea that more data is always better. A system that captures everything but cannot be trusted or controlled will likely be rejected by users. Instead, the path forward lies in systems that are designed to be cooperative from the start. This means the software must be built to respect the user's control over what is observed, to explain its reasoning clearly, and to adapt its behavior based on the user's feedback. The goal is not to create a machine that knows everything about a person, but to create a partner that knows just enough to be helpful, and that earns the right to know more through consistent, useful action.

The researchers propose several ways to test this idea further. They suggest running studies where the amount of information the system can see is deliberately varied to see how it affects the quality of assistance. They also want to track how users change their sharing habits over time in response to the system's performance. Another key area of study is how the system's understanding of the user changes the user's own understanding of themselves. Does having a clear record of past actions and goals help people make better decisions? The paper leaves these questions open, inviting further research to determine if cooperative observation can truly become the standard for personal intelligence.

In the end, the work presented in this paper is a call for a different kind of relationship between humans and machines. It moves away from the idea of the AI as a tool that extracts data and toward the idea of the AI as a partner that earns access through usefulness. The prototype, Organizm, showed that when a user feels in control and sees real value, they are willing to let the system into their life. But this relationship is fragile; it depends on the system's ability to remain transparent, to respect privacy, and to prioritize the user's goals over its own. As technology continues to advance, with devices becoming more capable of capturing our lives in real time, the principles of cooperative observation offer a roadmap for ensuring that these powerful tools serve the people who own them, rather than the other way around. The success of personal AI may not depend on how smart the machine becomes, but on how well it learns to cooperate with the human it is meant to serve.

Technical Summary: Toward Personal Intelligence Through Cooperative Observation

Problem Statement

Personal AI systems require a model of the user's goals, constraints, and ongoing commitments to plan and act effectively. However, the quality of this model is fundamentally bounded by the system's ability to observe the user's life. Current developments in sensing hardware (wearables, smart glasses, neural interfaces) and agent software are approaching this "observation bottleneck" from opposite sides: agents require richer context to act personally, while new devices capture context that remains useless without an agent capable of acting on it.

The core problem is that richer observation does not automatically improve assistance. A bounded system must select and compress information for specific tasks. Furthermore, existing modeling systems often serve platform objectives (engagement, revenue) rather than user goals, leading to surveillance and manipulation. In personal AI, the user must define success and control data access, yet most personalization systems split the roles of state modeling, action evaluation, and observation control across different parties. The paper identifies a need for a framework where the observation channel is governed by the user and shaped by a feedback loop of usefulness and trust.

Methodology and Framework

The paper proposes Cooperative Observation, a framework for user-governed personal intelligence. This framework treats the observation process not as a fixed input stream, but as a dynamic, cooperative feedback loop involving three components:

A One-Step Decision Model:
- State ( $S_t$ ): The user's evolving state, including goals, constraints, commitments, and internal variables (motivation, overload).
- Observation ( $O_t$ ): Data received from a channel configuration ( $\Phi_t$ ) controlled by the user.
- Action ( $A_t$ ): The system's output, selected via a policy $\pi$ based on information history $H_t$ .
- Reward ( $R_t$ ) and Reflective Benefit ( $U_t$ ): Immediate user feedback versus long-term assessment of whether the action was beneficial. The framework distinguishes between optimizing for immediate approval (which can lead to Goodhart's law) and long-term reflective benefit.
The Observation Ceiling:
Using the data processing inequality, the authors establish that state-dependent personalization ( $I(S_t; A_t)$ ) cannot exceed the information the history carries about the user ( $I(S_t; H_t)$ ). Expanding the observation channel raises this upper bound, but the system must still compress information for specific tasks.
The Cooperative Feedback Loop:
The user evaluates the system's actions and subsequently adjusts the channel configuration ( $\Phi_{t+1}$ ). If the system provides useful assistance, the user may expand the channel; if it fails or violates trust, the user may narrow, correct, or revoke access. This loop is defined as "cooperative" only when changes in access improve the user's reflective benefit and remain under user control.

Observation Channel Spectrum:
The paper outlines a spectrum of observation fidelity, ranging from:

Phase 1: Manual reports (deliberate, low bandwidth).
Phase 2: Automated behavioral context (calendars, health data).
Phase 3: Continuous multimodal capture (audio-visual streams).
Phase 4: Neural or near-neural interfaces (direct measurement of neural activity).
Moving toward later phases increases information availability but raises significant privacy, consent, and autonomy risks.

Prototype: Organizm
To illustrate the framework, the authors present Organizm, a prototype system used over six months. Key architectural features include:

File-Native Memory: All data (logs, notes, summaries) is stored as ordinary files in a user-owned hierarchy, ensuring local control and inspectability.
Coupled Agent and Information Hierarchies: The system uses a "society of mind" approach where specialized agent personas are aligned with information tiers (e.g., immediate note processing vs. long-term strategy). Agents use index-first traversal to select task-appropriate context, compressing history for specific decisions.
Feedback-Driven Planning: The system updates plans based on user corrections and comparisons between planned vs. actual activity, treating self-report as a recurring observation channel.

Results

The paper reports a preliminary single-subject account of the Organizm prototype used from January to June 2026.

Observation Channel Evolution: Over six months, the user voluntarily expanded the observation channel within Phases 1 and 2. This began with manual logs, expanded to include calendar events, a shared deadline registry, and a finance channel loaded by a specialized agent.
Reporting Frequency: Manual logging frequency varied initially but stabilized near daily by May and June (averaging ~25 logs/month).
User Corrections: The user actively corrected the system's model, such as fixing a cost model that omitted recurring expenses or revising daily plans based on calendar conflicts. These corrections updated the memory and future planning via the feedback loop.
Limitations of Results: The authors explicitly state that this single-subject record is illustrative. It cannot identify causal effects of the feedback loop, nor can it support generalization. The changes observed (increased reporting, channel expansion) are consistent with the cooperative observation hypothesis but require controlled multi-user studies to verify.

Key Contributions

Cooperative Observation Framework: A formal proposal for personal AI where the observation channel is dynamically adjusted by the user based on the system's usefulness and trustworthiness, rather than being fixed or platform-controlled.
Theoretical Bound: Application of the data processing inequality to personal AI, demonstrating that the quality of assistance is bounded by the information content of the observation history.
Task-Conditioned Context Allocation: The identification of the problem of selecting and compressing user information for specific decisions under processing and disclosure constraints, proposing a hierarchical approach to manage this.
Prototype Implementation: The design and preliminary deployment of Organizm, demonstrating a "local-first" architecture that couples agent hierarchies with user-owned file hierarchies to support the cooperative loop.
Evaluation Directions: A proposal for three specific studies to validate the framework:
- Observation-channel ablation: Testing if task-relevant context improves assistance with a fixed model.
- Longitudinal deployment: Tracking if assistance quality affects what users share, correct, or revoke.
- Complementary self-model evaluation: Measuring changes in both system knowledge and user self-knowledge.

Significance and Claims

The paper claims that cooperative observation is a necessary structural condition for personal intelligence. It argues that without user control over the observation channel and the objective function, richer sensing will only deepen surveillance and dependency rather than personal assistance.

The significance of the work lies in shifting the paradigm from "more data is better" to "better cooperation is better." It posits that useful and inspectable behavior gives users a reason to maintain or expand observation channels, while failures lead to correction or revocation. The paper emphasizes that for personal AI to succeed, the system must operate under partial observability where the user is simultaneously the subject of the state model, the evaluator of actions, and the controller of the observation channel.

The authors remain modest about their claims, noting that the current work is a framework and a preliminary prototype. They explicitly call for controlled multi-user studies to test the causal mechanisms of the feedback loop and to evaluate the behavioral effects of their coupled agent/memory hierarchies. They also highlight safety concerns, particularly regarding the integrity of observation streams, the distinction between permission to collect data and permission to infer latent attributes (especially in neural interfaces), and the need for revocable, portable, and auditable system architectures.